Re: [I18ndir] [art] Fwd: New Version Notification for draft-bray-unichars-04.txt

[Asmus Freytag <[e-mail]>]

On 9/23/2023 9:51 AM, Tim Bray wrote:
> On Sep 23, 2023 at 8:24:15 AM, Asmus Freytag <[e-mail]> wrote:
>> Identifiers benefit from a limited repertoire, often sharply limited, 
>> to aid in reliable recognition. Being mnemonic devices, there's no 
>> need to be able to represent all edge cases for all languages, as 
>> there would be for text. There are many other specifications that 
>> provide identifier repertoires, such as UAX#31, IDAN2008, 
>> domain-specific repertoires, and so on. For the purpose here, it 
>> would be useful to mention that there is a class of use cases that 
>> does need stricter repertoire limits than discussed here and that 
>> what is discussed here isn't recommended for those purposes.
>
> So, you’re suggesting a note saying that protocols can further 
> restrict repertoires on a per-field basis.  This seems so obvious that 
> I doubt it adds value, but if multiple people want it I guess it can 
> do no harm.
Yes and no. The fact that JSON decided to use code points as the 
repertoire for names means that the *need* to make a restriction is 
_not_ as obvious as you might think.

I understand one the key points of your draft to be that protocol 
designers should think through what the appropriate repertoire should be.

There are other constraints that people put on text fields (like 
formatted date, etc) but they are not usually based on the notion of a 
repertoire. Identifiers are an interesting case in that modern ones 
support fairly large repertoires, but making the repertoire too large 
doesn't make the result better. There is a tendency for people to think 
that for Unicode "the more the merrier" is always the right answer. You 
are going to the trouble of writing a draft that explains why that isn't 
so, and I'm suggesting you take a well known class of use cases to point 
out that there are occasions where even the "Unicode Assignables" are 
too unrestricted.

To me, that level discussion is different from saying "you can always be 
more selective" which contains just one bit of information but 0 
guidance or context.
>>
>> Then there's a the string data type. We don't need to define it, 
>> because Unicode defines it already, but it would be useful to put it 
>> into perspective. As defined, the string data type includes code 
>> points that can only ever be used in internal processing, and it 
>> includes ill-formed encoding forms, because they will normally occur 
>> in transient states during processing and may need to be temporarily 
>> held when receiving input (before and during verification). Those are 
>> the reasons that generic string data types are not restricted, unlike 
>> text content fields conforming to a protocol. (There may be protocols 
>> that should be defined on the basis of strings and not text content 
>> and for those, the recommendations here would not apply. And that 
>> should be explained).
>>
> I can’t see any reason to discuss string processing in programming 
> languages. The IETF is concerned with the bytes that flow across the 
> network. Our job is to document what to transmit and what to receive.

Again, my suggestion was for a discussion limited to explaining what is 
out of scope and why.

A./