Re: [I18ndir] draft-bray-unichars-01

[Asmus Freytag <[e-mail]>]

On 8/29/2023 5:05 PM, Tim Bray wrote:
> Asmus, these are great! Thanks. If this document goes forward I think 
> it will include almost all of what you suggest.
I would be thrilled. I think it's useful to move this forward (with 
changes).
>
> My favorite input is (7) - Neither co-author is in love with “Basic”, 
> a better suggestion would be greeted with joy.

"Minimally restrictive" describes what you are aiming for. You are 
restricting everything that has no business being interchanged and no more.'

Something like this then sets up the idea that there are other scenarios 
where the likely direction is "more restrictive", except perhaps some 
protocols that do assign meaning to certain controls and therefore rely 
on them -- we all think that these should be the exception; in other 
words, if you picked something that indicates an endpoint (like 
"minimally") the nice feature would be that most other specifications 
would tend to subset.

For example, by restricting private use.

A./

PS: you might think about

(8)

Suggest a standard methodology for specifying subsets other than the 
"XML" or "minimally restrictive" ones / or at least discuss some 
concepts around this area, or point to specs in and out of ITETF that 
create limited subsets for various purposes.

There's always a tension of defining data formats that don't impose 
practical restrictions on the text content and situation where general 
text is not what you need (but pure ASCII strings don't cut it either).


>
> On Aug 29, 2023 at 12:10:42 PM, Asmus Freytag <[e-mail]> 
> wrote:
>> Comments on the draft (in no particular order or priority).
>>
>> (1)
>> "Since the C0 controls include zero and the 32 smallest integers, 
>> they are likely to occur in data as a result of programming errors."
>>
>> In the way the Unicode standard refers to characters "the control 
>> codes" are the characters, so "the C0 control codes include" would be 
>> read that way. What you are referring to here are the integer values, 
>> corresponding to a code point. Suggest rewording because "zero" as a 
>> digit is not a control code and while it's possible to figure out 
>> what must be meant, it's a needless ambiguity.
>>
>> How about "Since the code points for C0 controls include the 32 
>> smallest integers including zero..."
>>
>> (2)
>> I recommend thinking not of "problematic code points" but of 
>> "problematic code point types" as per this definition from the 
>> Unicode Glossary. /
>> /
>>
>>     /Code Point Type
>>     <https://www.unicode.org/glossary/#code_point_type>/. Any of the
>>     seven fundamental classes of code points in the standard:
>>     Graphic, Format, Control, Private-Use, Surrogate, Noncharacter,
>>     Reserved. (See definition D10a in Section 3.4, Characters and
>>     Encoding <https://www.unicode.org/versions/latest/ch03.pdf#G2212>.)
>>
>> This would let you also address the private-use problematic in 
>> interchange (even if you conclude they might be useful enough not to 
>> restrict them by default, they suffer from the same lack of consensus 
>> interpretation as the controls).
>>
>> (3)
>>
>> "reserved for internal use" should be "reserved for internal use by 
>> application" (as opposed to internal use by the standard).
>>
>>
>> (4)
>> This subset has the advantage of excluding surrogates, which can 
>> never add any value and have the potential to cause problems.
>>
>> should be reworded a bit to:
>>
>> "This subset has the advantage of excluding surrogates, which are not 
>> assigned to any characters, and thus can never add any value.
>> They have the potential to cause problems, for example it is not 
>> possible to represent them individually in UTF-8."
>>
>> Rationale for this suggestion is to be slightly more specific, so the 
>> reader comes away with the conclusion that the "can never add any 
>> value" is based on well-founded reasons and not editorial opinion by 
>> the writers.
>>
>> (5)
>> The ABNF in section 4.1: the comments are confusing.
>>
>> (6)
>> I would like to refer you to table 2-3 in the Unicode Standard. The 
>> "Basic Unicode Characters" that you propagate consist of what Unicode 
>> considers "Assigned" code points plus "Reserved", with the 
>> modification that you are subtracting the "useless controls".
>>
>> Reserved code points are sometimes referred to as "assignable code 
>> points" (for example on the bottom of page 30 in Unicode 15.0.0). 
>> That makes that subset the combination of "assigned" plus 
>> "assignable" code points. (Which then, modulo "useless" controls, 
>> corresponds to the bulk of the basic set.)
>>
>> I would suggest to explicitly relate your definition to those terms.
>>
>> (6)
>> The definition "useful controls" is currently buried in the text and 
>> there's not even a header to locate the definition. Because this (or 
>> the complementary definition of "useless" ones) is the only value 
>> added piece over "assigned + assignable", I suggest elevating the 
>> definition.
>>
>> I'm not going to quibble about the very opinionated naming of the 
>> concept.
>>
>> (7)
>> The term "Basic" is an interesting choice. Because the set is 
>> anything but "basic" -- it includes all code points that can be 
>> maximally assigned, except for 61 C0 / C1 Controls. The only part of 
>> the set that is "basic" is in fact the subset of control codes.
>>
>> What you have defined is the "maximally useful set of Unicode code 
>> points for data interchange, absent a protocol defining specific 
>> control code semantics".
>>
>>
>> Hope you find some of these comments useful,
>> A./
>>
>>
>>
>>
>>
>>
>>
>>
>> On 8/29/2023 9:37 AM, Tim Bray wrote:
>>> Hello I18ndir (anyone still here?), Paul Hoffman and I just 
>>> submitted draft-bray-unichars-01 - our AD Francesca Palombini 
>>> suggested we notify this list: 
>>> https://datatracker.ietf.org/doc/draft-bray-unichars/
>>>
>>> This draft fell out of a conversation originally provoked by this 
>>> errata report: https://www.rfc-editor.org/errata/eid7603 
>>> <https://www.rfc-editor.org/errata/eid7603>
>>>
>>> It revealed a distressing lack of consensus about Unicode characters 
>>> and code points and character repertoires. I feel personally bad 
>>> because I am the editor of a couple of RFCs that are open to 
>>> criticism on this front.
>>>
>>> So, this tries to say “here’s how an RFC should specify which 
>>> Unicode characters it supports”.  We think this would be useful to 
>>> multiple groups almost immediately, including especially those who 
>>> don’t realize the area can be problematic.
>>>
>>> Anyhow, the purpose of this note is to ask your advice on how to 
>>> take this forward. We think this works well as an individual 
>>> submission.   We don’t *think* it needs a working group, but that’s 
>>> not our call. The draft doesn’t express any opinions about best 
>>> practices, it just points out several alternative character 
>>> repertoires, provides ABNF, and discusses their trade-offs.  None of 
>>> them are wrong.
>>>
>>