Re: [I18ndir] [art] Just uploaded draft-bray-unichars-03

[Tim Bray <[e-mail]>]

On Sep 9, 2023 at 1:56:47 PM, Rob Sayre <[e-mail]> wrote:

> 5. Refining Character Repertoires
>
> The IETF typically uses well-known data formats such as JSON, I-JSON,
> CBOR, YAML, and XML. These formats have default character repertoires. For
> example, JSON allows member names and string values to include any Unicode
> code points, including all the problematic types; the following is a legal
> JSON document:
>

I’m OK with Rob’s redraft, if only to lose the fuzzy word “typically”.
Anyone object?

{"example": "\u0000\U0089\uDEAD\u7FFFF"}
>
> The value of the "example" field contains the C0 Control NUL, the C1
> Control "CHARACTER TABULATION WITH JUSTIFICATION", an unpaired surrogate,
> and the noncharacter U+7FFFF. It is unlikely to be useful as the value of a
> text field. It cannot be serialized into legal UTF-8, but many libraries
> will silently parse this and generate an ill-formed UTF-8 string.
> Implementors must be prepared to deal with these sorts of problematic code
> points
>
> [ The first part, "unlikely to be useful as the value of a text field", is
> good. But, the next part mixes "legal" and "ill-formed", and I don't
> think that is a good idea. There is still a lowercase requirement after
> that, and I think I disagree. Implementors do not have to be "prepared to
> deal with these sorts of problematic code points". Maybe: "Some messages
> will contain these problematic code points". That is true, but you don't
> have to deal with them. ]
>

Yeah, I’m pretty sure you do, but by “deal with” I include rejecting such
input docs, e.g. by raising a well-known exception with a helpful error
message.  I guess that should be spelled out explicitly?

It is unlikely that anyone specifying a new data format would choose to
> allow this character repertoire.
>
> [ Instead: The JSON character repertoire is too permissive, so it's best
> for new specifications to require that the contents of member names and
> string values contain only Useful Assignables (see Section 4.2). ]
>

I prefer the current language. Anyone prefer Rob’s or to propose another
option?

Then, I got to the end, and noticed that "character repertoire" might not
> be the best choice. "Character encoding" or "character set"? "Vocabulary"?
> No shade for the authors here, writing about language itself is really
> difficult.
>

I personally want to stay with “repertoire” if only by extension from its
well-understood meaning when applied to encoding systems.

>
>>