Re: [I18ndir] [art] Just uploaded draft-bray-unichars-03

[Tim Bray <[e-mail]>]

On Sep 9, 2023 at 9:58:43 AM, Steffen Nurpmeso <[e-mail]> wrote:

>
> In 2.2 i would not give the count on code point types.
> Instead i would only give the problem statement "among Unicode
> code point types .. are questionable".  This seems more generic.
>

Left to myself I’d leave it as-is, but I don’t care that much, if anyone
else agrees with Steffen I don’t mind changing it.

In 2.2.2.2 i would not say "legacy controls", and that they are
> "mostly obsolete".  ECMA-48 is very alive in at least the POSIX
> aka Linux world, for many purposes, for example terminal
> interaction.  "Likely to occur in data as a result of
> a programming error"?  Any preformatted Unix manual page will come
> with lots of CSI sequences, or backspace-based ones.
> ASCII NUL is the base of ISO C-style strings.  In fact many
> network protocols (not enough!!) still seem to use
> KEY=VALUE\0KEY=VALUE\0\0 style transports.
>

So, in section 23.1 of [UNICODE] it says "There are 65 code points set
aside in the Unicode Standard for compatibility with the C0 and C1 control
codes defined in the ISO/IEC 2022 framework.”  Some shreds of ISO 2022
practices may survive in legacy systems but I am pretty sure they are not
relevant to any protocol or data format the IETF might work on. This
clearly feels like “legacy” and “mostly obsolete” to me. (For those who
don’t know, ISO 2022 worked with the "ISO-Latin" 8859-1 code pages to shift
between them in the middle of a string. It is now forgotten for a good
reason.) I’d never heard of ECMA 48 and I am dubious that that use of
control codes would be appropriate in anything the IETF would specify these
days.

In 5.:
>
>  [JSON..] It cannot be serialized into legal UTF-8, but many
>  libraries will silently parse this and generate an ill-formed
>  UTF-8 string. Implementors must be prepared to deal with these
>  sorts of problematic code points.
>
> But RFC 3629 is very clear and says in 3. (being lengthy)
>

You are correct, but the assertion in 5. is also empirically correct, this
is exactly what many (most?) existing libraries will do. Presumably on the
basis of Postel’s law?

So even the weird JSON "string" can be made valid UTF-8, one just
> has to walk around the corner.  (Possibly.)
>

In fact, does that happen? I haven’t seen it but I suppose it is somewhat
possible.

Sorry, but _I_ do not get that JSON supports _that_ "string",
> RFC 8259, 7.:
>
>   To escape an extended character that is not in the Basic Multilingual
>   Plane, the character is represented as a 12-character sequence,
>   encoding the UTF-16 surrogate pair.
>

Once again, you are probably correct, but the caution that implementers
will have to be “prepared to deal with” this is true, because of what
implementations do.  You are correct that it would probably be better if
JSON parsers were stricter.

And then in 8.
>
>  8.  String and Character Issues
>  8.1.  Character Encoding
>     JSON text exchanged between systems that are not part of a
>     closed ecosystem MUST be encoded using UTF-8 [RFC3629].
>

As Rob Sayre said above, the proposed document probably has to address the
issue of JSON escapes and emphasize that they are not relevant to code
point subsets.

5. also says
>
>   It is unlikely that anyone specifying a new data format would
>   choose to allow this character repertoire.
>
> And
>
>   A protocol based on JSON could be made more robust and
>   implementor-friendly by requiring that the contents of member
>   names and string values contain only Useful Assignables
>
> No.  Not me.  Sorry .. we are talking string data?
> I mean, with your restriction one (possibly) cannot even generate
> a protocol that carries around Linux/POSIX path names?
>

Which characters necessary for Linux/POSIX path names are being excluded?
I went looking for the appropriate specs but ended up getting lost in
opengroup.org pages.  I note that for 25 years, the lifetime of XML, the
exclusion of control codes with the exception of \r, \t, and \n, seems to
have worked fine.

>
>