Re: [I18ndir] [art] Fwd: New Version Notification for draft-bray-unichars-04.txt

[Rob Sayre <[e-mail]>]

On Mon, Sep 18, 2023 at 7:05 AM Manger, James <James.H.Manger=
[e-mail]> wrote:

> Defining unicode-scalar-values, xml-chars, and useful-assignables are 3
> helpful “subsets of Unicode characters” that can be used in protocols and
> data formats.
>
> Defining unicode-code-points as though it is similar is a category error,
> however.
>
>
>
> Section 3.1 and 3.2 deliberately make %x0-10FFFF and %x0-D7FF /
> %xE000-10FFFF looks like similar repertoires, which is too misleading.
>
> UTF-8, UTF-16 and UTF-32 are only defined for the latter. There may be
> “obvious” extensions of UTF-8 (WTF-8) and UTF-32 that can cover %x0-10FFFF,
> but they are simply not widely supported in modern software, so those
> extensions are no use in an IETF standard....
>

Yes, they are very widely supported, although they may not have names, or
be exactly the same across languages. In Rust, refer to the OSString docs:
<https://doc.rust-lang.org/src/std/ffi/os_str.rs.html#17>

That's why they're out there.

Here, you've gotten into transformation formats right away, but the
document is not really about transformation formats. Section 6 would seem
to cover this: "...this sort of character-repertoire restriction, which
applies to data content, not textual representation in packaging formats."



> For understandable reasons, JSON supports both *(%x0-D7FF / %xE000-10FFFF)
> and *(%x0-FFFF) (arbitrary 16-bit data) as models for the logical strings
> it can represent.
>

ECMA-404 is clear: "JSON syntax describes a sequence of Unicode code
points." and the discrepancy between this text and RFC8259 is what
motivated this document. The document also seems to fairly clearly
recommend against using this production if you can help it.

thanks,
Rob