Re: [I18ndir] Fwd: New Version Notification for draft-bray-unichars-04.txt

[Tim Bray <[e-mail]>]

On Sep 15, 2023 at 2:01:48 PM, Asmus Freytag <[e-mail]> wrote:

> This time I looked only at the diffs and sometimes a bit of adjacent text.
>
> This first one is not major, but a small fix would avoid a contradiction
> in terms.
>
> The numbers assigned to Unicode characters are called “code points”;
>
> This is backwards, as can be seen by "unassigned" code points. Easy to fix:
>

You are correct that there’s a problem here.  However, as we previously
established (on the art@ list only, so you probably didn’t see it), there’s
not really a “backward" and “forward". The Unicode Standard is inconsistent
on “assignment”, saying both that integer code points are assigned to
characters, and that characters are assigned to code points. See
https://mailarchive.ietf.org/arch/msg/art/3s70K1uHFguqeL8DFkIjiNfP5ZY/

The current draft talks about code points being assigned to characters and
is consistent, end to end. That’s because we felt that the abstract
character was the important thing, and the code point is a useful piece of
attached metadata.  I’m strongly of the opinion that we should be
consistent even if the Unicode standard isn’t.  So we can either address
this problem by reversing the direction of assignment end-to-end and
adopting your suggestion, or by modifying the language to make it clearer
what code points are - and in fact the definition from the Unicode standard
(which we quote later) says explicitly it's the numbers in the code space
0-0x10FFFF.  I’ll figure out which gives a better result.

> New text on dealing with problematic code points:
>
…

> In applying the recommendations of RFC19413 for text fields containing
> ill-formed UTF-8, for example, the recommendations must be applied to the
> field as a whole, not on the character or byte level. In fact, silently
> ignoring an ill-formed part of a string is a known security risk.
> Responding to that risk, [UNICODE] section 3.2 ....
>
>
Looks good to me.

> The last paragraph is overselling RFC9413, because the phrasing
> conceivably implies that it contains guidance specific to code points, when
> it is more generically concerned with problematic input. It also doesn't
> flow particularly well.
>
> You could move it at the head of Section 5, with tweak.
>
> Problematic code points are an example of problematic input. [RFC9413],
> "Maintaining Robust Protocols", provides a thorough discussion of
> error-handling options when choosing a strategy for dealing with
> problematic input. Different types of problematic code points cause
> different issues.
>
> Noncharacters....
>
> (I'm also suggesting adding a sentence to make the transition)
>
Will do something like what you suggest.

>
>