Re: [Alldispatch] Proposed discussion topic for ALLDISPATCH: draft-bray-unichars

[Peter Saint-Andre <[e-mail]>]

I'm late to this party, so I can't reply in the existing thread. Sorry!

In a previous message [1] in the existing thread, Tim Bray made what to 
me is a surprising claim:

###

In the context of this discussion, perhaps the notable thing about the
Unichars draft is that it’s the first (AFAIK) attempt to seriously 
consider all the flavors of code points, with plenty of community input, 
and identify the ones that nobody should really ever use, and to provide 
ABNF to support the result of that research.

###

ABNF aside, I would posit that we have a long history of considering all 
the flavors of Unicode code points and identifying which ones should and 
should not be used for various purposes in IETF protocols. That was the 
whole point of the IETF's work on IDNA2003, IDNA2008, EAI, PRECIS, etc.

More specifically, in RFC 8264 (the PRECIS framework) we defined two 
string classes for general use in IETF protocols based on years of 
experience by various "customers" of Stringprep (see Appendix B of RFC 
6885 for a list of the existing protocols whose needs were considered in 
developing PRECIS):

1. IdentifierClass specifies a smaller subset of Unicode characters for 
use when safety and identifier stability are more important than 
expressiveness.

2. FreeformClass specifies a larger subset of Unicode characters for use 
when expressiveness is more important than safety and stability.

We also provided mechanisms for profiling those string classes into more 
restrictive subsets (to meet the needs of specific protocols), and for 
defining new string classes if the PRECIS classes don't meet people's needs.

In defining these classes the PRECIS framework (following on IDNA2008) 
considered over a dozen different categories of Unicode code points, 
based on the Unicode properties of those code points, in much greater 
detail than draft-bray-unichars does (see RFC 8264 for details).

Also surprising to me is that draft-bray-unichars does not discuss a 
number of thorny issues related to the use of Unicode code points in 
text fields, including but not limited to visually similar characters, 
directionality, string comparison, normalization, width mapping, and 
case mapping.

At the very least, it seems to me that draft-bray-unichars needs to make 
a strong argument for why we should consider the PRECIS string classes 
to be wrong, harmful, unusable, or obsolete.

Peter

[1] 
https://mailarchive.ietf.org/arch/msg/alldispatch/-OTmPk46W4wkIVvjCkB2IMfYElc/