Re: [Idna-update] Expiration impending: <draft-klensin-idna-rfc5891bis-01.txt>

[John C Klensin <[e-mail]>]

--On Thursday, March 8, 2018 12:47 -0500 Andrew Sullivan
<[e-mail]> wrote:

> On Thu, Mar 08, 2018 at 12:11:48PM -0500, John C Klensin wrote:
>> If that were the rule and someone really, really, wanted a
>> grapheme that could only be formed in Unicode with a combining
>> sequence, it would be up to them to convince the Unicode
>> Consortium that their favorite character (grapheme) needed to
>> be added to Unicode as a single code point.   However hard
>> that might be, it would not be our problem.
> 
> I thought the argument at the time was not just that it was
> safe, but that some characters (in conceptual sense that
> Unicode uses it) were necessarily made of combining
> characters.  This was also I guess the reason that the two
> joiners ended up in under context rules, I think, yes?

Yes.  The joiners normally (and pre-emoji) had more to do with
what would ordinarily be considered phrase or sentence
construction than with character formation, but, yes, they were
included because of strong arguments that the use of some
scripts (at least with some languages) was impossible without
them.  The distinction is, IMO, worth making only because it
would have been plausible to prohibit combining sequences and
allow ZWJ and ZWNJ (with or without contextual conditions)anyway
== really separate issues.

> Moreover, there was the principle that we didn't want to make
> restrictions low in the tree even if we thought they were
> mostly a bad idea near the top (this was literally the
> reasoning for including hieroglyphs as I recall).

Certainly it was a key part of the reasoning for allowing
archaic scripts.  Whether hieroglyphs were the critical issue or
merely a common example probably depends on who you ask.

I should stress that I'm not at all convinced that we made the
wrong decision about combining sequences, nor that we would have
made a different decision had we had access to better and more
accurate information.   I would feel better now if we could look
back at the decision and its consequences and say "there was an
informed discussion based on full information and we decided to
do it and accept the consequential risks" -- just as we did
with, e.g., the characters that were classified as valid (PVALID
or CONTEXTx) in IDNA2008 but mapped to something or discarded in
IDNA2003.

On the other hand, the same kind of thinking  that argues for a
"troublesome characters" list (at least as I've understood it)
-- more "you SHOULD NOT use these unless you are extra-sure you
know what you are doing and what the implications might be" than
any sort of absolute prohibition -- might imply a similar
statement about combining sequences.  In other words, whether
they are prohibited by the requirement that putative labels be
in NFC form or not (and many are), one should not be using them
without really good reason.  Not a protocol prohibition, but an
extra layer of protection against edge cases and some
potentially-malicious behavior.

best,
    john