Re: [Idna-update] IDNA and combining sequences (was: Re: Expiration impending: <draft-klensin-idna-rfc5891bis-01.txt>)

[Asmus Freytag <[e-mail]>]

On 3/12/2018 8:03 AM, John R Levine wrote:
>> Even though it was generally assumed that
>> a registry would not allow labels that did not conform to the
>> labels it registered, that was never a requirement that anyone
>> expected to be enforced at least up to the time those tables got
>> entangled with the new gTLD program.
>
> I've tried some experiments and it appears that gTLDs now enforce the 
> script rules.  It is certainly true that there is some grandfathered 
> garbage from before the IDNA rules, and ccTLDs do what they do.

John, (L, not K)

Restricting repertoire is only the tip of the iceberg.

Any TLD that allows SEA and/or South Asian scripts, but does not place 
limits on the context in which certain code points may occur (those code 
points mostly being the combining marks) will allow the registration of 
"garbage" labels - even though each element in the label, if taken by 
itself may be well-vetted. In those scripts, the "unit" that readers 
perceive is is not the code point, but the syllable; sequences of code 
points that do not form valid syllables are expected neither by readers 
nor by rendering engines, making them ambiguous or unreliable for 
identifier purposes. (For those used to scripts like Latin or Chinese 
were "random" strings of code points are merely "not words" but 
otherwise valid, this constraint inherent to "complex" scripts takes 
some getting used to, but it is quite real).

A cursory glance at the IDN tables published for some of these scripts 
reveals them as not necessarily vetted well enough and definitely devoid 
of any contextual restrictions.

Before RFC 7940, the only way contextual restrictions could be expressed 
was in natural language or "pseudo code". The former was done in the 
registry policy documents or the preamble to a few exceptional idn 
tables, while you can find the latter in RFC 5892 for the few context rules.

With RFC 7940, it is finally possible to capture these rules in a 
machine readable form, and if you look at the second iteration of the 
Root Zone LGR (LGR-2) you will find these in place for Thai, Khmer and 
Lao. (The next iteration will likely bring the ones for all the Indic 
scripts).

Here's a link to the human readable version of the Thai LGR:
https://www.icann.org/sites/default/files/lgr/lgr-2-thai-script-01jun17-en.html

And here's the overview document for LGR-2:
https://www.icann.org/sites/default/files/lgr/lgr-2-overview-01jun17-en.pdf

For a full description of the rationale for, say, the constraints on the 
Thai script, you'd need to follow the link to the original submission:
https://www.icann.org/en/system/files/files/proposal-thai-lgr-25may17-en.pdf

You mentioned that you were "interested" but felt that you didn't have 
the background to understand the issues in other scripts without some 
background and explanation.

I would be curious, whether you can make sense out of the description 
for Thai in the Root Zone. I also invite you to compare that LGR to some 
of the gTLD IDN tables for Thai.

I'd be interested in your conclusion,

A./