Re: [Idna-update] IDNA and combining sequences (was: Re: Expiration impending: <draft-klensin-idna-rfc5891bis-01.txt>)

["Asmus Freytag (c)" <[e-mail]>]

On 3/15/2018 12:42 PM, John Levine wrote:
> In article <[e-mail]> you write:
>>> I think I can follow them OK.  The characters each are characterized
>>> as consonant, various types of vowels, tones, and diacritics.  The
>>> ordering rules would make more sense to me as regular expressions.
>> Check out the bottom of the HTML version of each scripts LGR file.
>>
>> Click on
>>
>> https://www.icann.org/sites/default/files/lgr/lgr-2-thai-script-01jun17-en.html
>>
>> and go to WLE Rules. You should see a table of regexes.
> I was more thinking of regexes for an entire valid string, not for
> each rule, but close enough.

John, the rules are designed this way for a purpose. The reason is that 
the most common scenario is the need to prevent some Y from following 
some X, where the Y is typically a combining mark.

Sometimes, script rendering will "ligate" certain combinations, but fail 
if incompatible ones are encountered; in that case you may see right had 
contexts or both left and right contexts.

We originally thought we could start for the Indic scripts with an 
Akshar (syllable) for which there exist BNF definitions. A label would 
then be a concatenation of valid Akshars. The attempt resulted in 
horrifically complex looking rules that were impossible to review and  
tended to be overly restrictive: there are many strings that are a bit 
unusual/uncommon, but not "broken". Sometimes the available Akshar rules 
were language-specific.

For all of these reasons, but mainly to be able to break down things 
into easily understood bits, we ended up with the system we have now.

Now, given that an XML per RFC 7940 is fully machine parsable, it could 
be an exercise to the reader to compile a regex for full labels. I think 
that should be possible, however, I've not attempted to prove this - 
there may be some syntax construction that is difficult/impossible to 
translate into a regex for the whole lable. All I know is that I can 
translate each of the context rules to regexes (that's how the tool I 
wrote evaluates labels based on an LGR), but I apply them iteratively at 
each character position in the string rather than using the regex mechanism.

A./