Re: [Idna-update] Expiration impending: <draft-klensin-idna-rfc5891bis-01.txt>

[Asmus Freytag <[e-mail]>]

On 3/8/2018 9:11 AM, John C Klensin wrote:
> One more thought about something it may be helpful to remind
> ourselves periodically.
>
> One of the notions floated when IDNs were first being discussed
> and raised a few times after that was to simply ban combining
> forms of all types.  The theory was that there was no
> entitlement to write any character form (e.g., any "word") in
> DNS labels, that those labels were all about mnemonics and
> nothing but mnemonics, and that a "no combining sequences" rule
> would eliminate most issues about normalization and grapheme
> clusters and make everything a lot easier to explain to
> implementers and others who wanted to conform but were not
> willing to make the investment to actually understand all of the
> complex issues and rules with which we are now dealing.

It would also very  nicely prevent IDNs on the entire Indian Subcontinent.

Ironically, it is Arabic, where most (all) of the combining can safely be
excluded from IDNs. This is being done for the Root Zone, for example,
as you acknowledge below.

The reason is that in Arabic, these marks are generally optional and/or
used for specific purposes in specialized text. Therefore, leaving them
out is not detrimental to the usability of IDNs and the Root Zone will
not allow them (extending the set of prohibited ones from RFC 5564)

Therefore, in the Root Zone the Unicode 7 addition of the U+08A1 would
not be an issue.
>
> If that were the rule and someone really, really, wanted a
> grapheme that could only be formed in Unicode with a combining
> sequence, it would be up to them to convince the Unicode
> Consortium that their favorite character (grapheme) needed to be
> added to Unicode as a single code point.   However hard that
> might be, it would not be our problem.

Well, we see what happens if someone does get Unicode to add a pre-
composed form: the entire process is derailed. That's what happened
with the addition of U+08A1 even though it is NOT the case that this is
a true "precomposed" form - while the same graphical elements are
involved, the result does not look identical. There is a strong similarity
of course, but despite what people read into the Unicode character
name, this is not a case of an exact homoglyph.
>
> FWIW, a "nothing but precombined characters" rule is essentially
> the recommendation for Arabic IDNs in RFC 5564 and, I
> understand, in the emerging Arabic script rules for the root
> zone.

That is because of the way these function in Arabic.

Unicode could not generally add precomposed forms of, say, Latin code
points, say a new letter with a dot above, because of normalization
stability. In the Latin script, a combining dot above and a precomposed
dot above are identical.

However, even there you have a number of combining marks that are
not considered as part of possible decompositions: they are the code
points for various (stroke) overlays and some attached extenders.

Like the Arabic combining marks, they could (and should) be disallowed
from LGRs. (The Latin LGR for the Root Zone will not allow combining
marks other than in enumerated combinations - that's something that
works for Latin, Greek, Cyrillic and practically all scripts that are not
South or South East Asian complex scripts.
>
> We didn't go down that path, not only because of impassioned
> pleas for some of the character forms that might be excluded but
> because of precisely the reassurance that arguably led to the
> non-decomposing characters thread -- assurances that no new code
> points would be added to Unicode if there were already a
> combining sequence that could reasonably substitute for it in
> the same script except under very unusual circumstances and,
> when those circumstances occurred, the new code points would
> decompose to those sequences.

Other participants in that discussion remember this claim differently.
Unicode Normalization forms C and D were never about "reasonable
substitutions" but about "exact equivalents" or "the same thing,
except for the encoding".

As there is generally no benefit in encoding another representation of
"the same thing", Unicode does not allow addition of precomposed
code points that can be decomposed into something that is the
exact equivalent.

>
> I don't suggest that we try to reverse that decision at this
> point.   I assume that, if nothing else, it would just be too
> disruptive.  However, it is worth pointing out that a "no
> combining sequences" rule would eliminate the non-decomposing
> character problem and at least a few other potential spoofing
> and related cases.   It might also be worth examining as a
> guideline or advice for registries who are interesting in
> raising the safety level of what they allow to be registered
> without having to understand the underlying issues more deeply.

A useful set of recommendation for handling combining marks safely in LGRs
would consist of:

1) in all non-complex scripts: allow only fixed enumerations of base 
code point
and combining marks. (The number of required combinations is small, even
for a sprawling script like Latin).

2) in all complex scripts (where the number of combinations is too large),
provide context rules that assure combining marks are not placed in the
wrong part of a syllable (such wrong contexts cannot be "read" by humans
and not rendered correctly by machines). The Root LGR presents suitable
examples of this for SEA scripts, Indic scripts in preparation.

3) in scripts where combining marks express optional elements (vowels, etc.)
disallow all of them. (Arabic, see Root Zone LGR for example)

4) in scripts where combining marks are used for historical/ special 
purposes
disallow those (diacritics for classical Greek, stroke overlays and 
other marks
for linguistics).

5) in LGRs supporting variants, consider mutually blocking labels that vary
only in the presence or absence of some combining diacritic; some 
diacritics
are not reliably distinguished from each other (comma below, cedilla) or 
from
an undecorated base character (forms with/without Nukta).

What is a complex script: a good approximation is whether it contains a code
point with ccc=virama, plus Thaana, which isn't really complex but has 
mandatory
vowels classed as combining marks. Effectively all South and South East 
Asian
scripts that are abugidas.

In addition, you will want recommendations that actually address the 
homoglyph
issues: there are a fair number of non-combining mark exact visual 
duplicates in
Unicode. These are not exact equivalents, because Unicode does not treat
code points that differ in script, case or digit/letter properties as 
equivalent.

A./


>
> best,
>      john
>
>
>
> _______________________________________________
> IDNA-UPDATE mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/idna-update
>