Re: [Idna-update] IDNA and combining sequences

["Asmus Freytag (c)" <[e-mail]>]

Patrik asked, where do we go from here.

Let me propose something, but first some response to John.

On 3/10/2018 5:51 AM, John C Klensin wrote:
>
> --On Saturday, March 10, 2018 09:06 -0400 Patrik Fältström
> <[e-mail]> wrote:
>
>> I think we should do some scenario planning here. Remember
>> that we do not have a world where IDNA2008 based on Unicode
>> 6.x is what people use. People use all different kind of mix
>> between IDNA2008, IDNA2003 and Unicode versions. I have myself
>> worked with the curl library (that uses libidn) and to be
>> honest, I do not think people KNOW what they use. Or they
>> know, and they know they violate the rules. For the contracted
>> parties, they have the LGR coming down the road anyways, so...

Correct, and it doesn't stop there: some parties are off the reservation 
doing emoji...

>>
>> And *if* we go down this path, is it "enough" to do in the LGR
>> (i.e. ICANN) or should IETF do some adoption (and W3C?), or
>> should IETF say "we can move forward in a more safe way AS WE
>> KNOW ICANN DO LGR"...
> As I said at far more length in another note, the LGR is (at
> least by its charter/ Procedure) designed for, and applicable
> only to, the root.  While those code points would probably be
> safe to use in any zone, trying to apply them globally would be
> unduly restrictive, would violate the "administratively
> distributed hierarchy" principle, and, IMO, just wouldn't fly..
> The latter would probably quickly reach the point that attempts
> to apply the LGR to labels at the second level and beyond would
> encourage non-compliance and daring ICANN to do anything about
> it.

Totally agree. There's no benefit in "imposing" the RZ-LGR on other zones.

However, the various script LGRs can serve both as examples and as starting
points. They do contain (or are about to contain) worked examples of
repertoire context rules and or variant rules applicable to modern users
of each script. They are also heavily documented, so that anybody can
understand the "why" behind their design. That makes them useful as
a starting point as well, to which you can add features as needed. One
thing almost anyone will want to add outside the root zone is digits
(and the hyphen).

That's why I have consistently argued that the RZ-LGR is a useful example
and starting point.

>
> As an example of the overly restrictive part, I assume that the
> LGR rules, like the ccTLD Fast Track Procedures, would prohibit
> the use of "ур" (U+0443 U+0440) or other permutations of those
> characters) in to root on the grounds of either intrinsic
> confusability or the potential conflict with the 3166-1 alpha-2
> list.

The LGR does not have to "prohibit" these. Instead, they are recognized as
so-called cross-script variants of the Latin letter.

Meaning, if you have a label containing Latin "yp" in the root, then you
do not get to use Cyrillic "ур"in some other label, if the two would 
otherwise
be (or look) the same.

However, you remain free to use Cyrillic "ур" in any label that does not
collide with some Latin label, either because no homograph label exists,
or because yours has some specific Cyrillic code point in it that would
distinguish it.

The ability to use the variant mechanism for such cases is  the main
difference between using LGRs (aka idn tables) compared to the protocol
in enforcing restrictions.

The cross-script variant mechanism is a more appropriate method to address
issues like that, because it doesn't blindly ban perfectly acceptable 
uses of code
points, but instead resolves any conflicts in favor of the first mover.

This reflects linguistic reality - in most cases, cross-script labels 
that are
homographs would look more than a bit "contrived", such the licence
plate with the Russian expletive AXY HEXO that some Unicoder had for
a while - it makes no sense in English, and passed the DMV.

At one point, we did some research in one of the scripts and found
that defining even a substantial numbers of such variants had little
impact on blocking legitimate words. Had a similar number of code
points been banned, the utility of the IDN labels would have been
seriously compromised.

For that reason alone, simply banning code points is more restrictive
than necessary.

> However, if we accept the logic that justifies IDN TLDs,
> that sequuence would be acceptable in a subtree of a domain
> whose TLD label was in Cyrillic and that published and applied a
> policy that its subdomains were entirely Cyrillic.

There are other RZ restrictions (no digits, no hyphen) that aren't
appropriate for zones lower down the tree.

And there are a few cases where it wasn't possible to cater to
all languages in a given script simultaneously, as some had
conflicting conventions (for complex scripts).

Such cases, while rare, are another good reason to use the Root Zone
as a starting point (and not the end-point) of LGR design for other Zones.

However,  the audience of 90%+ of existing zones would have been served
equally well had their idntables and policies been based on nothing more
than the RZ-LGR plus digits and hyphen. The stuff that the Root Zone
restricts is very often simply not in use.


>
>> Or...
> I think the above puts us well into the "or..." range.
>

I think we've come to that conclusion before - and we keep revisiting
ground that I thought had somewhat settled. We had been working
on a two-prong approach, of which your ID was one of the prongs:

1) to reiterate that the "raw" protocol doesn't by itself design
useful and secure LGRs (aka. idn tables/policies), but requires
conscious selection (and other decisions).

2) to give guidance as to what the issues are and how to address
them.

Now, we thought that the best approach for the second prong was
to create a registry of "troublesome" code points.

That turned out to be more challenging than anticipated. Not
because it is difficult to come up with a list. That's actually not
as difficult as it looks - for modern scripts.

No, the issue is that the code points, like your example of "yp"
are not problematic in isolation. What then should get listed? If we list
just the Cyrillic code points, then people creating a Cyrillic zone
will complain loudly that the entry is unjustified. If we list both the
Latin and the Cyrillic code point, on the basis that their interaction
is problematic, we get that reaction from two groups.

However, in the RZ effort, both groups were perfectly happy
defining mutually blocked variants - in fact, we had to stop
them from going overboard. . . .

Where to we need to go?

We need some recommendation that goes beyond "know what
you are doing" (while true, it's too vague to be helpful). We
need a set of recommendations that is more or less like the
example I wrote down for the combining marks. Something
that's specific enough that people can act on it, but also can
be adjusted to fit circumstances.

In writing recommendations for (more) "secure" IDNs, it's
important to make a distinction between modern-use scripts
and the vast number of historical scripts and technical (phonetic)
notations. The latter are really not well suited to secure use
in public zones. (The protocol must cater for non-public zones
as well).

Now, one option would be to create something like a "profile"
on IDNA2008 for "(more) secure IDNs for public zone."

Doing that would sidestep the issue of backwards compatibility
because you could be more restrictive; however, there's
the same issue that any enumerated restrictions would
run into the same issues of changes in usage (spelling
reforms) or changes in encoding.

However, qualitative descriptions wouldn't suffer as much
from that issue; I think they would add value, whether
as recommendation or as "profile".

A./