Re: [I18n-discuss] Comments on "troublesome-characters" from Arabic script

[Asmus Freytag <[e-mail]>]

>
>       [I18n-discuss] Comments on "troublesome-characters" from Arabic
>       script
>
> Abdulaziz Al-Zoman <[e-mail]> Mon, 24 July 2017 04:42 UTC
>
> Dear Sir/Mam (i18n-discuss)
>
> Please find my comments concerning the draft-freytag-troublesome-characters-01.txt:
> “Those Troublesome Characters: A Registry of Unicode Code Points Needing Special
> Consideration When Used in Network Identifiers”
>
> As the aim of this Internet draft was to create a registry of code points that
> need special consideration when used as identifiers so that it would guide
> system administrators in setting parameters for allowable code points in an
> identifier system, and to aid applications in creating security aids for users.
> I have substantial concern on what code points should be part of this registry
> and hence how the recipients (software developer, registries, registrars, etc.)
> interpret the inclusion of a code point in the repository.

Dear Abdulaziz Al-Zoman,

The aim is indeed to identify code points that need "special 
considerations" (read: if used
indiscriminately in the context of other code points can be problematic).

This is not the same as suggesting that all the code points listed in 
the registry need to be
restricted. The vast majority of code points that ought to be restricted 
outright is already
DISALLOWED under IDNA 2008.

There are a handful of code points that should not be used because they 
have been
deprecated by Unicode (Unicode recommends that these code points not be 
used for
any purpose) but were not DISALLOWED, but these are an exception.

The majority of code points that are proposed for the registry are those 
that happen to
be identical in appearance to either another code point in the same 
script, or a sequence
of code points (usually a combining sequence).

If both code points, or both the code point and the matching combining 
sequence are
included in the repertoire of eligible code points for a zone, then 
labels can be registered
that are identical in appearance but use different code points, and thus 
are distinct, but
indistinguishable by any user.

The registry attempts to list known examples of this, and it lists both 
sides of each matching
pair, not making any value judgments as to whether one or the other is 
preferred.


>
> For example, the registry includes some essential characters (letters) that may
> result at the end useless identifiers if these characters are restricted or
> blocked (because they are part of the repository).   For instance, with
> respect to the Arabic language, the registry consists of a large portion of
> the Arabic basic alphabet that may result to a limited character set for creating
> identifiers (e.g., the Arabic language consists of 28 essential alphabet characters,
> while the repository suggests that out of these 28 characters there are 22
> troublesome-characters; i.e. more than 78% of the Arabic language characters
> can't be used if they are blocked).  This will cause impractical use of the
> Arabic language in network identifier. While the actual "troublesome-characters"
> are the non-spacing marks.

In the Arabic case, different groups at different times have come to the 
conclusion that
combining marks should not be used. The latest such group was TF-AIDN 
who concluded
that the root zone should exclude all Arabic combining marks.

If the policy for any zone follows this approach and excludes all 
combining marks, then
the remaining code points are no longer in danger of being misidentified 
for a combining
sequence or vice versa. We would say, the problem that lead to them 
being listed in the
registry has been successfully mitigated.

However, if a zone decides to allow Arabic combining marks, then other 
steps would
be required to mitigate the problem. As you note, excluding the basic 
letters of the
Arabic alphabet is not a useful approach of mitigation.

>
> Therefore,  we would suggest that the registry includes only the problematic
> code points such as non-spacing marks but not the basic characters (they can
> be in the comment field or the reason for including such a non-spacing mark).

This would make a value judgment as to which are preferred; we felt that 
while one can
more or less objectively identify which code points and sequences have 
an issue, such as
identical appearance, there is no such objective grounds for the 
preference of one code point
over another.

In the Arabic case, there are good reasons for the approach used in RFC 
5564 and also in the
Arabic Root Zone LGR. The registry already notes those combining marks 
that RFC 5564
recommends for exclusion, and provides a reference to the Root Zone LGR 
for the others.
This language could be more explicit to help users of the registry 
arrive at a better conclusion
on what form of mitigation to choose (without making an actual 
prescription).

However, even after excluding all combining marks, some code points will 
still be in need
of "special considerations" in Arabic. These include some of the digits 
and some of the basic
letters.

Again, for the general case, attempting to mitigate any issues by 
blanket exclusion of all
of these code points would severely affect the usability of the set for 
purposes of network
identifiers, and is therefore not practical. Therefore, some other means 
should be investigated.

For the digits, for any zone, like the Root Zone, where it is not 
possible to exclude one of the
two sets, the goal ought to be to prevent two labels that differ only in 
which code point is used
for some digit, but that look identical to the user. If one of these 
labels is delegated, the other
should be blocked. Making each pair of identical appearing digits 
*blocked* variants of each
other would achieve this. (Using RFC 7940 this policy can be expressed 
in a machine-readable
way).

Blocked variants thus implement "mutual exclusion" of identical looking 
identifiers; this
mitigates the issue without having to apply an a-priori preference for 
one code point over
another.

Some of the basic letters in Arabic share visual appearance, at least in 
some positional forms.
Identical labels, that differ only by using an alternate letter (but are 
indistinguishable because
of the shared forms) could be blocked from each other as discussed above.

However, this may be unsatisfactory, because of the barriers it creates 
for some user communities
of accessing network identifiers created by other communities if these 
two communities happen
to differ in their default choice of which code point to use at that 
position in the word.

A possible solution might be to allow variants that are *allocatable*, 
so that some entity is
able to request both versions of the identifier, while all other would 
be blocked from registering
a look-alike label. While this may be a suitable choice, it would be 
beyond the purposes of
the proposed repository to give any prescriptions.

> As adding basic characters to the repository  may lead to preventing their use
>   in identifiers while the origin of the problem is due to the misuse of
> non-spacing marks. Thus, the repository should cover only the non-spacing
> marks indicating their risk-free usage, and their harmful usage which should be blocked.


Arabic is a bit of a special case in that excluding all combining marks 
is a viable approach.
In the Latin script, for example, it is not possible to exclude all 
combining marks, because there
are some languages that do not have precomposed letters for all members 
of their alphabet.

In those cases, successful mitigation might consist of enumerating all 
combining sequences
known to be required for such alphabets (the list is not very long) and 
then to exclude all
other combining sequences. (Using RFC 7940 this can be done in a 
machine-readable way).

In conclusion, it appears to me that the concern is in how the facts are 
presented and a
possible lack of guidance on how to use the listed information.

A./