[idn] Re: stability

[Simon Josefsson <[e-mail]>]

"Adam M. Costello" <[e-mail]> writes:

> Erik van der Poel <[e-mail]> wrote:
>
>> I was referring to the RFC that has a pointer to UAX #15, i.e. RFC
>> 3454 (Stringprep).  The pointer should be updated to tracking number
>> 24 or higher.
>>
>> http://www.unicode.org/reports/tr15/tr15-24.html
>
> I agree.  Right now there does not exist a correct implementation of
> Unicode normalization anywhere.  Some implementations are wrong because
> they deviate from the spec, and other implementations are wrong because
> they adhere to the spec which itself is wrong (because it violates all
> the fundamental properties that anything called a canonical form is
> assumed to have).  It's an unfortunate situation, but the best thing to
> do now is fix the spec and encourage all implementations to converge on
> the fixed spec, which is what the Unicode Consortium is doing.  Our part
> in this is to update our pointer, so that it points at a correct spec
> rather than a wrong spec.

I'm less certain that is the best thing.  I'd like to see alternatives
explored before making such confident statement.

There appears to me be a lot of decisions made out of subjective
opinions on how normalization "should" behave, or is "assumed" to
behave.  I'd like to see things focused on what the changes actually
will mean, technically, in practice.  Or perhaps I'm simply not
clued-in on the evaluations of different approaches.

>> I believe it would be useful to start thinking of the problem in terms
>> of a transition plan from what we have today and what we would like to
>> have tomorrow.  It is not clear to me exactly what we would like to
>> have tomorrow, so settling that would have to be part of the plan as
>> well.
>
> It's clear to me what we ought to have tomorrow: a canonical form
> (that is, a function that selects a unique representative from every
> equivalence class).  The Unicode Consortium is taking care of defining
> that.
>
> Do you have any ideas for a transition to that?

I think reaching that goal can be done in different ways.

One way is to incorporate the PR-29 fix, declare the earlier attempt
as buggy, and re-cycle at PROPOSED.  I suspect you prefer that way?  I
am hesitant about that approach, because we have already deployed the
old RFC and it is not clear what problems there will be in mixing the
old and the new code.  Both Kerberos and SASL appears to be going to
use the old StringPrep as well, so we will be seeing security critical
infrastructure based on the old interpretation.

Another way is to carry on with the Unicode 3.2 NFKC even though it
breaks some human's assumptions on what "normalization" means in a
theoretic setting.  Only a select few weird cases (the PR-29 problem
sequences) will be affected.  Machines will cope, they compute an
algorithm, they don't care if the output meet some unstated invariant
or not.

A third way, which is what I am deploying, is to use the Unicode 3.2
NFKC together with a filter to reject the PR-29 problem sequences.
This is in line with the RFC's, it solves problems related to PR-29
problem sequences, and is simple to implement.

Thanks,
Simon