Re: [I18ndir] draft-faltstrom-unicode and "outdated versions of Unicode" ... and the "review model" document

[Asmus Freytag <[e-mail]>]

I find the IANA tables useful myself. While I'm fully capable of 
deriving them, not having to do that is a benefit. Having them 
unpredictably "out-of-date" adds an uncertainty.

Two things should be avoided in the future:

 1. any consistent lag between the IANA tables and Unicode, say
    something like a 1-year delay (amounting to being at least one
    version off).
 2. any arbitrary "halt" to generating the tables.


As you write, the tables aren't normative because the application of the 
current state of the algorithm to the latest version of the Unicode 
Standard defines the normative data.

The way Unicode works is that it releases the Unicode Character Database 
ahead of time for beta review. It is during this time (which now will 
occur relatively predictably on a yearly pattern for major versions) 
that the review of new characters and effect of potential property 
revisions should be performed.

At the same time a set of "beta" IANA tables can be generated (draft 
update). These can be sent for review in IETF.

If no issues are found and Unicode's beta process does not result in a 
(rare) last minute correction, then the draft tables can be released 
concurrent with Unicode's version release (or as nearly concurrent as 
practical).

If issues are found, the process of adding an exception can be started 
before final release of the Unicode version. If the process of reaching 
consensus in IETF does converge in a timely manner (either before 
Unicode's final release or in a short time window), only the IANA tables 
including the exception should be posted.

However, if the consensus on an exception does not converge, the 
unmodified tables should be posted; while this would make visible the 
"retroactive" nature of applying an exception it only documents the 
normative contents of IDNA2008: as soon as the Unicode version is 
public, the properties can be calculated, and until the RFC with the new 
exception is published, the original, unmodified calculation is what is 
defined normatively by IDNA2008. Therefore, as far as normative content 
of IDNA2008 is concerned, any exception declared after a code point was 
encoded is "retroactive". (The same applies even more so for exceptions 
based on property changes). Therefore, normatively, it doesn't help to 
hold updates of these tables.

A./

PS: freezing any particular zone to an earlier version of Unicode is 
outside the scope of IDNA2008 or the IANA tables. However, it is a major 
pain. For the Root Zone it turns out that the only code point that might 
have been added is, of all cases U+08A1, which is completely harmless in 
the root because none of the Arabic combining marks are allowed. 
However, in terms of the tools we use to create the MSR and the RZ-LGR 
tables and documentations we definitely have "leakage" from later 
versions of Unicode. For example, the tables we generate show fonts and 
annotation data from later versions, and, while marked as "excluded", 
some tables show the presence of "future" characters.





On 3/16/2019 1:58 PM, John C Klensin wrote:
> Hi.  In the confusion of the last week, I apparently wrote one
> note that I never sent.  As we think about what needs to be said
> or clarified going forward, some of that note may be useful even
> though I think all of us are now in synch about it and hence,
> for the purposes of the directorate of
> draft-faltstrom-unicodeXX, I'm probably kicking the proverbial
> dead horse.
>
> --On Sunday, March 10, 2019 13:35 -0700 Asmus Freytag
> <[e-mail]> wrote:
>
>> ...
>> "The Unicode Standard is being updated at least yearly and
>> implementors and vendors of OS, libraries and other tools are
>> aggressively updating their systems/libraries to the new
>> versions of
>> Unicode. Therefore, IETF recognizes the burden on implementers
>> to support IDNA2008 corresponding to an outdated version of
>> Unicode.
> It is probably worth having tables as part of an IANA registry.
> It is probably worth keeping them up to date (see below about
> both of those statements). But those tables are not normative
> and do not define a particular version of Unicode for use with
> IDNA2008.  The requirement on implementers is that they choose a
> version of Unicode and then work either directly with the
> [current] rules of IDNA2008 or derive or find a table that
> corresponds to that version of Unicode.  There is absolutely
> nothing about IDNA2008 that would require an implementer to work
> with an outdated version of Unicode of to "support IDNA2008
> corresponding with it" (whatever that means).
>
> I say "probably" above because, if the existence of the IANA
> tables going forward is to create confusion about this --
> especially if we don't get IANA to keep versions of each table
> corresponding to every version of Unicode from 5.2 forward -- I
> think it may be critical to either add more notes to the top of
> that IANA table or to get rid of the IANA tables entirely.
> That is something we should think about when we are working on
> the "review model" document because that would be a logical
> place to adjust the instructions to IANA going forward.
>
> Now, should some body with a claim to regulatory or contractual
> authority over some collection of zones say "you are allowed to
> only use Unicode version X" or "you must use the version of
> Unicode corresponding to the IANA tables at the time of
> registration of a label", IDNA2008 does not prohibit such a
> restriction and the decision to impose it is Not An IETF
> Problem.  I would, personally, consider such a restriction
> rather dumb unless it turned out to be a requirement of some
> more elaborate system for advising registries, but, even if
> people agreed with me, that evaluation doesn't belong in
> draft-falstrom or any of the core IDNA2008 documents.
>
> I believe and hope that anything in the above that needs to be
> said in draft-faltstrom-unicode11 (or 12) has already been said.
> However, for work going forward, are we all in agreement about
> the above?
>
> best,
>     john
>
>
>      john
>
>
>