Re: [precis] Category changes with Unicode 6.3

Patrik Fältström <paf@frobbit.se> Wed, 16 October 2013 07:16 UTC

Return-Path: <paf@frobbit.se>
X-Original-To: precis@ietfa.amsl.com
Delivered-To: precis@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F291921F9D5E for <precis@ietfa.amsl.com>; Wed, 16 Oct 2013 00:16:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.3
X-Spam-Level:
X-Spam-Status: No, score=-4.3 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, GB_I_LETTER=-2, MIME_8BIT_HEADER=0.3, NO_RELAYS=-0.001]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JmBT6DwZF03W for <precis@ietfa.amsl.com>; Wed, 16 Oct 2013 00:16:15 -0700 (PDT)
Received: from mail.frobbit.se (mail.frobbit.se [IPv6:2a02:80:3ffe::176]) by ietfa.amsl.com (Postfix) with ESMTP id 0DF7A21F9D9A for <precis@ietf.org>; Wed, 16 Oct 2013 00:16:10 -0700 (PDT)
Received: from [IPv6:2001:67c:64:42:d997:fadd:90f:58bd] (unknown [IPv6:2001:67c:64:42:d997:fadd:90f:58bd]) by mail.frobbit.se (Postfix) with ESMTPSA id 70EDB219C6; Wed, 16 Oct 2013 09:16:09 +0200 (CEST)
Content-Type: multipart/signed; boundary="Apple-Mail=_C3314DB8-C154-4A26-A1EE-0E1672E12B35"; protocol="application/pgp-signature"; micalg=pgp-sha1
Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\))
From: =?iso-8859-1?Q?Patrik_F=E4ltstr=F6m?= <paf@frobbit.se>
In-Reply-To: <525E3A7F.9040204@it.aoyama.ac.jp>
Date: Wed, 16 Oct 2013 10:16:08 +0300
Message-Id: <DA8691AD-408B-409B-B6B6-3ECC048AB09A@frobbit.se>
References: <525DFB1F.8040401@it.aoyama.ac.jp> <7693B3B4-7204-48F0-8C42-EBF5D701BAF4@frobbit.se> <525E3A7F.9040204@it.aoyama.ac.jp>
To: =?iso-8859-1?Q?=22Martin_J=2E_D=FCrst=22?= <duerst@it.aoyama.ac.jp>
X-Mailer: Apple Mail (2.1510)
Cc: "idna-update@alvestrand.no" <idna-update@alvestrand.no>, "precis@ietf.org" <precis@ietf.org>
Subject: Re: [precis] Category changes with Unicode 6.3
X-BeenThere: precis@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Preparation and Comparison of Internationalized Strings <precis.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/precis>, <mailto:precis-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/precis>
List-Post: <mailto:precis@ietf.org>
List-Help: <mailto:precis-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/precis>, <mailto:precis-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 16 Oct 2013 07:16:17 -0000

Yeah, I had to do it anyways -- as I am expert reviewer of the IANA tables. New versions of the IANA tables where created by IANA the other day, and I just approved them as they match my own calculations. I.e. we have two completely independent implementations of IDNA2008 (one at IANA, one that I have -- see link below) and I compare the output of the two. If the output matches, then we are pretty sure we are correct.

   Patrik

On 16 okt 2013, at 10:04, "Martin J. Dürst" <duerst@it.aoyama.ac.jp> wrote:

> Hello Patrick,
> 
> Many thanks for checking. Great to see that everything is okay.
> 
> Regards,   Martin.
> 
> On 2013/10/16 14:40, Patrik Fältström wrote:
>> Executive summary: Does not impact IDNA2008.
>> 
>> Longer explanation:
>> 
>> On 16 okt 2013, at 05:34, Martin J. Dürst<duerst@it.aoyama.ac.jp>  wrote:
>> 
>>> Excuse me if this has been checked and/or discussed already, but I just downloaded the Unicode 6.3 version (officially published a few days ago) of http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt and found several changes in character classification:
>>> 
>>> OLD
>>> 180E;MONGOLIAN VOWEL SEPARATOR;Zs;0;WS;;;;;N;;;;;
>>> NEW
>>> 180E;MONGOLIAN VOWEL SEPARATOR;Cf;0;BN;;;;;N;;;;;
>> 
>> No change:
>> 
>> $ grep '^180E;' ../6.[23].0/allcodepoints.txt
>> ../6.2.0/allcodepoints.txt:180E;DISALLOWED;I;C;MONGOLIAN VOWEL SEPARATOR
>> ../6.3.0/allcodepoints.txt:180E;DISALLOWED;I;C;MONGOLIAN VOWEL SEPARATOR
>> 
>>> OLD
>>> 1A1B;BUGINESE VOWEL SIGN AE;Mc;0;L;;;;;N;;;;;
>>> NEW
>>> 1A1B;BUGINESE VOWEL SIGN AE;Mn;0;NSM;;;;;N;;;;;
>> 
>> No change:
>> 
>> $ grep '^1A1B;' ../6.[23].0/allcodepoints.txt
>> ../6.2.0/allcodepoints.txt:1A1B;PVALID;I;A;BUGINESE VOWEL SIGN AE
>> ../6.3.0/allcodepoints.txt:1A1B;PVALID;I;A;BUGINESE VOWEL SIGN AE
>> 
>>> OLD
>>> 2308;LEFT CEILING;Sm;0;ON;;;;;Y;;;;;
>>> 2309;RIGHT CEILING;Sm;0;ON;;;;;Y;;;;;
>>> 230A;LEFT FLOOR;Sm;0;ON;;;;;Y;;;;;
>>> 230B;RIGHT FLOOR;Sm;0;ON;;;;;Y;;;;;
>>> NEW
>>> 2308;LEFT CEILING;Ps;0;ON;;;;;Y;;;;;
>>> 2309;RIGHT CEILING;Pe;0;ON;;;;;Y;;;;;
>>> 230A;LEFT FLOOR;Ps;0;ON;;;;;Y;;;;;
>>> 230B;RIGHT FLOOR;Pe;0;ON;;;;;Y;;;;;
>> 
>> No change:
>> 
>> $ egrep '^230[89AB];' ../6.[23].0/allcodepoints.txt
>> ../6.2.0/allcodepoints.txt:2308;DISALLOWED;I;;LEFT CEILING
>> ../6.2.0/allcodepoints.txt:2309;DISALLOWED;I;;RIGHT CEILING
>> ../6.2.0/allcodepoints.txt:230A;DISALLOWED;I;;LEFT FLOOR
>> ../6.2.0/allcodepoints.txt:230B;DISALLOWED;I;;RIGHT FLOOR
>> ../6.3.0/allcodepoints.txt:2308;DISALLOWED;I;;LEFT CEILING
>> ../6.3.0/allcodepoints.txt:2309;DISALLOWED;I;;RIGHT CEILING
>> ../6.3.0/allcodepoints.txt:230A;DISALLOWED;I;;LEFT FLOOR
>> ../6.3.0/allcodepoints.txt:230B;DISALLOWED;I;;RIGHT FLOOR
>> 
>>> Can somebody check whether and how they affect IDNA 2008 and/or precis?
>>> 
>>> Again, if that has already been done, sorry for the noise.
>>> 
>>> Regards,   Martin.
>>> 
>>> 
>>> P.S.:
>>> All the other changes in UnicodeData.txt:
>>> 
>>> Change in numerical value only:
>>> 
>>> OLD
>>> 12456;CUNEIFORM NUMERIC SIGN NIGIDAMIN;Nl;0;L;;;;-1;N;;;;;
>>> 12457;CUNEIFORM NUMERIC SIGN NIGIDAESH;Nl;0;L;;;;-1;N;;;;;
>>> NEW
>>> 12456;CUNEIFORM NUMERIC SIGN NIGIDAMIN;Nl;0;L;;;;2;N;;;;;
>>> 12457;CUNEIFORM NUMERIC SIGN NIGIDAESH;Nl;0;L;;;;3;N;;;;;
>> 
>> $ egrep '^1245[67];' ../6.[23].0/allcodepoints.txt
>> ../6.2.0/allcodepoints.txt:12456;DISALLOWED;I;;CUNEIFORM NUMERIC SIGN NIGIDAMIN
>> ../6.2.0/allcodepoints.txt:12457;DISALLOWED;I;;CUNEIFORM NUMERIC SIGN NIGIDAESH
>> ../6.3.0/allcodepoints.txt:12456;DISALLOWED;I;;CUNEIFORM NUMERIC SIGN NIGIDAMIN
>> ../6.3.0/allcodepoints.txt:12457;DISALLOWED;I;;CUNEIFORM NUMERIC SIGN NIGIDAESH
>> 
>>> New characters (my understanding is that these are taken care of automatically):
>>> 
>>> 061C;ARABIC LETTER MARK;Cf;0;AL;;;;;N;;;;;
>> 
>> $ egrep '^061C;' ../6.[23].0/allcodepoints.txt
>> ../6.2.0/allcodepoints.txt:061C;UNASSIGNED;I;J;<reserved>
>> ../6.3.0/allcodepoints.txt:061C;DISALLOWED;I;C;ARABIC LETTER MARK
>> 
>>> 2066;LEFT-TO-RIGHT ISOLATE;Cf;0;LRI;;;;;N;;;;;
>>> 2067;RIGHT-TO-LEFT ISOLATE;Cf;0;RLI;;;;;N;;;;;
>>> 2068;FIRST STRONG ISOLATE;Cf;0;FSI;;;;;N;;;;;
>>> 2069;POP DIRECTIONAL ISOLATE;Cf;0;PDI;;;;;N;;;;;
>> 
>> $ egrep '^206[6789];' ../6.[23].0/allcodepoints.txt
>> ../6.2.0/allcodepoints.txt:2066;UNASSIGNED;I;CJ;<reserved>
>> ../6.2.0/allcodepoints.txt:2067;UNASSIGNED;I;CJ;<reserved>
>> ../6.2.0/allcodepoints.txt:2068;UNASSIGNED;I;CJ;<reserved>
>> ../6.2.0/allcodepoints.txt:2069;UNASSIGNED;I;CJ;<reserved>
>> ../6.3.0/allcodepoints.txt:2066;DISALLOWED;I;C;LEFT-TO-RIGHT ISOLATE
>> ../6.3.0/allcodepoints.txt:2067;DISALLOWED;I;C;RIGHT-TO-LEFT ISOLATE
>> ../6.3.0/allcodepoints.txt:2068;DISALLOWED;I;C;FIRST STRONG ISOLATE
>> ../6.3.0/allcodepoints.txt:2069;DISALLOWED;I;C;POP DIRECTIONAL ISOLATE
>> 
>>    Patrik
>> 
> _______________________________________________
> Idna-update mailing list
> Idna-update@alvestrand.no
> http://www.alvestrand.no/mailman/listinfo/idna-update