Re: [http-auth] Normalization forms in draft-ietf-httpauth-basicauth-enc

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 7/2/13 1:59 AM, Yoav Nir wrote:
> Hi
> 
> For those of us not so well versed in I18N issues, what do you mean
> by normalization?

Martin described it already.

You might find it useful to look at the slides I put together for a
two-hour tutorial on internationalization at an IETF meeting a few
years ago:

https://stpeter.im/files/i18n-into.pdf

Perhaps I need to offer that tutorial again? Or maybe turn it into a
small book?

> Is it just consolidating look-alike code points, like the multiple
> hyphens that exist in Unicode?

There is no complete solution to the problem of confusable characters.
Here is a fun example for you:

https://stpeter.im/journal/1420.html

However, normalization takes care of some (but not all) instances of
code points that look alike.

> Does it also involve removing Arabic and Hebrew points? (I think
> Paul raised this one)

There is no reason to *remove* characters from left-to-right scripts.
However, allowing them means you need to deal with interesting bidi
issues.

> Does it involve removing diacritics?

Not necessarily -- why can't my username be saintandré or whatever?
And *removing* diacritics might be problematic, if it implies that 'é'
would be transformed to 'e' (thus introducing the possibility of
additional false positives).

> Does it involve splitting combined characters (like U+00E6 into 'a'
> and 'e')?

Actually, there is no decomposition of U+00E6 æ into 'a' and 'e' in
Unicode. This might be counter-intuitive, but it's true.

> Is there a standard we can point to and say "do this before
> comparing or hashing"?

I'd suggest looking carefully at SASLprep (RFC 4013) and its proposed
replacement:

http://datatracker.ietf.org/doc/draft-ietf-precis-saslprepbis/

If you have feedback on the latter, please post to the precis@ietf.org
list:

https://www.ietf.org/mailman/listinfo/precis

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.19 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJR0tMXAAoJEOoGpJErxa2ptSwP/iKfNqKsVsw+e2C8BSfkHhgw
HWrqWOkbIfvmQyOCssdsEuw6+cjZCtyDpGYKbQIP1WulxFlvjTT7DxEaEkVExuFk
kHYVc16AL0HxBBfMZlGAgtHQDm0J0P4FQ1V0A5JFckh31ZnE2lgbcF1aQGwHm0CS
58IY+4bMdZP7DFoKPSSU7p7DGw7dpacY9GdMGNAnCopPFfFsuaP2WEA7Q8P+HcEk
0nK7/ucc72DcagpzAFfXQ7MQ5TaMy1echSmw2w7SmuSr3KmYOaDqiEj17azTEKlX
lwsfGnZ06Exd6Q1qgLfwmXJjmaa0bU2NWKOtsAMVbcKpSMUa3838KEynSxd/j5fT
+2+t8KKAVf8uKs5wC6IEFn3qGeORbD4/q3v+GhLve9hksmu7OaIhQ8/EqpkyomUP
cD6SU3DfFas2PSAynOWqkkHPOFPGaGrVGwn/9bQrtsXtC5rrlMphLLKngzD4Jf68
GeyJXYZ4GCp8K6xNAeiA9Tu9PVS3Cpvo1tgq2/c+oxVTAZc3PWCp7oUOfYxzG0c0
oLv7/Fwfio8PP94rRjWgD/iP7+ZVzMplA7FSL0tkYUtnLL/9zohlFZYmLgcZcTiZ
vIkuCSY0o7hh6p2VVPFEgxFm4M0Y1WxkZe1C8FKL+S7/PBF6Xi4eUCkqx66EdrDF
vS/SGLuK5sl4t81vXXjH
=sP1o
-----END PGP SIGNATURE-----