Re: [Json] About JSON equality

[Nico Williams <[e-mail]>]

On Tue, Feb 19, 2013 at 4:55 PM, Francis Galiegue <[e-mail]> wrote:
> This definition is only lacking for strings. On some other draft which
> I cannot remember off the top of my head, two JSON String values are
> to be considered equal if, position wise, their Unicode code points
> are the same (and THAT INCLUDES \u0000).

String equality is definitely a case where normalization does come in.

In practice most Unicode strings are in NFC (normalization form
composed) by accident: it's what input methods typically produce.  But
it doesn't have to be that way.  For some scripts it almost certainly
isn't always that way (I'm thinking of Hangul in particular).  And
then there's the dreaded HFS+, which, for some unknown [to me] reason
that I cannot fathom, normalizes to NFD on create(!).

So, suppose you were creating a JSON representation of file names in a
filesystem.  If that filesystem were HFS+ then you're guaranteed to
have NFD.  If it's any other filesystem (except, perhaps, other
filesystems on OS X, like at least one port of ZFS to it) then chances
are very high that the file names will be in NFC.  This is not really
a contrived example -- I really, really wish it were, but I think it
more than likely that we'll have such uses of JSON.

So, if you want to be able to compare JSON strings for equality, you
do need to specify whether you care about normalization.
Normalization-insensitive string comparison is *not* that expensive
for mostly ASCII strings: the trivial optimization is to have a fast
path that compares characters in logical two-byte windows that move
one byte at a time, and a slow path that gets triggered only whenever
one of those bytes is non-ASCII.  But it's still a PITA.

Nico
--