Re: [Json] On characters and code points

[Tim Bray <[e-mail]>]

John is right and I am wrong.  What I really want to legislate against is:

{
 "Mis-use of UTF-16 surrogates" : "\udf46\ud800AB\udf11CD\ud812EF",
 "Mis-use of flipped BOM" : "AB\ufffeCD"
}


On Fri, Jun 7, 2013 at 10:19 AM, John Cowan <[e-mail]> wrote:

> Tim Bray scripsit:
>
> > >    { "End of data marker": "\uFFFF" }
> > >
> >
> > Yes, I *really* want to prohibit that. The one corner case it buys you is
> > outweighed by a factor of a thousand or so in not being able to use
> > general-purpose string processing software to deal with JSON payloads.
>
> Most general-purpose string processing software is perfectly happy
> with U+FFFF.  There are three different kinds of code points here,
> and it doesn't help to conflate them:
>
> 1) Surrogate code points.  These will never be assigned to any characters,
> and reserved for use as UTF-16 code units.  There are exactly 2048 of
> these, from U+DC00 to U+DFFF.
>
> 2) Non-character code points.  These will never be assigned to any
> characters, and are not meant to be interchanged, but internal software
> is expected to handle them.  There are exactly 66 of these, and U+FFFF
> is one.  See <http://www.unicode.org/faq/private_use.html#noncharacters>
> for more about this group.
>
> 3) Unassigned code points.  These are not assigned to any characters
> today, but may be assigned in future.  They may be interchanged.
> Internal libraries should process them.
>
> My view is that group 1 are and should be disallowed in JSON; others
> disagree.  Group 2 should be avoided by JSON creators, but accepted by
> JSON parsers, which may choose to change them to U+FFFD (replacement
> character).  Group 3 are and should be valid in JSON.
>
> --
> Values of beeta will give rise to dom!          John Cowan
> (5th/6th edition 'mv' said this if you tried    http://www.ccil.org/~cowan
> to rename '.' or '..' entries; see              [e-mail]
> http://cm.bell-labs.com/cm/cs/who/dmr/odd.html)
>