Re: [Json] On characters and code points

[John Cowan <[e-mail]>]

Tim Bray scripsit:

> >    { "End of data marker": "\uFFFF" }
> >
> 
> Yes, I *really* want to prohibit that. The one corner case it buys you is
> outweighed by a factor of a thousand or so in not being able to use
> general-purpose string processing software to deal with JSON payloads.

Most general-purpose string processing software is perfectly happy
with U+FFFF.  There are three different kinds of code points here,
and it doesn't help to conflate them:

1) Surrogate code points.  These will never be assigned to any characters,
and reserved for use as UTF-16 code units.  There are exactly 2048 of
these, from U+DC00 to U+DFFF.

2) Non-character code points.  These will never be assigned to any
characters, and are not meant to be interchanged, but internal software
is expected to handle them.  There are exactly 66 of these, and U+FFFF
is one.  See <http://www.unicode.org/faq/private_use.html#noncharacters>
for more about this group.

3) Unassigned code points.  These are not assigned to any characters
today, but may be assigned in future.  They may be interchanged.
Internal libraries should process them.

My view is that group 1 are and should be disallowed in JSON; others
disagree.  Group 2 should be avoided by JSON creators, but accepted by
JSON parsers, which may choose to change them to U+FFFD (replacement
character).  Group 3 are and should be valid in JSON.

-- 
Values of beeta will give rise to dom!          John Cowan
(5th/6th edition 'mv' said this if you tried    http://www.ccil.org/~cowan
to rename '.' or '..' entries; see              [e-mail]
http://cm.bell-labs.com/cm/cs/who/dmr/odd.html)