Re: [Json] On characters and code points

[Nico Williams <[e-mail]>]

On Sat, Jun 8, 2013 at 10:25 AM, Paul Hoffman <[e-mail]> wrote:
> On Jun 8, 2013, at 2:38 AM, Carsten Bormann <[e-mail]> wrote:
>> (4) The remaining question appears what we do with unpaired escaped surrogates.
>> The answer will have to be a bit wishy-washy, because anything strong will invalidate half of the implementations.  If is probably a good idea not to "break" those applications that compensate JavaScript's lack of a binary string type by using UTF-16 as a vector of unconstrained 16-bit values, but we also cannot mandate that everyone adopt this 1990s style hack.

I'm starting to conclude that "anything goes..."

"...with the understanding that in some cases some, er, code points
get dropped".  E.g., a surrogate encoded in UTF-8 or UTF-32 makes no
sense, and though some parsers can leave it be, a parser that converts
to UTF-16 probably can't leave there.  Note that there's a bit of a
difference between an escaped and an unescaped such code point: the
former might be an attempt to pass binary data, while the latter could
just be either an error or the result of unescaping the first.

> Ummm, how is that "much better"? "Code points minus THEONESWEHATE" seems a lot simpler than "characters plus ADDITIONAL1 plus ADDITIONAL2 plus ADDITIONAL3".

The latter seems better in that we defer the definition of "character"
to the UC, but the former seems better in that it's much more specific
about THEONESWEHATE.  The latter seems likely to allow more
interoperability with any implementations that allow just about
anything.

Nico
--