Re: [Json] On characters and code points

[Tim Bray <[e-mail]>]

On Fri, Jun 7, 2013 at 8:56 AM, Paul Hoffman <[e-mail]> wrote:

> This may be a part of the spec where some people have to hold their noses.
> The Unicode definition of "character" does not include non-characters, and
> the code points for some of those non-characters make sense in JSON strings
> when those strings. Bjoern has pointed out a good one: strings used for
> test cases of other code. The issue not just unpaired surrogates. Do we
> *really* want to prohibit:
>    { "End of data marker": "\uFFFF" }
>

Yes, I *really* want to prohibit that. The one corner case it buys you is
outweighed by a factor of a thousand or so in not being able to use
general-purpose string processing software to deal with JSON payloads.
BTW, a huge amount of deployed software out there ALREADY processes JSON
text fields using general-purpose string processing libraries, and will
explode unpredictably and in hard-to-debug ways if this starts happening.

Also, consider the lovely consequences when unpaired surrogates start
showing  up in key fields and are fed to hash functions in every
programming language in the world, which expect to receive Unicode
characters.

 -T



>
> Proposal:
>
> Remove the word "character" from the spec except in an explanatory
> paragraph in Section 2.5 that says:
>    All code points, even those that represent non-characters in the
> Unicode specification [UNICODE], are allowed in JSON strings.
>
> --Paul Hoffman
> _______________________________________________
> json mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/json
>