Re: [Json] Proposal for strings/Unicode text

[Tim Bray <[e-mail]>]

On Wed, Jun 12, 2013 at 3:46 PM, Norbert Lindenberg <
[e-mail]> wrote:

> The JSON RFC seems to use Unicode character names, in this case case
> "reverse solidus".
>

We’re not allowed to change JSON but we are allowed to improve the spec.
The term “solidus” is obscure and off-putting.

> Strings begin and end with quotation marks.  They are intended,to contain
sequences of Unicode characters; Note however that the ABNF in this section
allows the inclusion of 16-bit quantities in ways which can never be useful
for representing characters and is likely to cause breakage in software
designed to process Unicode text.

This warning is too vague to be useful. Which specific risks do you think
> need to be discussed here? Also, the ABNF doesn't do anything specifically
> for 16-bit quantities, as far as I can see.
>

Agreed; I introduced a specific example.



>  > 16-bit quantities (normally Unicode characters from the Basic
> Multingual Pane(U+0000 through U+FFFF) may be “escaped”, or represented as
> a six-character sequence: a backslash (U+005C REVERSE SOLIDUS), followed
>  by the lowercase letter u, followed by four hexadecimal digits that encode
> the character's code point.  The hexadecimal letters A though F can be
> upper or lower case.  So, for example, a string containing only a single
> backslash may be represented as "\u005C".
>
> These escape sequences aren't about 16-bit quantities - they represent
> Unicode BMP code points.


They also represent surrogates.  Thus they combine fish and bicycles, and I
think the only accurate way to describe what you can escape is “16-bit
quantities”


> If we discuss these escape sequences in prose, then the sequences for BMP
> and supplementary characters need to be discussed together so that it's
> clear what's a surrogate pair and how unpaired surrogates are handled.
>

Hm?  I see no point in replicating the discussion of surrogates in Unicode,
and there is nothing useful to say about the handling of unpaired
surrogates.


> > Alternatively, there are two-character sequence escape representations
> of some popular characters.  So, for example, a string containing only a
> single backslash may be represented more compactly as "\\".
>
> I'd assume that this is not about popularity, but about the need to
> represent control characters and characters that are also used within the
> JSON syntax. I don't see two-character escapes for "e" or "的".
>

I tried to stay close to the language of the original.



>
> Norbert
>
>