Re: [Json] A possible summary of the discussion so far on code points and characters

[R S <[e-mail]>]

On Sat, Jun 8, 2013 at 2:11 PM, Stephen Dolan <[e-mail]>wrote;wrote:

> On Sat, Jun 8, 2013 at 9:52 PM, R S <[e-mail]> wrote:
> > A seventh point of view, which I happen to agree with: JSON strings are a
> > sequence of code units.
> >
> > This is similar to the definition of 'source text' in ECMAScript:
> >
> > "ECMAScript source text is assumed to be a sequence of 16-bit code units
> for
> > the purposes of this specification. Such a source text may include
> sequences
> > of 16-bit code units that are not valid UTF-16 character encodings."
>
> That's a very out-of-context quote. The linked document states:
>
> "ECMAScript source text is represented as a sequence of characters in
> the Unicode character encoding, version 3.0 or later."
>
> It then gives your quote, and states "If an actual source text is
> encoded in a form other than 16-bit code units it must be processed as
> if it was first convert [sic] to UTF-16". It seems like UTF-16 is a
> convenient way to frame the document, rather than a requirement of the
> specification.


It's a requirement. Here are some additional references:

<
http://wiki.ecmascript.org/doku.php?id=strawman:support_full_unicode_in_strings&rev=1305822947
>
<https://mail.mozilla.org/pipermail/es-discuss/2011-May/014337.html>

The paragraph following the one I cited:

'Throughout the rest of this document, the phrase “code unit” and the word
“character” will be used to refer to a 16-bit unsigned value used to
represent a single 16-bit unit of text. The phrase “Unicode character” will
be used to refer to the abstract linguistic or typographical unit
represented by a single Unicode scalar value (which may be longer than 16
bits and thus may be represented by more than one code unit). The phrase
“code point” refers to such a Unicode scalar value. “Unicode character”
only refers to entities represented by single Unicode scalar values: the
components of a combining character sequence are still individual “Unicode
characters,” even though a user might think of the whole sequence as a
single character.' <http://es5.github.io/x6.html>

- Rob