Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"

[Julian Reschke <[e-mail]>]

...and here is a concrete proposal:

Original text:

> 8.1.  Character Encoding
>
>    JSON text SHALL be encoded in UTF-8, UTF-16, or UTF-32 [UNICODE]
>    (Section 3).  The default encoding is UTF-8, and JSON texts that are

Change 1:

Say "MUST" instead of "SHALL", as it's the more common form of 
expressing this requirement.

Change 2:

Replace "[UNICODE] (Section 3)" by "Section 3 of [UNICODE]".

That said, this citation isn't as stable as it should, as [UNICODE] 
refers to <http://www.unicode.org/versions/latest/> and unless I'm 
missing something, there's no guarantee that future versions will have 
the relevant bits in Section 3.

>    encoded in UTF-8 are interoperable in the sense that they will be
>    read successfully by the maximum number of implementations; there are
>    many implementations that cannot successfully read texts in other
>    encodings (such as UTF-16 and UTF-32).

Change 3:

Add "Text encoded in character encodings other than UTF-8, UTF-16, or 
UTF-32 can not be used with the media type "application/json".

(this explains the implications of the SHALL/MUST)


>    Implementations MUST NOT add a byte order mark (U+FEFF) to the
>    beginning of a JSON text.  In the interests of interoperability,
>    implementations that parse JSON texts MAY ignore the presence of a
>    byte order mark rather than treating it as an error.


Finally, change 4:

Add a new paragraph:

"Recipients that wish to support Unicode encodings other than UTF-8 can 
do this using a detection mechanism that is based on the fact that the 
first character will always have a Unicode code point less or equal than 
127, thus the UTF-16/32 variants can be detected by inspecting the first 
octets for nulls."


I believe none of these changes affects anything normative, but that 
they absolutely clarify the spec. In particular, having them in the spec 
would have avoided this whole discussion we just had.

Best regards, Julian