Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"

[Pete Cordell <[e-mail]>]

On 28/03/2017 05:48, Tim Bray wrote:
> First of all, let me say that I’m delighted with, and fully support, the
> promotion of the status of UTF-8 in the JSON RFC to MUST.  I suspect
> this steps way outside the JSONbis charter, but that’s a problem for
> chairs and ADs, not yr humble editor.
>
> Comments on Matt's proposed text:
>
> 1. How about a very short historical note, along the lines of: “Previous
> specifications of JSON, including the predecessor RFCs, have not
> required the use of UTF-8 for use with the application/json media type.
> However, implementors of JSON-based software have overwhelmingly chosen
> to use the UTF-8 encoding, to the extent that it is the only realistic
> way to achieve interoperability in software which generates or consumes
> JSON.”
>
> ... moving on...


If I ruled the world, I'd strip it down even more.  Merging with Matt's 
text, I'd go with something like:

"""
JSON text SHOULD be encoded in UTF-8 (Section 3 of [UNICODE]).  When
used with media type "application/json" the JSON text MUST be encoded
as UTF-8.

     Previous specifications of JSON, including the predecessor RFCs,
     have not required the use of UTF-8 for use with the
     application/json media type. However, implementors of JSON-based
     software have overwhelmingly chosen to use the UTF-8 encoding, to
     the extent that it is the only realistic way to achieve
     interoperability in software which generates or consumes JSON.

Implementations MUST NOT add a byte order mark (U+FEFF) to the
beginning of a JSON text.  In the interests of interoperability,
implementations that parse JSON texts MAY ignore the presence of a
byte order mark rather than treating it as an error.
"""

But I'm very happy with Matt's proposal, so please consider this more 
towards the "editorial typo" class than a blocker :-)

On:
 > ​2. Seriously, ... we shouldn't waste RFC space talking about
 > practices that are not remotely interoperable.  The I in IETF stands
 > for Internet, and JSON on the Internet is UTF-8, end of story.

The title of the RFC is "_The_ JavaScript Object Notation (JSON) Data 
Interchange Format".  To me that sounds like a fairly definitive 
definition of JSON.  As the IETF often writes stuff that's more general 
than just usage on the Internet, I think it's reasonable to believe that 
this text covers all JSON usages; and people would be justified to argue 
that.  If it is just for the Internet, we could tweak the title, add 
something to that effect to the Introduction, and just say MUST be UTF-8.

Thanks,

Pete Cordell
Codalogic Ltd