Re: [Json] I-D Action: draft-ietf-jsonbis-rfc7159bis-04.txt

[Peter Saint-Andre - Filament <[e-mail]>]

On 7/24/17 9:12 AM, Pete Cordell wrote:
> On 22/07/2017 08:51, Anders Rundgren wrote:
>> I don't have any text to offer but I believe that UFT-8 encoding is the
>> only recommendable way for transferring JSON data from one system to
>> another which includes lots of other use case than HTTP.  For example:
>> Writing JSON to databases, usage in WebSocket messages, or simply putting
>> JSON in file.
>>
>> Yes, it seems to me that "Serialized format"/"External representation" is
>> really the common denominator rather than networks and/or protocols.
> 
> And:
> 
> On 22/07/2017 10:18, Anders Rundgren wrote:
>> I would consider "external form" as an alternative to limiting the
>> specification
>> to special cases like networks and or protocols.
> 
> And:
> 
> On 22/07/2017 08:53, Julian Reschke wrote:
>> Whatever we say, it should make clear that it includes all derived
>> media types (*/*+json) as well.
> 
> Thanks Anders and Julian.
> 
> Makes me wonder whether the best approach is to start with what is
> currently the second paragraph of 8.1 as a 'softener', and then hit the
> reader with UTF-8 being the only real way to go for interoperability of
> open systems. 

We're not writing a novel here. :-)

> So maybe 8.1 becomes something like:
> 
> 8.1.  Character Encoding
> 
>    Previous specifications of JSON have not required the use of UTF-8
>    when transmitting JSON text.  However, the vast majority of JSON-
>    based software implementations have chosen to use the UTF-8 encoding,
>    to the extent that it is the only encoding that achieves
>    interoperability.
> 
>    Therefore, JSON text exchanged between systems that don't form a
>    closed ecosystem MUST be encoded using UTF-8.
> 
>    Implementations MUST NOT add a byte order mark (U+FEFF) to the
>    beginning of a networked-transmitted JSON text.  In the interests of
>    interoperability, implementations that parse JSON texts MAY ignore
>    the presence of a byte order mark rather than treating it as an
>    error.

I feel like we're over-complicating things. I had proposed:

   When an entity transmits JSON text over a network, e.g. as the
   payload of an application protocol, it MUST encode that text using
   UTF-8 [RFC3629].

Carsten said that looked fine to him. Julian suggested that we mention
the application/json media type and types derived from it. Anders
suggested that we specify this in terms of serialized formats between
systems (even storage in a file) rather than networks or protocols.

Taking that feedback into account, I suggest:

   When an entity represents JSON text in a serialized format (e.g., as
   the payload of an application protocol using the application/json
   media type or a type derived therefrom), it MUST encode that text
   using UTF-8 [RFC3629].

Peter