Re: [Json] BOMs

["Martin J. Dürst" <[e-mail]>]

Hello Henry, others,

On 2013/11/20 3:55, Henry S. Thompson wrote:
> Allen Wirfs-Brock writes:
>
>> There can be no doubt that the most widely deployed JSON parsers are
>> those that are built intp the browser javascript implementations.
>> The ECMAScript 5 specification for JSON.parse that they implement
>> says BOM is an illegal character.  But what do the browser actually
>> implement?  This:
>
> No, try e.g. jsonviewer.stack.hu [1] (works in Chrome, Safari, Opera,
> not in IE or Firefox)

In Firefox, I got some garbled characters, in particular some question 
marks for each of the two bytes of the BOM and one question mark for the 
e-acute. Because of the type of the errors, I strongly suspect it is 
related to what we are trying to investigate, and so I don't think this 
can be taken as evidence one way or another.

or feed [2] to www.jsoneditoronline.org (Use
> Open/Url) (works in Chrome, IE, Firefox, ran out of time to test more).

The fact that some libraries or Web sites accept a BOM for JSON isn't a 
proof that all (well, let's say the majority) accept a BOM.

> As previously discussed, _no-one_ is arguing that BOMs are in the JSON
> language as such.  JSON parsers shouldn't accept BOMs.
>
> BOMs are, to quote the UNICODE spec, "not part of the text".  It is
> appropriate that specs concerned with JSON-on-the-wire, for example
> the media type registration for 'application/json', _should_ discuss
> the BOM, and it's open to them, _without changing the language at
> all_, to say that BOMs are acceptable but, again, are not part of the
> text which the parser has to accept.

I agree that *from a theoretical viewpoint*, this is correct. But theory 
isn't everything. As I have written before (and you have cited in 
another thread, for another spec):

   What's most important now is to know what receivers actually
   accept. We are not in a design phase, we are just updating the
   definition ... and making sure we fix problems if there are
   problems, but we have to use the installed base for the main
   guidance

For our update from RFC 4627, the null hypothesis is that there are no 
BOMs (neither for UTF-8 nor for UTF-16). The patterns given in 
http://tools.ietf.org/html/rfc4627#section-3 cannot apply to characters, 
they can only apply to bytes. If we want to allow a spec in 
application/json, then we have to have strong evidence that almost all 
parsers can deal with BOMs, not just fragmentary evidence that some 
parsers don't choke on a BOM.

Please note that there's some parallel to XML, in that neither Unicode 
(for the encoding form) nor the IETF (for the 'charset') require a BOM 
for "UTF-16", but XML nevertheless strictly requires it.

Regards,   Martin.

> ht
>
> [1] http://jsonviewer.stack.hu/#http://www.ltg.ed.ac.uk/~ht/ov-test/b16le.json