Re: [Json] BOMs

["Martin J. Dürst" <[e-mail]>]

Okay, here are some more tests.

http://www.sw.it.aoyama.ac.jp/2013/pub/json_tests/test1_utf8_nobom.json
http://www.sw.it.aoyama.ac.jp/2013/pub/json_tests/test2_utf8_bom.json

They are self-describing JSON files served with application/json, the 
first without a BOM, and the second with a BOM.

They contain some Japanese, and a tiny bit of Spanish.

[see more below]

On 2013/11/18 21:59, Henry S. Thompson wrote:
> Bjoern Hoehrmann writes:
>
>> Perl's JSON module gives me
>>
>>    malformed JSON string, neither array, object, number, string
>>    or atom, at character offset 0 (before "\x{ef}\x{bb}\x{bf}[]")
>>
>> Python's json module gives me
>>
>>    ValueError: No JSON object could be decoded
>>
>> Go's "encoding/json" module gives me
>>
>>    invalid character 'ï' looking for beginning of value
>
> I'm curious to know what level you're invoking the parser at.  As
> implied by my previous post about the Python 'requests' package, it
> handles application/json resources by stripping any initial BOM it
> finds -- you can try this with
>
>>>> import requests
>>>> r=requests.get("http://www.ltg.ed.ac.uk/ov-test/b16le.json")
>>>> r.json()

I get a 404 on this example. I can put up UTF-16 examples, too.

Regards,   Martin.

> Signatures are not part of the text of a document, as the UNICODE spec
> makes clear, so asking what happens when you pass a string beginning
> with a BOM to a parser is not really the right question in this
> context, is it?
>
> As I tried to say in an earlier post, there's a distinction which
> needs to be carefully insisted on between, on the one hand, languages
> and their parsers, where I agree signatures/BOMs have no place, and,
> on the other hand, (media-typed) resources/entities/payloads and _their_
> processing, where a discussion of BOMs/signatures _is_ appropriate
> and, often, necessary.
>
> BTW I agree that the status of the UTF-8 BOM as signature is slightly
> hazy, but again the UNICODE spec itself [1] says
>
>    "this sequence can serve as signature for UTF-8 encoded text where
>     the character set is unmarked"
>
> ht
>
> [1] http://www.unicode.org/versions/Unicode6.2.0/ch16.pdf