Re: [Json] Allow any JSON value at the top level

["Martin J. Dürst" <[e-mail]>]

On 2013/06/12 3:41, Tatu Saloranta wrote:
> On Tue, Jun 11, 2013 at 2:25 AM, Carsten Bormann<[e-mail]>  wrote:

>> This is irrelevant in practice as JSON is used with UTF-8 in practice.
>>
>
> My main concern is with UTF-16. My understanding is that for "Big 5"
> languages its use make sense, from efficiency perspective. I do not have
> data on this; in XML space document test sets had non-trivial amount of
> content in various encodings.
>
> If UTF-16 is not widely used then I can see why this would be considered of
> little significance.

There are a lot of scripts (and therefore languages) where a character 
takes 3 bytes in UTF-8 but only 2 bytes in UTF-16. In particular, this 
includes all the languages of East/South East/South Asia, a huge area 
with a huge population. It's the reason why UTF-16 was made mandatory 
for XML.

However, as predicted by some, and widely confirmed in practice, most 
actual data (including XML and of course JSON) contains a significant 
amount of characters from the ASCII repertoire (syntax such as []{}"", 
and space for JSON, plus many if not most names and many values). These 
characters take only one byte in UTF-8, but two bytes in UTF-16. As a 
result, content is very often shorter in UTF-8, and when it happens to 
be longer, it's usually not much longer.

Overall, the advantage of occasionally shorter data is clearly 
outweighted by the simplicity of a single encoding (even if not on the 
receiver side, then on the generating side).

As a result, virtually the whole Web ecosystem is moving towards using 
UTF-8 only for public interchange, at a surprising speed. (Of course, 
many other encodings will still remain for years, but in lower and lower 
numbers).

Regards,   Martin.