Re: [Json] What is a JSON-text? [REVISITED]

[Bjoern Hoehrmann <[e-mail]>]

* Matt Miller (mamille2) wrote:
>What do others think about changing JSON-text to allow
>any JSON value versus just objects or arrays?

As far as I am concerned, the Working Group is supposed to standardise
JSON and right now we have a chaotic mix of implementations and usages
where a parser might default to "lax" parsing permitting `null` at the
top-level and a "strict RFC 4627" mode disallowing that, others might
default to the strict mode and offer a lax one, others might offer only
one mode; there are probably even cases where `json_encode` is con-
servative in what it sends, while the corresponding `json_decode` func-
tion is liberal in what it accepts, with the unfortunate property that

  json_encode(json_decode(...))

might fail, even though nobody would suspect that reading the code. In
some cases the "lax" parser mode might also support other features not
normally allowed, say trailing commas, comments, or the empty string as
JSON text. A consequence might be that turning on the "lax" mode adds
security vulnerabilities or other hazards to a system. It seems pretty
clear to me that we would be better off now if RFC 4627 had made it
clear that all values can be used at the top level.

I assume some people see JSON primarily in the sense of "JSON file" or
"complete message of a JSON-based protocol" and dislike scalar values
at the top level because you would not encounter them in such settings.

In practise, JSON is frequently used in many other settings, to encode
values for other protocol elements or APIs for instance. In web browser
settings for instance people might put JSON-serialised data into cookies
or the text-only "local storage", in `application/x-www-form-urlencoded`
parameters, or communication across windows using `.postMessage` and its
ilk, or as message in low-overhead protocols like the WebSocket.

With such usage it is fairly common that values might be either an array
of numbers, or just the value null, or an object or null, just as it may
be a string or null. Sometimes it's either an object, or a error string.

So, if one were to design a JSON parsing and serialisation API, it seems
clear that one would allow serialisation of scalar values and would make
the parser accept whatever the serialiser generates; especially this

  var json_text = "...";
  var obj = json_decode(json_text);

  assert( obj.fragment eq json_decode(json_encode(obj.fragment)) );

should succeed without failure if `json_text` is properly formed JSON.
If that is not the case, users of the parsing and serialisation API are
likely to have plenty of rarely surfacing hazards in their code. As a
simple example, consider

  var json_text = "...";
  var obj = json_decode(json_text);
  ...
  var response = send_network_request(..., json_encode(obj.fragment));

  if (!response.is_success) {
    ...
  }

If `json_encode` serialises `obj.fragment` as requested, even when it
is `null`, then send_network_request() would be passed a normal string
just like it would in all other cases, and assuming the other endpoint
in the network connection regards a top level `null` as error, the error
would likely be caught alongside other network errors. If instead the
`json_encode` method threw an exception, who knows what would happen; if
it returned a special value, say an `undef` value, again, who knows how
`send_network_request` will react.

Now, if the JSON library happily generates scalar values as the top then
it becomes the responsibility of the caller to ensure scalar values at
the top level are not generated when it's considered "wrong". That is
very hard: there seems to be a large grey area where it's not clear if
scalar top level values are a bad practise, and if you take the example
above, telling whether `obj.fragment` might ever be a scalar can take a
lot of effort, especially if the code around json_encode can change, and
the `json_encode` caller might not even know because the `json_text` is
produced by a third party network service where the documentation does
not say conclusively whether it may ever be a scalar; and guarding each
and every `json_encode` call with checks for top level scalars is hard
aswell, especially considering the costs of the resulting code bloat.
And the same goes for stuffing everything into an extra array or object.

Worse, even in cases like "JSON in HTTP request body" there appears to
be no consensus that sending top level scalar values is bad practise; I
remind once more of the proposed JSON support in XMLHttpRequest that has
no problems generating them, and the lack of complaints against that.

The "arguments" against allowing scalar values at the top level so far,
well, I have seen no attempt to argue that, all things considered, those
in charge of running code that supports JSON-based protocols are best
off with top level scalars disallowed, and it seems a rather unlikely
proposition to me. So I am very much against allowing people to say
strings like `null`, `""`, `1.0` are not JSON on their own.
-- 
Björn Höhrmann · mailto:[e-mail] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: [phone] · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/