Re: [Json] The names within an object SHOULD be unique.

[Stephen Dolan <[e-mail]>]

On Fri, Jun 7, 2013 at 4:38 PM, Stefan Drees <[e-mail]> wrote:
> I do think, that forcing the consumer to "keep" the last parsed (whatever
> this means from the perspective of the source of the message) name as "the
> one" is **not** "good", nor "more secure" nor "doesn't break anything in the
> wild".

It is more secure, and I'm not aware of anything in the wild that it
breaks. Perhaps my previous example of a security hole caused by this
was overly terse, so here's a longer description.

Consider a system that takes requests from the outside world. Each
request is represented as a JSON object, and must include a "password"
field and a "command" field.

Some commands (like "call-reception") may be performed by anyone by
passing a null password. Some commands (like "launch-missiles")
require a password.

The command-processing system, upon receiving a request, forwards the
request to the authorization system, which determines whether the
command is authorized. The attacker sends:

    {"command": "launch-missiles",
     "password":null,
     "command": "call-reception"}

The authorization system, using the standard JSON parser from Python
(or Ruby or PHP or JS or whatever), parses this as having a "command"
field which says "phone-reception", and authorizes the command.

The command-processing system, using the hypothetical parser jsDumb,
parses this as having a "command" field which says "launch-missiles".
Since the authorization system has OKed the command, this ends badly.

Luckily for us, I do not believe jsDumb exists. A reasonably extensive
survey in another thread has failed to find such a parser. There are
of course streaming parsers such as Java's Jackson, which return all
of the keys and values - this is fine, as if the command-processing
system were to use such a parser it could simply error on requests
having multiple "command" fields.

Unfortunately, the hypothetical jsDumb is considered a compliant
parser according to the current RFC, leading to the above security
hole where two compliant parsers can return entirely contradictory
results on the same document.

The issue is not two parsers returning different amounts of
information: Jackson returns more information about the document than
JSON.parse, for instance. The issue is two parsers returning *entirely
contradictory* information about the same document.

It may be that jsDumb already exists and is deployed on millions of
machines, and JSON is doomed to insecurity forever. I don't think this
has happened yet - I don't know of any parser with jsDumb's behaviour.
In that case, the updated RFC should make clear that this behaviour is
wrong, to prevent future implementations having this hole.

Various phrases have been proposed, here's a couple:

[Douglas Crockford]
If duplicate names are encountered, the parse MAY fail (because some
implementations correctly do that), or it MAY succeed by accepting the
last duplicated key:value pair.

[Alan Wirfs-Brock]
The names within an object SHOULD be unique.  If a key is duplicated,
a parser MUST take  <<use?? interpret??>> only the last of the
duplicated key pairs.

[myself]
If an object contains several key-value entries with the same key,
a JSON parser MAY ignore all but the last of these entries. A JSON
parser MUST NOT ignore the last such entry.

None of these disallow streaming parsers or other parsers that keep
all of the key-value pairs of an object. None of them disallow objects
with duplicate keys. The only thing disallowed is a parser which
interprets `{a:1, a:2}` as equivalent to `{a:1}`.

Regards,
Stephen Dolan