Re: [Json] JSON for Internet messages

[Tatu Saloranta <[e-mail]>]

On Wed, Jul 3, 2013 at 12:29 PM, Jacob Davies <[e-mail]> wrote:

> On Wed, Jul 3, 2013 at 9:34 AM, Tim Bray <[e-mail]> wrote:
> > So I care a lot about JSON, but I don’t care in the slightest about
> usages
> > where the JSON isn’t being used for application-level message protocol
> > payloads (are there such usages?  I'm curious).
>
> There are a number of practical use cases for JSON that don't fall
> into the neat "externally delimited stream of bytes described by an
> application/json media-type" category. That is a separate question to
> whether the RFC needs to do much to accomodate them.
>
> 1. Files named ".json" and containing a single JSON object or array.
> 2. Text documents containing embedded JSON - informally, as in
> documentation or books, or formally, as in Javascript in HTML or .js
> files.
> 3. Database records where text or binary columns store embedded JSON
> representing complex nested objects.
>
> Case 1 is handled the same as a message described as application/json,
> no big deal. Cases 2 & 3 may require accommodating non-Unicode
> encodings, and may have some special security concerns (e.g.
> additional escaping for HTML or Javascript embedding). It would be
> nice if the spec separated format from encoding, but it's unlikely to
> be the source of any significant problems since in most cases the
> handling of encoding will have happened by the time you parse the
> text, and the security concerns can be addressed in best-practices.
>
> > Also, I’ve never encountered a scenario where the messages were of
> sufficient size that
> > anyone gave a rat’s ass about streaming.
>
> Me either, and even if there were such cases I find the concerns about
> maintaining state unconvincing - maintaining a seen-keys dictionary
> for each level of nesting you're in is not all that much of an
> imposition or incompatible with a streaming API.
>
>
Believe it or not, but this actually would become major overhead for
streaming level; as well as for data-binding tools of statically typed
languages (Java, C#).
I am sure that there are clever ways to limit the overhead, but a naive
implementation in Java would give us 30-50% processing time boost, which
generally adds no value (assuming "last value wins" is adhered to and/or if
generator can avoid producing dups).

Common use case for me is as follows:

1. Input comes as a typed object
2. Serializer accesses properties and guarantees uniqueness of keys, calls
generator
3. Generator simply outputs tokens as requested (maintaining basic stack to
ensure syntactic output)
4. Receiving end uses parser to expose JSON as tokens, without dup checks
5. Deserializer maps properties to local objects created: if duplicates
were encountered, last value would stick due overwrites

in such a case, duplicates are not problematic; and this without explicit
checks.
Having to check duplicates on both sending and receiving end (at
parser/generator level) would most likely add +50% overhead for processing
time, because overhead on generator is relatively higher (JSON can be
written about 2x as fast as decoded).
Dup detection could work more efficiently on deserializer level, but would
require separate null markers to distinguish between JSON nulls and lack of
property.

Alternate case of using a Tree representation ("JsonNode" etc) would have
similar solution to duplicate problem: tree serializer would not produce
dups (since tree can not contain them); and tree deserializer would either
use last, or (if preferred) signal an error.

-+ Tatu +-