Re: [Json] JSON for Internet messages

[Nico Williams <[e-mail]>]

On Wed, Jul 3, 2013 at 11:34 AM, Tim Bray <[e-mail]> wrote:
> So I care a lot about JSON, but I don’t care in the slightest about usages
> where the JSON isn’t being used for application-level message protocol
> payloads (are there such usages?  I'm curious).  Also, I’ve never
> encountered a scenario where the messages were of sufficient size that
> anyone gave a rat’s ass about streaming.

Er, I work with an application where I need to atomically read *all*
the data (it's ok if it's slow) once, then poll for / subscribe to an
incremental update feed.  The first requirement has devolved into
doing a single GET with chunked data results -- pagination failed
because we couldn't get the application in question to provide
atomicity with pagination.

That data comes to me as JSON.  It can be huge.  It happens to be an
array of relatively small objects, so it's possible to implement the
generation (and parsing) in a streaming way for the top-level value
and still prevent / deal with duplicate names one level down.

But you can see how one can end up needing streaming generators and parsers.

I could write a parser that streams the array and doesn't stream the
objects.  But I'm going to use an off-the-shelf parser as much as
possible.  The pressure to use a streaming parser (and generator) will
increase as the dataset increases in size.  My fear is that the cost
of dealing with this dataset will increase superlinearly with dataset
size if I use a non-streaming parser; I fear that eventually I won't
be able to use a non-streaming parser at all.

For all other purposes the application in question uses small enough
JSON documents that I wouldn't need streaming anything.

Now, in this application it just so happens that there are other ways
to guarantee non-duplicate names that are already in place, so the use
of a streaming generator/parser that does nothing about duplicate
names would be just fine.

I don't mind saying "MUST NOT" have dups.  I just want the text
written in such a way that this application can use JSON after all.
For me that would mean that it's OK to have streaming generators and
parsers, as long as the application makes sure there's no dups in that
case.

Nico
--