Re: [Json] serializing sequences of JSON values

Marc Lehmann <schmorp@schmorp.de> Mon, 10 March 2014 22:59 UTC

Return-Path: <schmorp@schmorp.de>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 802291A0527 for <json@ietfa.amsl.com>; Mon, 10 Mar 2014 15:59:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.809
X-Spam-Level:
X-Spam-Status: No, score=0.809 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_DE=0.35, HELO_MISMATCH_DE=1.448, HOST_MISMATCH_NET=0.311, J_CHICKENPOX_47=0.6] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i2eCrEei4Rqt for <json@ietfa.amsl.com>; Mon, 10 Mar 2014 15:59:23 -0700 (PDT)
Received: from mail.nethype.de (schmorp-1-pt.tunnel.tserv6.fra1.ipv6.he.net [IPv6:2001:470:1f0a:b09::2]) by ietfa.amsl.com (Postfix) with ESMTP id 8BB171A0647 for <json@ietf.org>; Mon, 10 Mar 2014 15:59:20 -0700 (PDT)
Received: from [10.0.0.5] (helo=doom.schmorp.de) by mail.nethype.de with esmtp (Exim 4.80) (envelope-from <schmorp@schmorp.de>) id 1WN9A0-00068k-SH; Mon, 10 Mar 2014 22:59:04 +0000
Received: from [10.0.0.1] (helo=cerebro.laendle) by doom.schmorp.de with esmtp (Exim 4.80) (envelope-from <schmorp@schmorp.de>) id 1WN99x-0000gx-Dz; Mon, 10 Mar 2014 22:59:01 +0000
Received: from root by cerebro.laendle with local (Exim 4.80) (envelope-from <root@schmorp.de>) id 1WN99x-0001lw-DS; Mon, 10 Mar 2014 23:59:01 +0100
Date: Mon, 10 Mar 2014 23:59:01 +0100
From: Marc Lehmann <schmorp@schmorp.de>
To: Jacob Davies <jacob@well.com>
Message-ID: <20140310225901.GA6673@schmorp.de>
References: <c19534113ff9489abcc4402fae3c1f62@BL2PR02MB307.namprd02.prod.outlook.com> <CAK3OfOigDS2CizGAtdRaQWbSgiJqw0Ogi-TPkWv35GjPB=CQGw@mail.gmail.com> <CAO1wJ5SLFoUSGoyM4WZa+r2Sf1A_-9e1DmUtRQqfx0UT77VXTA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAO1wJ5SLFoUSGoyM4WZa+r2Sf1A_-9e1DmUtRQqfx0UT77VXTA@mail.gmail.com>
X-PGP: "1024D/DA743396 1999-01-26 Marc Alexander Lehmann <schmorp@schmorp.de> Key fingerprint = 475A FE9B D1D4 039E 01AC C217 A1E8 0270 DA74 3396"
Archived-At: http://mailarchive.ietf.org/arch/msg/json/xVf24w7WumEkDNrdpb8KFY_taUI
X-Mailman-Approved-At: Mon, 10 Mar 2014 16:18:10 -0700
Cc: Barry Leiba <barryleiba@computer.org>, Pete Resnick <presnick@qti.qualcomm.com>, Bjoern Hoehrmann <derhoermi@gmx.net>, Paul Hoffman <paul.hoffman@vpnc.org>, "json@ietf.org" <json@ietf.org>, "Paul E. Jones" <paulej@packetizer.com>, Tim Bray <tbray@textuality.com>, "Joe Hildebrand \(jhildebr\)" <jhildebr@cisco.com>, "Matt Miller \(mamille2\)" <mamille2@cisco.com>, Nico Williams <nico@cryptonector.com>, Larry Masinter <masinter@adobe.com>, "rfc7158@schmorp.de" <rfc7158@schmorp.de>
Subject: Re: [Json] serializing sequences of JSON values
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Mar 2014 23:04:34 -0000

On Mon, Mar 10, 2014 at 11:14:29AM -0700, Jacob Davies <jacob@well.com> wrote:
> Does either the old or new specification say anything about multiple
> values in sequence? I'm pretty darn sure the old version at least said
> that application/json documents contain one and only one value, and
> says nothing about serializing JSON in other formats, so encapsulation
> is not addressed.

Keep in mind that a) JSON existed before it was an RFC and b) that the RFC
says a json document contains only one value is unrelated to this issue,
as one should be able to rely on the format specification.

As JSON is (or was...) self-delimited, and this is a common concept among
serialisation formats, it shouldn't be surprising that some people relied
on it.

> I also don't see how any process that expects to read sequences of
> self-delimited JSON objects and arrays is going to blithely accept
> numbers and strings and whatnot. I find the idea unlikely that there
> is some running code that 1. expects sequences of JSON values with
> no delimiters, 2. uses a parser that precisely followed the JSON
> specification and rejected anything but JSON objects and arrays prior to
> this change, 3. has that parser change under it to accept all kinds of
> values and 4. is so indifferent to the contents of what it is processing
> that it doesn't notice that it's getting numbers and strings instead of
> objects or arrays.

Well, the code exists (making this unlikely idea a certainty), I have
written it, and it is used, and it is in trouble now. I also don't
think it is rare, because this situation arises quite naturally in
non-monolithic designs:

   1. you have some application that exchange json texts, using
   2. a protocol that tunnels json texts, which uses
   3. some generic json coder.

neither of these might know about the other, e.g. because the json-tunnel (2
above) is used to connect applications (1 above) on different hosts for
example.

after upgrading the json coder, apps can silently exchange json texts,
and the protocol that worked fine before might silently corrupt data. and
since the json coder is often a generic component that comes with some
"framework" (likewise 2), this situation is quite likely in practise.

Even if unlikely, given the implications, the RFC should not just have
ignored this issue.

As an implementer (with JSON::XS, which so precisely followed the spec
that it found at least one bug in the official testsuite btw.), I am now
in a fairly bad situation - silently switch to the new JSON format and
push the problem to the users, or force them to enable the new version
manually. JSON::XS did support the "new JSON" format via a switch as an
extension, but applications had to explicitly ask for it, so presumably
know what they are dealing with.

Both solutions (silently switching, or defaulting to "old JSON") obviously
don't appeal to me, and having "new json" and "old json" is such a
horrible concept in itself, especially with JSON, which really shouldn't
be split into two different formats after it has been deployed to such an
extent.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp@schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\