Re: [Json] Using a non-whitespace separator (Re: Working Group Last Call on draft-ietf-json-text-sequence)

"Manger, James" <James.H.Manger@team.telstra.com> Thu, 05 June 2014 06:25 UTC

Return-Path: <James.H.Manger@team.telstra.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8419A1A0331 for <json@ietfa.amsl.com>; Wed, 4 Jun 2014 23:25:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.202
X-Spam-Level:
X-Spam-Status: No, score=-0.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, RCVD_IN_DNSWL_NONE=-0.0001, RELAY_IS_203=0.994] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XnoxQoYvnQqK for <json@ietfa.amsl.com>; Wed, 4 Jun 2014 23:25:30 -0700 (PDT)
Received: from ipxbvo.tcif.telstra.com.au (ipxbvo.tcif.telstra.com.au [203.35.135.204]) by ietfa.amsl.com (Postfix) with ESMTP id E069F1A02F8 for <json@ietf.org>; Wed, 4 Jun 2014 23:25:28 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="4.98,978,1392123600"; d="scan'208";a="217000968"
Received: from unknown (HELO ipcavi.tcif.telstra.com.au) ([10.97.217.200]) by ipobvi.tcif.telstra.com.au with ESMTP; 05 Jun 2014 16:25:03 +1000
X-IronPort-AV: E=McAfee;i="5600,1067,7459"; a="284387393"
Received: from wsmsg3757.srv.dir.telstra.com ([172.49.40.85]) by ipcavi.tcif.telstra.com.au with ESMTP; 05 Jun 2014 16:25:03 +1000
Received: from WSMSG3153V.srv.dir.telstra.com ([172.49.40.159]) by wsmsg3757.srv.dir.telstra.com ([172.49.40.85]) with mapi; Thu, 5 Jun 2014 16:25:03 +1000
From: "Manger, James" <James.H.Manger@team.telstra.com>
To: Nico Williams <nico@cryptonector.com>, Phillip Hallam-Baker <ietf@hallambaker.com>
Date: Thu, 05 Jun 2014 16:25:01 +1000
Thread-Topic: [Json] Using a non-whitespace separator (Re: Working Group Last Call on draft-ietf-json-text-sequence)
Thread-Index: Ac+AMuq+MLrk5FTwQ6i0mvLUdD7wdgATJ9PQ
Message-ID: <255B9BB34FB7D647A506DC292726F6E11546B21D22@WSMSG3153V.srv.dir.telstra.com>
References: <CAK3OfOidgk13ShPzpF-cxBHeg34s99CHs=bpY1rW-yBwnpPC-g@mail.gmail.com> <CAHBU6itr=ogxP4uoj57goEUSOCpsRx1AXVnW1NQwSTPxbbttkw@mail.gmail.com> <CAK3OfOhft+XJeMrg5rdY9E6fxAkJ2qsT3UHwu7zt=NEz2Q3XOQ@mail.gmail.com> <CAK3OfOhy-N0zjCVxtOMB8SqZEKceVvBz9Y6i0fo2W8i+gHKm4Q@mail.gmail.com> <CAK3OfOiQnLq29cv+kas3B8it-+82VmXvL3Rq1C5_767FDhBjRg@mail.gmail.com> <03CFAB3E-F4C6-4AE8-A501-8525376C4AA7@vpnc.org> <CAK3OfOja-17V391tTK91R98X8XQzd0iPnur2=oo4ii+MCOt+Rg@mail.gmail.com> <CFB42410.4EDDC%jhildebr@cisco.com> <CAMm+Lwime-=UQPu3t2ty05CZLb7xUMi9KGi31Xi2B7RNF5S3Og@mail.gmail.com> <CAK3OfOg_k4Ngq+z1pn4b+XRf0M1Hqx8qZ9BtW0sa8QQ+bjKJyA@mail.gmail.com> <084664DB-A55D-465E-8888-97BA0BB59637@vpnc.org> <CAHBU6itEph5GzB-P8bUUvUMopRNxcCE-16qys7ofhdmsDvpN4w@mail.gmail.com> <CAMm+LwjoeC1R4O2iCPo+RfUFn4Qca4zyytqa817ayH60mNaWLg@mail.gmail.com> <CAK3OfOhjPZUXK6C0qSsQQZvOgR3Sv3SWpyH=qTuihuDC9uvXrA@mail.gmail.com>
In-Reply-To: <CAK3OfOhjPZUXK6C0qSsQQZvOgR3Sv3SWpyH=qTuihuDC9uvXrA@mail.gmail.com>
Accept-Language: en-US, en-AU
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US, en-AU
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Archived-At: http://mailarchive.ietf.org/arch/msg/json/Px3NVp4COpedoTJZAUZHedQb3fM
Cc: Tim Bray <tbray@textuality.com>, Paul Hoffman <paul.hoffman@vpnc.org>, Joe Hildebrand Hildebrand <jhildebr@cisco.com>, IETF JSON WG <json@ietf.org>
Subject: Re: [Json] Using a non-whitespace separator (Re: Working Group Last Call on draft-ietf-json-text-sequence)
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 05 Jun 2014 06:25:31 -0000

>> JSON-sequence = *( ws %1e JSON-text )

RS as a JSON sequence prefix or separator was a bad idea when discussed a month ago and still is.

* You cannot (easily) enter an RS in notepad.
* You cannot (easily) enter an RS in vi.
* You cannot see an RS.
* An RS causes Chrome to treat a file as binary data, instead of text.
* Cut-n-paste a JSON value with an invisible RS prefix and the result is NOT JSON, ie it will fail with a JSON parser as RS is not allowed in JSON.
* No one uses RS.
* RS is now labelled INFORMATION SEPARATOR TWO, not RECORD SEPARATOR.
* We aren't using INFORMATION SEPARATOR ONE, THREE or FOUR.
* A newline as a JSON value terminator is sufficient to parse a JSON sequence unambiguously.
* RS doesn't work well with APIs that read text by the line.
* Detecting a newline that separates JSON values is more complex than detecting an RS character, but it is not that complex (eg handful of lines of code).
* An RS prefix detects only slightly more cases of accidentally truncated writes (in the middle of a top-level number, in a top-level string in the middle of an escape sequence) -- not enough to be compelling.
* The awkwardness of RS will mean many implementations will be lenient, but leniency becomes "expected" which leads to interop problems.

"A JSON sequence is the concatenation of zero or more JSON values, where each JSON value is terminated with a newline."

Simple to understand. Simple to write. Simple enough to parse. Simple enough to resync from the middle of a sequence. Almost identical recovery from accidental corruption is possible in almost all the same instances regardless of whether an RS prefix or newline suffix is used.

--
James Manger