Re: [Json] Regarding JSON text sequence ambiguities (Re: serializing sequences of JSON values)

Nico Williams <nico@cryptonector.com> Fri, 14 March 2014 04:17 UTC

Return-Path: <nico@cryptonector.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FE4F1A000A for <json@ietfa.amsl.com>; Thu, 13 Mar 2014 21:17:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.603
X-Spam-Level:
X-Spam-Status: No, score=0.603 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, IP_NOT_FRIENDLY=0.334, MIME_8BIT_HEADER=0.3, RCVD_IN_BL_SPAMCOP_NET=1.347] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CAprLzX4fmkF for <json@ietfa.amsl.com>; Thu, 13 Mar 2014 21:17:30 -0700 (PDT)
Received: from homiemail-a27.g.dreamhost.com (agjbgdcfdbfh.dreamhost.com [69.163.253.157]) by ietfa.amsl.com (Postfix) with ESMTP id D90B41A0002 for <json@ietf.org>; Thu, 13 Mar 2014 21:17:29 -0700 (PDT)
Received: from homiemail-a27.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a27.g.dreamhost.com (Postfix) with ESMTP id D7EDB59805F for <json@ietf.org>; Thu, 13 Mar 2014 21:17:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cryptonector.com; h= mime-version:in-reply-to:references:date:message-id:subject:from :to:cc:content-type:content-transfer-encoding; s= cryptonector.com; bh=NKIcoAzkCTuJI9NWp9ph34lhOD4=; b=HVlO1H8tqkS rQnaqHvtfl34KbyfsKX4+QHBjIKt5PPCy52biAs+9qMwmWw+upYE5KvOgqgFsys2 1krnZ7DdsiOtl1nmlL3PlkjflBZnKQ0JPmdeZYjxyPTaNKVCJgXrWNyeuvr3fRju CudGsvoXZoEsqhGDAbRMEeu5ZMWeoypo=
Received: from mail-we0-f181.google.com (mail-we0-f181.google.com [74.125.82.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: nico@cryptonector.com) by homiemail-a27.g.dreamhost.com (Postfix) with ESMTPSA id 8C518598057 for <json@ietf.org>; Thu, 13 Mar 2014 21:17:22 -0700 (PDT)
Received: by mail-we0-f181.google.com with SMTP id q58so1594078wes.26 for <json@ietf.org>; Thu, 13 Mar 2014 21:17:21 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=dLkfk3iTBoWM+elW6+TqaZuGNnOz/KETdpbGLNwf194=; b=Xv5e9wkse4zbKVmaaBLRJh/Fn2lc/OMUEgM1X+WpRkKNmtj/Lzmupq4Sy7yMXu2JcY qEugJTzYb0C11XUpo57z9lf/0ZiZE2zPE6sgFc8t6i9SD4TEu64HtMnoe+cMHhOjciUa yXbJgXUgJkB77DItHFOU7K/t5ZZcl3mZW+axQuTqJkg1N/11x0Y48kinJjFD+MWKClWW iDDSmX791p+kJgjNuPwVAikC6p6zDUUmMy/B+Khsuq5QLXHFCB/kS1kQdal3KN3RgjLO Fyt2z/d+B6tYTJwC8nJz2yya7jcdZi6s3rrKz8AW3iyOD+Y+jEKLVaSY7V3z5KliFP4K aS7g==
MIME-Version: 1.0
X-Received: by 10.180.98.35 with SMTP id ef3mr4310617wib.39.1394770641233; Thu, 13 Mar 2014 21:17:21 -0700 (PDT)
Received: by 10.216.199.6 with HTTP; Thu, 13 Mar 2014 21:17:21 -0700 (PDT)
In-Reply-To: <53227E37.9050300@it.aoyama.ac.jp>
References: <CAK3OfOio58+1yuxQOcvWep1CADMfE1PVC48XDid0dWvd8=SVjA@mail.gmail.com> <CAOXDeqoYb=NXz4ikMxAg3EHFA+903bFgdpR_BL-K18U2oYriXQ@mail.gmail.com> <CAK3OfOiPDfWpOZgExTmwwq6WFcuVbyi_z3C0=M9RhQveBhV_+w@mail.gmail.com> <3416a6327a8a4cd3b49bee3c2e548870@BL2PR02MB307.namprd02.prod.outlook.com> <CAK3OfOifggWiLMiTLt8_9rDVd0anrpp+1P2bwvUSJWbaUsVKNQ@mail.gmail.com> <255B9BB34FB7D647A506DC292726F6E115402F019B@WSMSG3153V.srv.dir.telstra.com> <j7j4i9llmc2tfvjnrmllkpjso9hcp0li0i@hive.bjoern.hoehrmann.de> <255B9BB34FB7D647A506DC292726F6E115402F04C6@WSMSG3153V.srv.dir.telstra.com> <CAK3OfOh0_cgArYrTNsWe1ZiXyaam34gtrgWcTs3LJWVvEf+4uw@mail.gmail.com> <53226496.5020806@it.aoyama.ac.jp> <20140314022711.GL1554@mercury.ccil.org> <53227E37.9050300@it.aoyama.ac.jp>
Date: Thu, 13 Mar 2014 23:17:21 -0500
Message-ID: <CAK3OfOh2MUMaVW4Z4SJNoKfteMi7_McUHm0m=jW-uxFQ4ivVZA@mail.gmail.com>
From: Nico Williams <nico@cryptonector.com>
To: "Martin J. Dürst" <duerst@it.aoyama.ac.jp>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: http://mailarchive.ietf.org/arch/msg/json/ZfUhyzlQeFkYKOGHtRrIQ7pXQcs
Cc: "Manger, James" <James.H.Manger@team.telstra.com>, Bjoern Hoehrmann <derhoermi@gmx.net>, John Cowan <cowan@mercury.ccil.org>, "json@ietf.org" <json@ietf.org>
Subject: Re: [Json] Regarding JSON text sequence ambiguities (Re: serializing sequences of JSON values)
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Mar 2014 04:17:31 -0000

On Thu, Mar 13, 2014 at 10:57 PM, "Martin J. Dürst"
<duerst@it.aoyama.ac.jp> wrote:
> The JSON text sequence idea comes from a minor inconvenience of JSON arrays.

Hmm?  For me it comes from jq, and there it's just... natural, nothing
to do with inconveniences of JSON arrays, and everything to do with
the need to have indeterminate-length streams of values.

Exploring that further there's the fact that non-online JSON parsers
can be used to construct JSON text sequence parsers that handle each
text in a non-online manner, but the sequence in an online manner --
this makes a world of a difference when processing large amounts of
data encoded in JSON:

 - One big array + not-online parser -> can't process more than fits
in memory parsed, and can't start until it's all parsed.

 - JSON text sequence + not-online parser -> no limit based on number
of texts, just size of each text.

Needless to say, online parsers can handle huge texts just fine, but
they are very difficult to use.  JSON text sequences are just the
sweet spot for easy of use and performance.

> The benefit of JSON is that it is simple and straightforward, but that's
> apparently lost on most people in this discussion ;-(.

Which is why people want to use it more and more, even for large
datasets, and then you run into the need for an online parser.  Or
JSON text sequencing.

Nico
--