Re: [Json] [Acme] Signed JSON document / Json Content Metaheader / JSON Container

Nico Williams <> Wed, 28 January 2015 22:43 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id C41C41A1AD3; Wed, 28 Jan 2015 14:43:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.666
X-Spam-Status: No, score=-1.666 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, IP_NOT_FRIENDLY=0.334, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 6nVJaxwQW4og; Wed, 28 Jan 2015 14:43:50 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id B7CA71A1AB9; Wed, 28 Jan 2015 14:43:50 -0800 (PST)
Received: from (localhost []) by (Postfix) with ESMTP id 7F2BF2005E609; Wed, 28 Jan 2015 14:43:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed;; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to;; bh=iHi19S9EBgDxe8 fxfILPQABaB9M=; b=gyvPrQamHqg9DSW6YvzZQ+ZID3VEiUWDhBy0MDF0ShJhng 00HdW6JoDE+mEf7zc1IREkDGsAac/fM9EkJ8GlSZSh9wS+2niOGCqKRhLrrLDfKD wC+J/wHF6nxaHHM5uslM+Y/U520OmrHdC18b1/tRIWTIm9rP8uf6wUHEYPaUg=
Received: from localhost ( []) (Authenticated sender: by (Postfix) with ESMTPA id 21F3E2005E607; Wed, 28 Jan 2015 14:43:50 -0800 (PST)
Date: Wed, 28 Jan 2015 16:43:49 -0600
From: Nico Williams <>
To: Phillip Hallam-Baker <>
Message-ID: <20150128224346.GF3110@localhost>
References: <>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <>
User-Agent: Mutt/1.5.21 (2010-09-15)
Archived-At: <>
Cc: "" <>, JSON WG <>
Subject: Re: [Json] [Acme] Signed JSON document / Json Content Metaheader / JSON Container
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 28 Jan 2015 22:43:52 -0000

On Wed, Jan 28, 2015 at 01:14:58PM -0500, Phillip Hallam-Baker wrote:
> <JSON-BLOB> [Separator] <content-blob>

I'll bite.

Why not:

<metadata JSON> <separator> <signature> <content>

> Advantages:
> * Can read the metadata for a file etc with a plain ASCII editor or command
> line tools like cat, more, etc.
> * Avoids the need to BASE64 armor the content. So if the content is JSON or
> other ASCII/Unicode text it remains readable in an ASCII/Unicode editor.

Why even have to base64-encode the signature?

> * Can add signatures, digests and other metadata items to content in a
> simple regular fashion.

Ah.  Hmm, right!  It'd have ot be:

<metadata JSON> <separator> <signature> <content> [<extra signatures>]


<metadata JSON> <separator> <content> [<signatures>]

or as you suggest, put the [base64-encoded] signatures in the metadata.

> Right now, the IETF spec for describing content metadata is based on
> RFC822. And by 'based on', I mean that it is a find the needle in the RFC
> haystack approach. RFC822 style content headers as currently used have
> several disadvantages:
> * It is a flat structure, headers can have attributes but trying to add
> more structure is painful.

Flat structure has its advantages.

A while back I posted a list of kinds of encodings, asking if there was
any I'd missed.  I did miss one important one myself: Lustre's RPC.

Lustre's RPC doesn't support seqeuences-within-sequences-within-
sequences like all the others do.  It's more like a single top-level
sequence of fixed- and variable-length things, where the fixed-length
ones contain C-like structs (with all-fixed-length fields), and the
variable-length ones containing only things like filenames.  (Bulk data
is sent separately, via RDMA-like protocols.)

The main advantage to this is that all the critical length checks go in
one place in the code.

Whereas a JSON blob is not flat.  It could be arbitrarily deep.

> * It is a historical artifact and content metadata headers are mixed in
> with protocol metadata headers without rhyme or reason.
> * It is not JSON and JSON is the spec we are now converging on for this
> type of work. It has all the expressive capabilities of XML and ASM.1
> without the insanities and stupid. It is as easy to read and write as
> RFC822 but without the limitations.

Meh.  It's just one of many suitable encodings.  Pick one, any one.

> The spec itself would be simple, its
> basically draft-ietf-json-text-sequence applied to a sequence of one json
> object and one content blob.

Pretty much, yes (but it's not the same: the blob would not be a JSON

> JSON sequence specifies that "A JSON text sequence consists of any number
> of JSON texts, all encoded in UTF-8, each prefixed by an ASCII Record
> Separator (0x1E), and each ending with an ASCII Line Feed character (0x1A)"
> This is not ideal for JSON Container. We would prefer that the character
> which is illegal in a JSON document be the one that is the separator
> between the header and content.

Just make it:

RS JSON-blob LF FS content

(That would be as close as this could get to being a JSON text
sequence while also being sufficiently different.)

> Applying this in a Web Service is very simple, our messages now have the
> form:
> POST / HTTP/1.1
> Host:
> Content-Type: application/json-container
> Content-Length: 666
> { "Signature" : "wefwkjefkljwehfjklwhejkflh" }
> <0x1E>{ "Service-Type" : "",
>    "Transaction-ID" : "2h23roih23oih23orh",
>    "Register" : { ....<web service parameters here> ... } }

OK, but why not put all of this into the headers anyways?

> Notice how we have just developed a format that allows us to sign a
> complete code distribution including content data very easily. This is
> obviously out of scope for ACME but the fact that an approach transfers so
> neatly to a completely different problem suggests that it is the right
> track.

Wasn't JOSE about this sort of thing?