Re: [jose] canonical JSON

"Manger, James H" <James.H.Manger@team.telstra.com> Tue, 19 February 2013 06:58 UTC

Return-Path: <James.H.Manger@team.telstra.com>
X-Original-To: jose@ietfa.amsl.com
Delivered-To: jose@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4352F21F8D02 for <jose@ietfa.amsl.com>; Mon, 18 Feb 2013 22:58:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.9
X-Spam-Level:
X-Spam-Status: No, score=-0.9 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, HTML_MESSAGE=0.001, RELAY_IS_203=0.994]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OZVdLMU-Jn6M for <jose@ietfa.amsl.com>; Mon, 18 Feb 2013 22:58:05 -0800 (PST)
Received: from ipxcvo.tcif.telstra.com.au (ipxcvo.tcif.telstra.com.au [203.35.135.208]) by ietfa.amsl.com (Postfix) with ESMTP id 5446121F8CED for <jose@ietf.org>; Mon, 18 Feb 2013 22:58:02 -0800 (PST)
X-IronPort-AV: E=Sophos; i="4.84,692,1355058000"; d="scan'208,217"; a="120269462"
Received: from unknown (HELO ipcavi.tcif.telstra.com.au) ([10.97.217.200]) by ipocvi.tcif.telstra.com.au with ESMTP; 19 Feb 2013 17:57:58 +1100
X-IronPort-AV: E=McAfee;i="5400,1158,6990"; a="164713982"
Received: from wsmsg3755.srv.dir.telstra.com ([172.49.40.196]) by ipcavi.tcif.telstra.com.au with ESMTP; 19 Feb 2013 17:57:58 +1100
Received: from WSMSG3153V.srv.dir.telstra.com ([172.49.40.159]) by WSMSG3755.srv.dir.telstra.com ([172.49.40.196]) with mapi; Tue, 19 Feb 2013 17:57:57 +1100
From: "Manger, James H" <James.H.Manger@team.telstra.com>
To: Richard Barnes <rlb@ipv.sx>
Date: Tue, 19 Feb 2013 17:57:56 +1100
Thread-Topic: [jose] canonical JSON
Thread-Index: Ac4OAKN7hJhygQ27TUCi1SOcjb5NTQAV59LA
Message-ID: <255B9BB34FB7D647A506DC292726F6E1150757902D@WSMSG3153V.srv.dir.telstra.com>
References: <CAG8k2+4xaAUBPs=Kw-=eBHZNyOMs6VYByPEb1jnAv1aGjLupng@mail.gmail.com> <CABkgnnWzdoo6b0ZymF0cv_v9zOjJKTWuUhkWuxiA-cM9qgu0jg@mail.gmail.com> <CAG8k2+47GQXHhWBdqd82UEAPZUfAigYE-vwxpaMJm4F5i8098A@mail.gmail.com> <CAL02cgQ3Oh1D9qHW7XWAZqzmfnE5T6-FjNydjpMEMhaHf2d7Xw@mail.gmail.com>
In-Reply-To: <CAL02cgQ3Oh1D9qHW7XWAZqzmfnE5T6-FjNydjpMEMhaHf2d7Xw@mail.gmail.com>
Accept-Language: en-US, en-AU
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US, en-AU
Content-Type: multipart/alternative; boundary="_000_255B9BB34FB7D647A506DC292726F6E1150757902DWSMSG3153Vsrv_"
MIME-Version: 1.0
Cc: jose <jose@ietf.org>
Subject: Re: [jose] canonical JSON
X-BeenThere: jose@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Javascript Object Signing and Encryption <jose.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jose>, <mailto:jose-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/jose>
List-Post: <mailto:jose@ietf.org>
List-Help: <mailto:jose-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jose>, <mailto:jose-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Feb 2013 06:58:06 -0000

A canonical form of JSON might be fairly easy, but the one you quote (http://wiki.laptop.org/go/Canonical_JSON) can’t handle floating point numbers (or very large integers), and produces invalid JSON if a string includes a tab! Fix those (escaping control chars [\u0000-\u001f]; use normalized scientific notation for numbers) and it might be worth considering.

Defining JOSE calculations in terms of 1 or more byte arrays, the first of which is a UTF-8-encoded JSON header, would be useful. It can then be packaged as dot-separated base64url-encoded segments to be HTTP-header-friendly, or packaged as a single JSON object to be programmer-friendly, or packaged as raw bytes to be efficient.

--
James Manger

From: jose-bounces@ietf.org [mailto:jose-bounces@ietf.org] On Behalf Of Richard Barnes
Sent: Tuesday, 19 February 2013 4:52 AM
To: Daniel Holth
Cc: Martin Thomson; jose
Subject: Re: [jose] JWK-specific key fingerprints?

Citation for Canonical JSON:
<http://wiki.laptop.org/go/Canonical_JSON>

I have implemented this in python.  It is surprisingly easy [1].  Canonicalization has been anathema to this group, but I suspect largely because people have been bitten by XML canonicalization, rather than because JSON canonicalization has been tried and found difficult.

If we're going to go down this path, I would prefer Canonical JSON to bencode, if only because bencode is an entirely different format to implement.

Personally, given how easy canonicalization is to implement (it actually requires fewer lines in my experience than base64!), it seems like a sensible choice in general.  In particular, for the JSON serialization, it would save some unnecessary base64 encoding that is currently required.  I would be glad to contr

--Richard

[1] My own canonicalization, not sure if it matches the one linked above <http://pastebin.com/ptUfn0c3>


On Tue, Feb 12, 2013 at 3:01 PM, Daniel Holth <dholth@gmail.com<mailto:dholth@gmail.com>> wrote:
On Tue, Feb 12, 2013 at 2:53 PM, Martin Thomson <martin.thomson@gmail.com<mailto:martin.thomson@gmail.com>> wrote:
On 12 February 2013 10:56, Daniel Holth <dholth@gmail.com<mailto:dholth@gmail.com>> wrote:
> ... canonical json ....

Isn't mention of canonical JSON a swear-jar offense?  Something needs
canonicalization, but JSON seems a poor candidate.

There is something called Canonical JSON. They take out all the whitespace, sort keys lexicographically, and only quote the " inside strings. It is used in the wild by the OLPC project I think.
bencode is an easier to implement protocol that is used in bittorrent and yields a tiny write-only implementation if all you want to do is get unique hashes for the subset of JSON used by JWK.

_______________________________________________
jose mailing list
jose@ietf.org<mailto:jose@ietf.org>
https://www.ietf.org/mailman/listinfo/jose