Re: [jose] Canonical JSON form

Jim Schaad <ietf@augustcellars.com> Fri, 12 October 2018 16:26 UTC

Return-Path: <ietf@augustcellars.com>
X-Original-To: jose@ietfa.amsl.com
Delivered-To: jose@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A9F9E130DE6 for <jose@ietfa.amsl.com>; Fri, 12 Oct 2018 09:26:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WCBbw5rM7UPP for <jose@ietfa.amsl.com>; Fri, 12 Oct 2018 09:26:26 -0700 (PDT)
Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B923A130DEB for <jose@ietf.org>; Fri, 12 Oct 2018 09:26:25 -0700 (PDT)
Received: from Jude (192.168.1.162) by mail2.augustcellars.com (192.168.1.201) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Fri, 12 Oct 2018 09:21:39 -0700
From: Jim Schaad <ietf@augustcellars.com>
To: 'Bret Jordan' <jordan.ietf@gmail.com>, 'Anders Rundgren' <anders.rundgren.net@gmail.com>
CC: 'Carsten Bormann' <cabo@tzi.org>, "'Manger, James'" <James.H.Manger@team.telstra.com>, 'Kathleen Moriarty' <kathleen.moriarty.ietf@gmail.com>, <jose@ietf.org>, 'Phil Hunt' <phil.hunt@oracle.com>
References: <12DD2F97-80C3-4606-9C6B-03F7A4BF19DE@gmail.com> <CAOASepNX4aYVmPWXyODn0E2Om_rimACPECqJBvZSOXVVd_p8LA@mail.gmail.com> <D21F3A95-0085-4DB7-A882-3496CC091B34@gmail.com> <CAOASepM=hB_k7Syqw4+b7L2vd6E_J0DSAAW0mHYdLExBZ6VBuw@mail.gmail.com> <00ad01d460f4$69ae8a00$3d0b9e00$@augustcellars.com> <8436AEE7-B25A-4538-B8F6-16D558D9A504@gmail.com> <MEAPR01MB35428606C09BF315DE04CC79E5E10@MEAPR01MB3542.ausprd01.prod.outlook.com> <CAHbuEH6DCD7Zc+PK3TnCBkKv1esnROwyCcDb8ZR+TKwgQQ+yXQ@mail.gmail.com> <0E6BD488-74D5-4640-BC31-5E45B0531AFC@gmail.com> <CAHbuEH5oH-Km6uAjrSr0pEHswFBLuDpfVweQ+gpj472yk+8iTQ@mail.gmail.com> <073CB50F-8D91-4EF6-90BE-FC897D557AA6@oracle.com> <A37D69B1-6B77-4E11-8BB9-A0209C77752C@tzi.org> <45bf6c0f-e510-4afc-4277-bdd486a8ce8c@gmail.com> <213796DB-D875-46B0-9F3C-1A56F9E154BA@gmail.com> <ff1dcd4e-2bf4-b85b-dde3-2cc8fe29fb17@gmail.com> <447AB837-7208-4A96-91CC-89D30A2734FA@gmail.com>
In-Reply-To: <447AB837-7208-4A96-91CC-89D30A2734FA@gmail.com>
Date: Fri, 12 Oct 2018 09:26:15 -0700
Message-ID: <017e01d46248$4e040280$ea0c0780$@augustcellars.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_017F_01D4620D.A1A837C0"
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AQNnP1/vuW81sZle9dbx86Tf75xU2AFuBXA/AqJ1qI4BeDi93AGy0bp+AoVGtcgBWQ3ouAHMBZ2PAgPtnXwBomSRnAJU7RiuARLhWd8DID9IqAHgTXhlAXrBW+MCTiSuwKEQJWTA
Content-Language: en-us
X-Originating-IP: [192.168.1.162]
Archived-At: <https://mailarchive.ietf.org/arch/msg/jose/OO_uQbOwvMcolaANCUdjn6iicrk>
Subject: Re: [jose] Canonical JSON form
X-BeenThere: jose@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Javascript Object Signing and Encryption <jose.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jose>, <mailto:jose-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/jose/>
List-Post: <mailto:jose@ietf.org>
List-Help: <mailto:jose-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jose>, <mailto:jose-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 Oct 2018 16:26:30 -0000

 

 

From: jose <jose-bounces@ietf.org> On Behalf Of Bret Jordan
Sent: Friday, October 12, 2018 8:20 AM
To: Anders Rundgren <anders.rundgren.net@gmail.com>
Cc: Carsten Bormann <cabo@tzi.org>rg>; Manger, James <James.H.Manger@team.telstra.com>om>; Kathleen Moriarty <kathleen.moriarty.ietf@gmail.com>om>; jose@ietf.org; Phil Hunt <phil.hunt@oracle.com>
Subject: Re: [jose] Canonical JSON form

 

Please correct me if I am wrong…. The way I understand the problem is as follows:

 

 

1) If you verified the JSON string at consumption time, before it has been unmarshal-ed, then all you need to do is decide how to handle white space and carriage returns.  You could basically regex remove all white space and CR / CRLF and have a workable solution.

 

[JLS] Which is what JOSE does – passes the string in a form that will not be unmarshal-ed.

 

2) Where this breaks down is, when a tool unmarshals the data into a map or struct, then you have no guarantee that you would recreate the keys in the same order (a struct may force it to the order of the struct definition). So you have no way of being able to verify the hash after it has been unmarshal-ed.  Further, if you recreate the JSON and send it back out, the next person that gets the data might have a hash that can not be verified in option 1 above.

 

3) Another problem once you have unmarshal-ed the data is what do you do with JSON numbers.  Some programming languages store them as a float, some as who-knows-what.  So you would need a way to ensure that the number was always stored in the same way, especially for strongly typed systems (is this architecture dependent too?). So the options here are, if the ontology / semantics of the JSON data were well defined in schema (a meaning it was standardized and documented), then the code could know what it should do and interoperability tests could be made to ensure that it worked.

 

[JLS]  The issue is not so much what the format that the number is stored internally in, the problem is how is the number serialized out. Some have dealt with this problem by storing the string version of the number along side the numeric version.  Not what I consider to be an acceptable solution but it may be the only one that works.

 

What am I not understanding here?  And what am I missing?

 

 

Thanks,

Bret

PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."





On Oct 12, 2018, at 12:38 AM, Anders Rundgren <anders.rundgren.net@gmail.com <mailto:anders.rundgren.net@gmail.com> > wrote:

 

On 2018-10-11 22:05, Bret Jordan wrote:



Anders,
I really like what you have done with this.  I am trying to figure out if it will work 100% for my needs, or if it will need some tweaking.  If it does work, then I think we should really try and figure out how we get your work standardized.


Thanx Bret!

The https://tools.ietf.org/html/draft-erdtman-jose-cleartext-jws-01 I-D provides quite a lot of features including an extension option that can be used for adding possibly missing functionality.

There is one thing that is good to know for anyone thinking about standardizing Canonical JSON and that's the fact that canonicalization also can be performed on the text level as described by: https://gibson042.github.io/canonicaljson-spec/

This has the advantage that it is very simple and supports the entire JSON RFC without restrictions.


So why didn't I took this [superficially obvious] route? There are several reasons for that:

A downside of source level canonicalization is that it doesn't integrate with JSON parsers and serializers. https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-01 was explicitly designed to eventually be an option of a standard JSON serializer as it already is in my Java reference implementation.

Another issue is that it is unclear what the value is with using the JSON "Number" format outside of the IEEE range.  In fact, it excludes parsers like JavaScript's JSON.parse() unless JavaScaript would be updated to always use a "BigNumber" as fundamental numeric type.

Regards,
Anders