Re: [Cbor] Ideal CDDL for simultaneous CBOR and JSON (was Re: What's your opinion of using CDDL to simultaneously define CBOR and JSON?)

Laurence Lundblade <lgl@island-resort.com> Mon, 20 September 2021 16:47 UTC

Return-Path: <lgl@island-resort.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5A8073A0863 for <cbor@ietfa.amsl.com>; Mon, 20 Sep 2021 09:47:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jLDONFc6TF12 for <cbor@ietfa.amsl.com>; Mon, 20 Sep 2021 09:47:18 -0700 (PDT)
Received: from p3plsmtpa11-06.prod.phx3.secureserver.net (p3plsmtpa11-06.prod.phx3.secureserver.net [68.178.252.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2ED8C3A085E for <cbor@ietf.org>; Mon, 20 Sep 2021 09:47:18 -0700 (PDT)
Received: from [192.168.1.3] ([75.80.148.243]) by :SMTPAUTH: with ESMTPSA id SMRpm4f8q4f29SMRpmXtb3; Mon, 20 Sep 2021 09:47:17 -0700
X-CMAE-Analysis: v=2.4 cv=aJc1FZxm c=1 sm=1 tr=0 ts=6148bb15 a=VPU1mRQhDhA4uSX60JRRww==:117 a=VPU1mRQhDhA4uSX60JRRww==:17 a=yeToiCuq0hHwDWm_:21 a=K6EGIJCdAAAA:8 a=48vgC7mUAAAA:8 a=0XtbOteLAAAA:20 a=gKmFwSsBAAAA:8 a=2OgNEXmPsKLP8B-sVvgA:9 a=QEXdDO2ut3YA:10 a=njWvSBtqtBULLWtTYDoA:9 a=QHhfRY6PuGXzLDgG:21 a=_W_S_7VecoQA:10 a=1TTHW-ywkDfVFKKrZ_wf:22 a=L6pVIi0Kn1GYQfi8-iRI:22 a=w1C3t2QeGrPiZgrLijVG:22 a=nnPW6aIcBuj1ljLj_o6Q:22
X-SECURESERVER-ACCT: lgl@island-resort.com
From: Laurence Lundblade <lgl@island-resort.com>
Message-Id: <F40141D9-F6BE-4056-891D-6514A07C517F@island-resort.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_B10EA964-6C61-4E1F-BE9C-8EF660CDDBE6"
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\))
Date: Mon, 20 Sep 2021 09:47:17 -0700
In-Reply-To: <B97D4B71-0921-4B67-9C7F-27EE25ABB467@island-resort.com>
Cc: cbor@ietf.org, Henk Birkholz <henk.birkholz@sit.fraunhofer.de>
To: Carsten Bormann <cabo@tzi.org>
References: <51537C68-F495-4750-9376-A637BD0E78DD@island-resort.com> <0d4b6a9a-dc26-ee31-725b-3689dfdce041@sit.fraunhofer.de> <4BD93D61-1668-4AE8-B59B-ABC2D3F5C455@island-resort.com> <3822F76A-CAA1-4479-9823-6B5FA0F76BFB@tzi.org> <B97D4B71-0921-4B67-9C7F-27EE25ABB467@island-resort.com>
X-Mailer: Apple Mail (2.3608.120.23.2.1)
X-CMAE-Envelope: MS4xfNTcgmboruFFr6o8ZNun6/fc0TY1GGq8+x+XXy9c+vbnHdsL0tdvLpo4m9kR6lvYBhLHXraoO8t4FxFS/W0rS40zkKM2EbHH/VQ5NAIdgx1NGakA6lKA kwom+0fre3i349FGndEENM5F8Y5yID9S0/ILlob8/+BPeHd1XAl3NfRknxgHJ0Vwq/YRKv0gvi7J9Xzv+QUC5MfHXL4+poBi2n2ZaHM2OtrbfsxdT0uzN6nS gTv3krpB3iZek6/D5N+YNg==
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/Mo71oqOQI-f_gopQt2YC_qQHZ_w>
Subject: Re: [Cbor] Ideal CDDL for simultaneous CBOR and JSON (was Re: What's your opinion of using CDDL to simultaneously define CBOR and JSON?)
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Sep 2021 16:47:24 -0000

Hi folks,

EAT allows one token to be put in another — nested tokens — even a CWT in a JWT in a UCCS… I think there are use cases for this in complicated devices made of multiple subcomponents (e.g., a car) and daisy-chained Verifiers.

I think this is a level of using CDDL=>CBOR/JSON that is beyond SenML and things I’ve seen so far.

Some of this is in the current EAT draft, but it needs work. I am also attempting to have have the cddl tool validate the CDDL and examples as part of the document build process. My plan is to publish a new EAT draft in the next few weeks that describes all this better.

The draft will include CDDL for a claims-set that is common to CWT, JWT and UCCS. That is the CDDL that describes CWT, JWT and UCCS.

We can discuss from there.

LL



> On Sep 10, 2021, at 12:02 PM, Laurence Lundblade <lgl@island-resort.com> wrote:
> 
> Thomas helped get EAT on track so the makefile for the document validates the CDDL in the document against the diag format examples using the cddl tool. It even pulls CDDL files from the SUIT and CoSWID GitHub repository to do that. It’s pretty cool. I’d like to really make it fully work at scale if possible. Here are some thoughts (you probably have had too) that seemed worth writing down and related issues:
> 
> Use CDDL rather than prose when possible
> The main structure for JSON and CBOR should be expressed in CDDL. Use prose to fill in what can’t be expressed in CDDL.
> 
> Understandably CDDL wasn’t the main expression for documents for documents in the past, but now that CDDL is well-established practice it can be the front-and-center main expression (maybe even for XML).
> 
> Explicitly author the CDDL to encode to both CBOR and JSON
> We’re close on this with RFC 8610 Appendix E <https://datatracker.ietf.org/doc/html/rfc8610#appendix-E>, but there are a some intrusive issues:
> 
> One is labels/keys. JSON requires text string. CBOR encoding should really use integers for compactness. My PR for UCCS <https://github.com/ietf-rats-wg/draft-ietf-rats-uccs/pull/6> accommodates this by having separate files for JSON and CBOR labels. I’m not sure I can use Carsten’s PR for UCCS <https://github.com/ietf-rats-wg/draft-ietf-rats-uccs/pull/7> because the integer labels are bound in. I’m not set on the CDDL in UCCS covering JWT, but EAT does somehow require one CDDL expression of a claims-set that is common to CBOR and JSON.
> 
> Binary data. This is straight forward in prose, but I don’t think “cddl spec.cddl validate instance.json” knows what to do with this.
> 
> Tags. Since JSON doesn’t have tags, other JSON structure has to be used to accommodate.
> 
> Some sort of #ifdef or conditionally constructed CDDL for JSON/CBOR variance is the knee-jerk solution, but maybe there is a better solution.
> 
> Put CDDL in a .cddl file and use the make process to pull it into the text
> Don’t put the CDDL in markdown / main text of the document. If it is in a separate file, it can be used in automated ways with the cddl tool.
> 
> Publish CDDL files for reference/inclusion
> Drafts and RFCs should publish CDDL files so that they can be made use of by referring specifications. 
> 
> Github for publishing and curl to make a copy for local use?
> 
> One challenge here is that some referred-to CDDL may be CBOR-only. For example, EAT refers to SUIT and SUIT is CBOR-only. EAT explicitly addresses this in prose, but there’s no good CDDL constructs (yet) to support this, particularly to support verifying a JSON example.
> 
> Validate diag and JSON format examples against CDDL
> The cddl tool is used to verify that any examples in the draft conform to the CDDL in the draft. This is a good check for both the CDDL and the example.
> 
> Support for diag-CBOR-CDDL validation seems pretty good. Less so for JSON.
> 
> 
> In EAT, trying to support this combo of CBOR and JSON seems a bit like going down a garden path. I’ve encountered some wilderness, but haven’t fallen off a cliff, so it seems worthwhile.
> 
> The most important thing for me here is figuring out the what and where for a claims-set in CDDl that works for simultaneously for JWT and CWT. I just don’t see how EAT can be expressed in CDDL without that.
> 
> LL
> 
> 
> 
>> On Sep 9, 2021, at 4:51 AM, Carsten Bormann <cabo@tzi.org <mailto:cabo@tzi.org>> wrote:
>> 
>> OK, I haven’t really answered the original question, and that is a good question.
>> 
>> Before I do, let me reiterate that it is not valuable per se to make a specific application available in all random data formats that come to mind.  Less is more.
>> 
>> But there may be a good reason to span multiple representation formats. SenML is probably the most pertinent example here, as practical use of the format already spans more than a decade, at the start of which XML and even EXI were more popular than today, and it makes a lot of sense to provide smooth backwards and forwards compatibility paths for a format like this that is intended to span rather different (in function and in vintage) processing and storing applications.
>> 
>> CDDL covers JSON and CBOR in RFC 8428; Relax-NG is used for XML in Section 7 of RFC 8428, and the EXI representation variant is also described in W3C XML Schema (WXS) in Section 8 of RFC 8428 (as the efficient use of EXI forces the use of WXS).
>> 
>>> As far as I know, no I-D or standard has defined a protocol solely, directly, and abstractly in CDDL such that it simultaneously specifies CBOR, JSON and other encoding except EAT.
>> 
>> Well, if you strike “and other encoding”, SenML does use CDDL simultaneously for CBOR and JSON.  It does have to provide variant parts in that CDDL spec as the data definition in JSON needs to live with the more limited expressibility in JSON.  More generally, any data definition that wants to use more of the CBOR generic data model than JSON supports will need some extra CDDL to handle that.
>> 
>> draft-ietf-cbor-cddl-control, in Section 4 (Features) has an example that discusses simultaneous specification of JSON and CBOR in passing.
>> But I don’t think we have common idioms or best practices here yet; SenML and the example in cddl-control are the published examples today.
>> 
>>> However, it [SenML] doesn’t entirely answer the question, because the primary definition of SenML is in prose, not in CDDL.
>> 
>> CDDL can only provide the data structure definition.
>> Semantics still needs to be in prose, so I don’t know how “solely” can be achieved.
>> 
>> But, yes, it is nice to see how much of the content of RFC 7071 collapses into a half-page of CDDL :-)
>> (Prose is still needed for the semantics.)
>> 
>>> For EAT, the primary definition of the protocol is in CDDL. There is no prose that says what the CDDL says in many cases. 
>> 
>> That is great (“the grace of being born late” in German politics).
>> 
>>> Note also that while the CDDL RFC does mention use with JSON (https://datatracker.ietf.org/doc/html/rfc8610#appendix-E <https://datatracker.ietf.org/doc/html/rfc8610#appendix-E>), it doesn’t quite say that it is for the simultaneous definition of CBOR and JSON.
>> 
>> No it doesn’t say that explicitly, probably because the target audience of that appendix is people who just want to get their JSON-based protocol done, for which CDDL is quite useful.
>> But RFC 8610 also points to RFC 8428 as an example for CDDL data definitions.
>> 
>>> I’ll ask my CWT/JWT question another way.  What if we had some brand new protocol to define and wanted it to encode in CBOR and JSON, would we use CDDL as the primary expression of it?
>> 
>> Absolutely.
>> When developing the brand new protocol, after finding a reason to actually support both representation formats, we probably would place more attention on making the two variants of the format similar enough that a single piece of CDDL can define both.
>> To me, JWT and CWT are different enough that I don’t see a big reason to squeeze both into a single piece of CDDL.
>> 
>>> Another way to ask this is to ask why SenML didn’t use CDDL as the primary representation. What is some limits in CDDL?
>> 
>> See above.
>> 
>>> Was it easier to do in prose?
>> 
>> See above.
>> 
>>> Lack of a convention to go from CDDl to XML?
>> 
>> Definitely.  
>> Actually, XML came first for SenML (note the ML at the end, which was artfully replaced by “measurement list” during its eight-year gestation :-).
>> 
>>> Didn’t think of it at the time?
>> 
>> Obviously we did think about it: 
>> We do have a common data definition for CBOR and JSON.
>> 
>> Grüße, Carsten
>> 
>> _______________________________________________
>> CBOR mailing list
>> CBOR@ietf.org <mailto:CBOR@ietf.org>
>> https://www.ietf.org/mailman/listinfo/cbor
> 
> _______________________________________________
> CBOR mailing list
> CBOR@ietf.org
> https://www.ietf.org/mailman/listinfo/cbor