Re: [Cbor] CDDL and b64u for JSON and byte array for CBOR

Oliver Terbu <oliver.terbu@mattr.global> Thu, 25 January 2024 15:38 UTC

Return-Path: <oliver.terbu@mattr.global>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9F962C15107A for <cbor@ietfa.amsl.com>; Thu, 25 Jan 2024 07:38:35 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.105
X-Spam-Level:
X-Spam-Status: No, score=-7.105 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=mattr.global
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n7dnPIgVcSUS for <cbor@ietfa.amsl.com>; Thu, 25 Jan 2024 07:38:31 -0800 (PST)
Received: from AUS01-ME3-obe.outbound.protection.outlook.com (mail-me3aus01on2099.outbound.protection.outlook.com [40.107.108.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0597AC151065 for <cbor@ietf.org>; Thu, 25 Jan 2024 07:38:30 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mdIcCKYJ6dYUI6wmfofxxuBDgAtohJj5jmxvNT1YSBxRK+7LC6PyD+oo2FiNKWiR8a+zSu9PJnS092skV0amRBfkKtWQmvjlMHUY+Gr/nOJhOpGx210Iba0NY9pEGXxB0ZPFIcAvFZKI6wdB/xg0+0Oyhp+IXk1Lrkp18lb0fcfodtq9bExpEX99RxMzLQgYlu7z8bZrO1bOvcBAR3RG7/ppWAYFWgBfClUAmVJOu8WBdlwv5a2SQnt3nlXSrk7C4/H5en6s8Ne8yQdRh8sv+preBBBrctokcySO1wmAm0ZqOlA6jX5H7W+RdPqRFcwrgw7LXV2FcjG1M4oCfxQYiQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=65l75j5+cAEBervK5qWO4cPmUGXTXhncXCGerXd7sWk=; b=G1kW8Q6BO8M1GU0AujiU1QoV8SRYQx005/++mgeGRhBter0GU/HVPzemVD3vZjIhae6ypr8ojD06P8ghNi4rtA5w0lUAiHe0pudQBi+LbbnHf2fg1M8F9GefDOyGbKO29f7oypS/L+pkWW5heq8Fj41YunZfhpQu3NrTE7B1PJ8Dgx9Fgost8KXbPGYmmnBRKMNdtu4fMEy8tyEL9BKXfJrFv4x/74xZf9pR7nCECgGm1Gm0N/2C6VDE5yWUKifH/4tbsSOXzBAYyjvFK0VIWOuH534Z+bZSXn2O6tU/LvRu4O7FU9YSB4ew10lTyAG5bnjgDMhwu2OxUHxzJGLkpA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mattr.global; dmarc=pass action=none header.from=mattr.global; dkim=pass header.d=mattr.global; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mattr.global; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=65l75j5+cAEBervK5qWO4cPmUGXTXhncXCGerXd7sWk=; b=BGi8CjvqTE7MFiBGA2w9H/SNa6jShm2Ox8/AO89QfXJqCelOaYzkGjVIVkT4mCmo6vHJWuKU9jLcLvuYE/X4ggPejjyojlQUheYwnFC5ww6yySy6yDNx5SBur42QzChNR/wGb4sR9crG7lHnvxWhSxNrimoBG6QGDvxceiJvFCw=
Received: from ME3P282MB1347.AUSP282.PROD.OUTLOOK.COM (2603:10c6:220:8e::13) by SY4P282MB4030.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:1c5::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.22; Thu, 25 Jan 2024 15:38:27 +0000
Received: from ME3P282MB1347.AUSP282.PROD.OUTLOOK.COM ([fe80::802b:411c:6b28:d23f]) by ME3P282MB1347.AUSP282.PROD.OUTLOOK.COM ([fe80::802b:411c:6b28:d23f%7]) with mapi id 15.20.7228.027; Thu, 25 Jan 2024 15:38:27 +0000
From: Oliver Terbu <oliver.terbu@mattr.global>
To: Carsten Bormann <cabo@tzi.org>, Oliver Terbu <oliver.terbu=40mattr.global@dmarc.ietf.org>
CC: "cbor@ietf.org" <cbor@ietf.org>
Thread-Topic: [Cbor] CDDL and b64u for JSON and byte array for CBOR
Thread-Index: AQHaTwFyBn4cVGjsYkeleznd79DbCbDqhxEAgAAimLw=
Date: Thu, 25 Jan 2024 15:38:27 +0000
Message-ID: <SY4P282MB1355DF5106963E2B0C0803A5F07A2@SY4P282MB1355.AUSP282.PROD.OUTLOOK.COM>
References: <ME3P282MB1347A5F2577BB44F8DA14C2BF07B2@ME3P282MB1347.AUSP282.PROD.OUTLOOK.COM> <CDE67E86-4CEF-4433-942E-2CB065C52054@tzi.org>
In-Reply-To: <CDE67E86-4CEF-4433-942E-2CB065C52054@tzi.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=mattr.global;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: ME3P282MB1347:EE_|SY4P282MB4030:EE_
x-ms-office365-filtering-correlation-id: af115551-c91c-4045-255b-08dc1dbbac8f
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: YWMCJq+vpcg6zX+CMYEFc0fal5BycMLsM/I70tfLOi8ccy0w/o1Uh6QqsFbwZ1yCed+dNvfHEU+0PKNiZQPduYrCnzLVFZ6yVKUOS6vYjGat4sCDN0MHEdkVKQjaHCl1mEp7UJR7zBY21/AHTIlOM+G9L2XG/+ZlRYO1JfnHI8kGrfzOB5dGXPVnr9LLJ3M26AeyeuD0d62K9UzBAPj8KBNuK75OrcGSny8hwaH0HW4XwFC8Um/bylWNjOXCiXKN1ls6PlGH2VM5ZIGi94xjpcY5ncXHvWmYua+n7byIvHqiPDyb9E8dFUXRuzF02494v2blqYaa6SC0sK0fy6ziBmlyrWUavWYNmaC7KaLI7DxpSTQXmqgdL0Oz2zOoprCEp/4Kd0sX1Wf8ypfwQqYPAQ/po0b51KAc8N3CPlM4xSelJcmq3lCiv06cSDGBHX61rDXAmRdsVGNz949F3Q36bYwLjW42dTi+XxgvNZexGOFTxk1CFxB3FPO/saTVTMNlGcgQ077UGy5f5nemUpv0bp+mefGY1pKGSo2rgPiKZMNv7tMZeXbSNZu/S+Bran6AJxlOvl+oX8wHL5kOjPgDTs5Xe+iqQegHNVpK+mqJtAdWIPXYoDVYCSjKdqEH99XooxcfT53MNMCeOw+4z8BkYg==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:ME3P282MB1347.AUSP282.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230031)(396003)(346002)(136003)(376002)(39840400004)(366004)(230273577357003)(230173577357003)(230922051799003)(1800799012)(186009)(451199024)(64100799003)(66476007)(6512007)(91956017)(6486002)(52536014)(38100700002)(122000001)(6506007)(26005)(4326008)(966005)(478600001)(8936002)(2906002)(66446008)(5660300002)(44832011)(8676002)(53546011)(110136005)(71200400001)(66946007)(316002)(76116006)(166002)(64756008)(66556008)(41300700001)(9686003)(86362001)(33656002)(38070700009)(19627405001); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: UN7e3x/Ivan9C21rawMC65ef/uJH2ZIcRGdyFto2xxtKiKjeH5AWDb2/fRQEjskXE1mnXz2C2fpiqp/QYR1a/cv1qh2GCNVImWh+Y2QqBYqPkiqO4TyIZmS9ZlDGVrF71eqDj2QKSeEd+mBDUVJMrvR4C6FR1jL+NwW8/35EY0LvyuExygh6YA+IH3tCP2QXsE+G9lQfzv2WRLZGomYEgtdWh2+dolHjJBmPNJnkdBdxD5kJpIjE9oggV4ZHEV/6oPY8I8CvCPMb6xnEIPrGg1px4pJXg+WJaL816LBaFPGXWjGGXmmU1aAOSlCVB0hcLTdc8XRLr9vodSKVsLs2vZMTbuuhjdkOFQ+5x4Q6qQPuKoPeFrntJlDamlyMkhXvhia55TO5q44kgraJCtqJtYclvG9+lcBLZMvqQjfPV2P6wkIpToK6fyHL29vdnFvvVcdytyV7/ckWTymlX1ciUI24IlO3LhCv+YLDX/bes+BniC0+pXwL3wynHYP+4osM5o54/MpF0qe5sX7SDQBKMLD7MWuadrAJlCZeYEwI1vG8BVqY4cygjIeawhKQm072Buy0awDV33WYTTqnMqj93Yd5je5++IHOv0Sw/lTTm5cq0vXpcrmvujTu9zC8uHCIy4VbBC6l2RkWI9yyRSEiIimPQfKD01dsaaduQagbt6Sbdaev8NWvdxUhuZs1CPfcE4cOggm6++6iumYK4mmcKL9bi5GwPc6aSc7RXx7ONjzhiNCb942KjYEXsERRNgruKD4XE9UvhDx0y0/tlDwp/Mos/JTmfL7NWja+xesygxRExF332XRUv1tJOqTbXTAT+N43Bi8pir0sFoBZXJPIWLY4IPAP4liXUxe5CLCVAAk6HSS+B5jZ6+rHDPw7pxyweYl2Kl3JJf1dq9boj9JFXav48WE1vnt370mwerj7S2Ura0mvXUZqYkBoVWAmhzisXsNluBMaOYjPHs/qhVDY6YKax2wWQLoieccaM20MLJWMKha/FXMkfb6Ul7Z+48PCglJXRkCxxEgK4jXDOaa2HqD0jCN6HhTpMVsDmP1/pus+Ui3o3xL7ZlrKuM/8RTnkT0yJmGExB3uEqqfXtl0o4qShe/+M38coiN/gEjPMWUsuQ8XVqGCYwfCvYEYqBxi1C+4PpMoZXGwvFwztPlASSt7TZOiAWGxFBF7+6BKi8w7I7gSZou7L8Sk1LaKKDggzJsObPGuU6JZ3juAaY5vD2qZIavbLD2CgxqkS7LMwWG7VhccAjvwOx2ZCPjAbsm2YhzAE0iwtb2APl8svFK56so677OirBKvxRT98ZqBYzV+fgxuhaE4pYhoGNLDGOEBDBQ5dm7RiGRGYY/51eDHuA0336A2klWVhDEoYb1IF9mRwlPFRSYVzHX29HFO3mCltG8qdqbM6JTCOHUfawrz6bJriH7CasBP12AmfO/DxTcNoxVj5WJkqIALHm3jsGa+KSANhXAuC8YeNUCNarnhy3pUKZFF9M6MWyVLjqr46WCTSzeIvIHQxtMgtRiBXiDUbY+2+THfsfUOy6K/OvpWJeO/AV09TcXhhoGGU47RFpn3eFbVZsA6F/igR84SWGSUw
Content-Type: multipart/alternative; boundary="_000_SY4P282MB1355DF5106963E2B0C0803A5F07A2SY4P282MB1355AUSP_"
MIME-Version: 1.0
X-OriginatorOrg: mattr.global
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: ME3P282MB1347.AUSP282.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-Network-Message-Id: af115551-c91c-4045-255b-08dc1dbbac8f
X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Jan 2024 15:38:27.0681 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: c2c9cf73-6aae-4702-9844-02adab723771
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: MrCV/bZNJq8kBQc6l0eOf4nHdhPcVnGb8x0uIv0qHWmzGKz8F10VARghKrsHBOyp/8csoQdMM/osxGPUGmw0/PQf2XmC6jlf50M1qce9TC0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SY4P282MB4030
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/yY-p2vbZ4wzFTOiQaaP7dSI43ec>
Subject: Re: [Cbor] CDDL and b64u for JSON and byte array for CBOR
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 25 Jan 2024 15:38:35 -0000

I was not aware of the .feature control operator. Thank you very much for that guidance and the elaborate examples. This is what I was looking for.

Thanks,
Oliver
________________________________
From: CBOR <cbor-bounces@ietf.org> on behalf of Carsten Bormann <cabo@tzi.org>
Sent: Thursday, January 25, 2024 2:29 PM
To: Oliver Terbu <oliver.terbu=40mattr.global@dmarc.ietf.org>
Cc: cbor@ietf.org <cbor@ietf.org>
Subject: Re: [Cbor] CDDL and b64u for JSON and byte array for CBOR

EXTERNAL EMAIL: This email originated outside of our organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.


Hi Oliver,

> On 2024-01-24, at 21:14, Oliver Terbu <oliver.terbu=40mattr.global@dmarc.ietf.org> wrote:
>
> Hi,
>
> I have a CDDL for a data model with potential serialization to JSON/CBOR and I have map entry that could be either a byte array or a base64url-encoded text representation of the byte array.
>
> Some = {
>   "key": #6.21(bstr)
> }
>
> Does the CDDL above express that correctly?

The CDDL expresses that the value under “key” needs to be a tag 21 on a byte string.  It does not directly allow the base64url-encoded text representation that JSON would need.

The purpose of the tag 21 would be to "indicate that a byte string might require a specific encoding when interoperating with a text-based representation” (Section 3.4.5.2 of RFC 8949 [1]).

[1]: https://www.rfc-editor.org/rfc/rfc8949.html#name-expected-later-encoding-for

It goes on to say:

  "These tags are useful when an encoder knows that the
   byte string data it is writing is likely to be later converted to a
   particular JSON-based usage.”

So the assumption here is that a processor that converts the CBOR to JSON can do this without knowing the CDDL data description, based on the mere presence of the tag 21.

If you can assume that the processor will be aware of the data definition, you can put a description both for a CBOR representation and a JSON representation into a single piece of CDDL, e.g., by using

  JC<J,C> = J .feature "json" / C .feature "cbor"

from RFC 9165, combined with .b64u from draft-ietf-cddl-cbor-more-control:

  binary = JC<text .b64u bstr, bstr>

or, more generally:

  binary-data<D> = JC<text .b64u D, D>

so you can be more specific what those data D is.
By relying on the processor’s awareness of the data definition, you save one byte that is needed for actually encoding tag 21 otherwise.

Examples that the CDDL tool generates for JSON and CBOR, respectively, are shown below.

Grüße, Carsten

$ cat jc.cddl
binary = JC<text .b64u bstr, bstr>
JC<J,C> = J .feature "json" / C .feature “cbor"

$ CDDL_FEATURE_OK=cbor,^json cddl jc.cddl gp 10
/binary/ h'717569766572697368'
/binary/ h'656E646F7374796C6172'
/binary/ h'6D616C6C6F736569736D6963'
/binary/ h'73757065726571756976616C656E74'
/binary/ h'706F726B776F6F64'
/binary/ h'44696374796F736970686F6E'
/binary/ h'4672616E636F6E69616E'
/binary/ h'73636C65726F7469636F63686F72696F696469746973'
/binary/ h'6174696C74'
/binary/ h’657061676F676963'

$ CDDL_FEATURE_OK=json,^cbor cddl jc.cddl gp 10
/binary/ "c25haWw"
/binary/ "VGV0cmFkeW5hbWlh"
/binary/ "aW50ZXJmZXJl"
/binary/ "cHJlY2F2YWw"
/binary/ "Y291bnRlcnN0YXRhbnQ"
/binary/ "ZW1lcmdlbmN5"
/binary/ "c2xlZXBlcmVk"
/binary/ "aG9ybndvcnQ"
/binary/ "cGhhcm1hY29tZXRlcg"
/binary/ “c3VwcmFzcGluYXRl"

_______________________________________________
CBOR mailing list
CBOR@ietf.org
https://www.ietf.org/mailman/listinfo/cbor