Re: [Cbor] Deterministic CBOR as a possible DISPATCH item

Laurence Lundblade <lgl@island-resort.com> Sun, 05 March 2023 20:10 UTC

Return-Path: <lgl@island-resort.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16318C14CEE5 for <cbor@ietfa.amsl.com>; Sun, 5 Mar 2023 12:10:31 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BZwNUoFyhtMu for <cbor@ietfa.amsl.com>; Sun, 5 Mar 2023 12:10:28 -0800 (PST)
Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2128.outbound.protection.outlook.com [40.107.92.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A3F36C14EAA3 for <cbor@ietf.org>; Sun, 5 Mar 2023 12:10:28 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R+ozmr2Ypk4Ff0hlQWVevuSpj5Q2nZMXYxxCmB8+lSADVSq7gHS00q8Txl3BcbLOh+3/rg/3u9VzpTeoG1k30a3LhERMYJd6fIEExi1RRaSDhqAIi071LGG9Tmqd03DGCu+OaRo2APmMj2cBOfJPMQtHT9tRETkDpe4glJlDznXRdUhKY1TbwxO2FZiRamOjSND/TTXRlwHTXZqwVZhQVNIC+cRsEUsHPxdKLiBpm7kJfSwPD3cv0W9Ridw/aBbpqraZaXlL89OMR7OYUo1elglkMqQM93fScwuL7f8wl1r7FJdYSgbwJOPStyWLCFUkKZQqAsflrkXPNdNrbtpvOQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bWe8bdGCPT70WZ1puklPJNYpbF11CtXGX6bSMS3NAJk=; b=Tp57K6klLCRRYJaw6sDvWJWotWAJpVa2WeKIgDoEf2JBXzE77MXb8DUyPfRBcvVvg61EMykQX+qHyyw7ei9J4QMmk/cFjZwtoC1/cq2puXDbYBwnTY4dtUygYfySYP3bLiqyf5OQ1YUOPchNKxsrvvv6VXa01liMqsTQRopqzec83tHTALQhkugoB3Kusw2ara0sMBDPgRDgm4xDTuqkxW2CScvN3wR3y2MbBAmT/wra8UvLG4N43iAiHCNEesg1M/PBVVZ/Wk35B6SZe1+uLBObN07wn5c+/9P4ej+kSCljXVL2PeghM95NXhFLHdssujZNDT4URDPj/Kozx210dQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=island-resort.com; dmarc=pass action=none header.from=island-resort.com; dkim=pass header.d=island-resort.com; arc=none
Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=island-resort.com;
Received: from PH7PR22MB3092.namprd22.prod.outlook.com (2603:10b6:510:13b::8) by PH0PR22MB3053.namprd22.prod.outlook.com (2603:10b6:510:141::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.21; Sun, 5 Mar 2023 20:10:25 +0000
Received: from PH7PR22MB3092.namprd22.prod.outlook.com ([fe80::1aae:283a:d7b:3d58]) by PH7PR22MB3092.namprd22.prod.outlook.com ([fe80::1aae:283a:d7b:3d58%3]) with mapi id 15.20.6156.027; Sun, 5 Mar 2023 20:10:25 +0000
From: Laurence Lundblade <lgl@island-resort.com>
Message-Id: <1DA00A88-64DF-48FD-B03E-10B520934DD2@island-resort.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_FA997996-875B-4AD2-9791-A5805E0FAEF8"
Date: Sun, 05 Mar 2023 12:10:23 -0800
In-Reply-To: <FA0E2D22-37F4-4C27-B5F5-E841D13EF0CF@tzi.org>
Cc: Wolf McNally <wolf@wolfmcnally.com>, Anders Rundgren <anders.rundgren.net@gmail.com>, cbor@ietf.org, Christopher Allen <ChristopherA@lifewithalacrity.com>
To: Carsten Bormann <cabo@tzi.org>
References: <A9CF043D-4FA9-48D4-B953-3BE7AA40D1E0@tzi.org> <D25A0C94-ADAD-4C3D-8669-AA7FE9A6B3C4@wolfmcnally.com> <FA0E2D22-37F4-4C27-B5F5-E841D13EF0CF@tzi.org>
X-Mailer: Apple Mail (2.3608.120.23.2.4)
X-ClientProxiedBy: BYAPR01CA0039.prod.exchangelabs.com (2603:10b6:a03:94::16) To PH7PR22MB3092.namprd22.prod.outlook.com (2603:10b6:510:13b::8)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH7PR22MB3092:EE_|PH0PR22MB3053:EE_
X-MS-Office365-Filtering-Correlation-Id: 47433227-15aa-4197-ae87-08db1db5a875
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: WR84o1cNPN5eqWX6X4EU+n9VvYK0Rv8VpP5ToNbFrskBwtqt3FCoGZ2ZwoGNGU5CRDcsrMsdi9YpoPe5dUQA3mSnCY2SeCllVVcI5EkxMLrAnEfb0Rs7y3KW0YJT+FDhec6HEkes78v9pLzxu4vGdoUwAJ0wd8PFB84Zd2uYTXUr4U2MkK8EpUCmEhbPh8PqKmPiAfWUxJJPpALy0hBhnVWHbPac0qG+2mFvxQI7SrvuHRvkycwbagUeELTrvD3COQW8kVZh/AfMjKdEvZ+6LI261NY1pVxkMOLAw0Y5bp6D6Hk00A2uN0pw57gT2JlGmOTK81xJ4bKj6W4xamomBRCsoyNnRswvo8wo3h9ckp2UN9qjGPllUv8MfAbyZOYffa8Fuz+haYc3QYrwdfdB9/+B8g6m570xKqa4ILzxsyt5QclWZUUNNL2Ox9XPJzhqgtqy7mdXP781Hpz9ZDEIqCzx+F4JsKw4ubwZzSOuYRTG69vY4HfK63MASONbhW+MDvEF5LjglGmKLbYYlrW+uJO+/JTCx2r7QF6g/RUCQlr6nY7cGOPRWg3Nc73QStD0i2fsIOp13p/3zIoOSDIgePtC4BWc4ARSR6qUuRrq+4rsiOfFCY3PP23X/2jJxUOpMmIXI4ybN/cIKgsiOuIdUOgJCIUaSfVi9zoC4ksFOBJ8xNaUEX16+MhTwl6r7gVopS938GMUyfrnx+wXXkPjCNKhP3DZKU/ZDCG7wCRTDy4=
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH7PR22MB3092.namprd22.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(136003)(346002)(396003)(376002)(39830400003)(366004)(451199018)(8936002)(5660300002)(66946007)(66476007)(4326008)(66556008)(6916009)(86362001)(8676002)(83380400001)(478600001)(54906003)(36756003)(52116002)(33656002)(41300700001)(316002)(6486002)(186003)(2906002)(166002)(38350700002)(38100700002)(26005)(6512007)(6506007)(33964004)(53546011)(2616005)(45980500001); DIR:OUT; SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: RMwYU97RL+gsaK07f8DMIkDRJBj2p9FBW4pvqbmsUEhKQKbR01bW78/Nqkbzt8FlWUXkzI7YRdu0UGe6Ciwm1bFZ0UqYPvlVcyivSp9yP7n2tehSK2zYXR35aR09r1ANItve3AUvv1xiiOH0EGCDQQzcridv6rDgRSsO8cvmuGvXoVl94pqT/2Nf564+H0/u9xq5UXRnxqkfo/nruY6Rnfwc1eGwJbmfEyHwJiMt7zkCK/aDNaHdNEX6OKnx92jlrC18eKA7kiA+V5jGzMAaT+c9iW23zQ5F5IeB/vUhly2u8NLNjdYmvPIUVLrIdP5mPArVzI+nXMuD9XCmAozT1Siswspx564IgPvP31kB/Gk+i3HXPvA1IZyrupB2Kc8DE3MllGadcTD5UsbImvmTS6AT5Oj1E4wrlnF26gsG4x4x/Xq5lBxeq7V39a0qJbSnOJLFa9qitGIKrFv9LNsolYBAoZOAp4Di+QoAiWyp1YebV+qZ0KNxU/U9kUJscdKD7NeaMNompxAMLgvpwb4NGlB0hyaYzNsxkiXOgrdSFjJgzUGZK84TUB91Hm81n0IJCzoZKaJ3xkEGbEJoLCim/eh11Tuv5CB8FNGErSOSvkACXUMMCNs/odSIpdIJC8kgZlyED7deFWsUlps9lI5gcfxQegd4XSTWi3MYbeJUaBJnm4sue/+RPOgwJRWNVzQfA5cP1fDKblA38E11jvlfnU+2JNp/qD8cD2eiyDO3RthNCAJfJ7pdAFUUBcOYtRxrSbra6T9KJlM0FiZyow+WQbJblJrAHosALA+ZG1guLAQEMaXsM57azAR8040AYKwoHoaGd11lPBb8SN6j2iasG823zkDN/ySI527GCjEdr8tBBFQteUH018w8ue1hdlH5K9W1FuYMb7OLutfrguaPkjH4aGR1BPcYsV1OMA5/XeRfO6JAET3SgEK8oQMCCGlLi3bKf8XrMv8RrXN4bFCuP1+LP/lU+vw1uqzqzrykWfYDIOxMiCYrksls1ifAeTc9ww2SyCzSCoVyCGetuvTzQa8x6sAogq1BkQW3ZGeQVr1E+fWeFAOMSH3EkImqjkjvv5qyPbFJ0DdpQeOcETFzsw+e6Aksq0hzeIFWRi5Gfk4qW4GDrE879dbwUlmogw4V3AOK9b+UhpWRsKASkYubzEW3oQIF/lLBASx1p7H32Ufkf2+/amQPfCGRE+H6MoS1wU7b2l+BbqBbF18meUx5XDU84Q9rm7vjuuWPx72pdxD5f8fotSnV2gzXVMtEoYXKNIrGgyaFAohnZri3aisgr0sazxpqj6bupyJRhCUTkO8DFbIY0nb7to/+R+zcg7lvqVepC2t8qLNM8/ZE6A9aLU7s8pOEzRZnc1lDLYpEy2v755dbtU2lDkd0beWJPBy+vXxXea1uYnaFUf14xs1K2h2yVHXkbNVQBmaOuUMKsj3Sdfi2jRuege9Rk7ZnknWOPJ/IwDga4zHQcmchcxndPYvxMdMuRBwaeST3ccxEfs/LQDCELqdA+BKBrTDAk17Nyt4H1+/pOStiXsv+DiLZQQ3boU2bCVyc7ikG/dHaxlIkY351V3Bu5JOTLTyg/gXSXCOk/Faz/bLTgXK1cELKiQ==
X-OriginatorOrg: island-resort.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 47433227-15aa-4197-ae87-08db1db5a875
X-MS-Exchange-CrossTenant-AuthSource: PH7PR22MB3092.namprd22.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Mar 2023 20:10:25.7815 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: ad4b5b91-a549-4435-8c42-a30bf94d14a8
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sc9S6FRXx0OwtFx131ROhFIlswxiYIWN/qkkWrsxL5/ESK+LCVQFTMtHQICs1aNSs/Hi/OYbgb8CV9DArEBGVA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR22MB3053
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/kHQVyfpBwoffLmxJIvG4nETk1f4>
Subject: Re: [Cbor] Deterministic CBOR as a possible DISPATCH item
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 05 Mar 2023 20:10:31 -0000

> On Mar 5, 2023, at 5:29 AM, Carsten Bormann <cabo@tzi.org> wrote:
> 
>> Determinism is one such good practice when it comes to creating cryptographically enhanced documents.
> 
> Determinism is mostly unneeded in interchanged data items.
> (It is needed for signing inputs, which are therefore best designed to make determinism inexpensive.)
> It still is nice to have, e.g., for test scripts.

I don’t think determinism is required for signing.

When the signed data is transmitted with the signature, then the receiver has the exact bytes that were signed and can verify the signature. For example, determinism is not needed for COSE_Sign payloads.

There are a few rare cases where determinism is needed for signing:

1) The signed data data is not transmitted. It is instead constructed separately by the sender when signing and the receiver when verifying. The Sig_structure internal to COSE is an example of this.

2) The data is transformed in transmit. An example of this email signing like S/MIME because the old email protocols are text-based and change line endings and such. Both the sender and receiver have to be able to construct the same canonical representation because you can’t rely on the transport to not change it. I don’t think this happens when CBOR is transmitted.

So to me, CBOR determinism is technically only needed in rare circumstances.


All that said, I can still see why it might be good to have something like dCBOR standardized in a way that 8949 doesn’t. Sometimes it is nice to be able to do a binary comparison of two encoded structures.The variability in CBOR serialization does seem to cause a lot of discussion and confusion (e.g., in work on EAT). Because of the variability, there isn’t guaranteed interoperability between CBOR implementations. I’d speculate that this is slowing the adoption of CBOR.

I recall always preferring DER to BER, but that was mostly because the difference was confusing, I needed to get my code working quickly and I didn’t want to take the time to understand what it was about and what I’d need to do to support BER in my code.

Another thing here is that the use cases where the CBOR serialization variations are needed are rare and implementing deterministic encoding is not that hard, though sorting map keys can require some memory.

Also, I agree that we want to be strict these days — Postel was wrong <https://datatracker.ietf.org/doc/html/draft-thomson-postel-was-wrong-03>.

LL


P.S. I am not opposed to fixing QCBOR encoding of subnormals. I just have been working on other things. Note also that QCBOR doesn’t use floating point for half-precision, so the fix is a bit more work.