[Cbor] New IETF Internet-Draft draft-mcnally-deterministic-cbor-00
Christopher Allen <christophera@lifewithalacrity.com> Fri, 10 March 2023 03:28 UTC
Return-Path: <christophera@lifewithalacrity.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 56EAAC169522 for <cbor@ietfa.amsl.com>; Thu, 9 Mar 2023 19:28:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.896
X-Spam-Level:
X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lifewithalacrity-com.20210112.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4e5ZbQGwyNvd for <cbor@ietfa.amsl.com>; Thu, 9 Mar 2023 19:28:15 -0800 (PST)
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BDB17C169539 for <cbor@ietf.org>; Thu, 9 Mar 2023 19:28:15 -0800 (PST)
Received: by mail-ed1-x534.google.com with SMTP id s11so15222326edy.8 for <cbor@ietf.org>; Thu, 09 Mar 2023 19:28:15 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifewithalacrity-com.20210112.gappssmtp.com; s=20210112; t=1678418893; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=k9ZRvPABBk2P/l3glpc+wZa31EczS4aYNgyWqJ35ZUQ=; b=0JR+NjSJn9KhfF1GNBWRtEeCE47STGlBLxZUIisdcYblUsNSardFol3pI+ueWOiF28 3D/K2hyDBgXeBwt+PrvslZi4wbXLMsYnb/qipp+LS4CPvpqFYFz6Y5iKaA6Znoee5K/J PB0rMVf99mnKaHekA40JFnSPGHk8bRgub5DwhTrAOYg0JODbsFfnlwPFigVUJb8xQXw/ f+Lkmg3ILlPcdjlefU3GTnJmaSvXHlwYk57B6JzogBoy13uH8ydBnhLhqqAyO5wIIQMJ gyYtGBcjqhV3Pih9waCvqVgsNSulxkYvf5g9L6kwlz0AWMnN9bQ2TTfzCR74Hwxtub5h kVIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678418893; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=k9ZRvPABBk2P/l3glpc+wZa31EczS4aYNgyWqJ35ZUQ=; b=rt2wyVMpRGFPpFibTM1dEW6+bPD00kOdXUMmfb+guiyMcHXBGtOc+CNOZ7ORJvSrko FhRCitK59Yw0XSSZtf3oV4X7vSIs8lhCGHc90YyrVFHMGhD884e9Gosxxj2jc6Ow7Syw 7cEhHc8+k7QxU/q0azl1j9/xdhuVnm1lor7vS5pU/91sjECoESUZzZNddwhVrbjtmv/k PwnqDiuolLmHtniI5sii5d0NwHWFg8moaRiRSrGRJLPE/ve4fwKuCJvD49vL/Zh0APll wx2RAVipaHUN2yBr0USA663g+ys/Tc0zLLeS6RPW9dlijibOOs10h3PUS4/1F4W+hlLX OH3Q==
X-Gm-Message-State: AO0yUKU00NNzl6b9FdnNqNUkU439ZuAgiBWuV8wSFmey6UE3390RpM+T MQvKuXcnkIQ1CLQH5BxhCKUQluPTYnM/wNByMUhz26hg5L2qDdGgoA9Oqg==
X-Google-Smtp-Source: AK7set9Xcu1tQLn5IP15lOZTigMfcQd81jaihCctqkLkYJZkvzRwJj+BHqb6IW05tq/DAUs+HJh3lAz4f/uZUxwQc2A=
X-Received: by 2002:a17:906:4901:b0:8eb:27de:447c with SMTP id b1-20020a170906490100b008eb27de447cmr12322584ejq.13.1678418892961; Thu, 09 Mar 2023 19:28:12 -0800 (PST)
MIME-Version: 1.0
From: Christopher Allen <christophera@lifewithalacrity.com>
Date: Thu, 09 Mar 2023 19:28:01 -0800
Message-ID: <CAAse2dG0xwdToi_+B0UbhBo3ZYYqPTv9oiA=OkaYQbojENiSGg@mail.gmail.com>
To: cbor@ietf.org
Content-Type: multipart/alternative; boundary="000000000000e64ec605f68359ec"
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/fnz_F5lQNiDiTJFAaJGB3YJdPik>
Subject: [Cbor] New IETF Internet-Draft draft-mcnally-deterministic-cbor-00
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 10 Mar 2023 03:28:20 -0000
TL;DR --- Blockchain Commons has posted an IETF Internet Draft (I-D) for discussion about deterministic CBOR *https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/ <https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/>* Blockchain Commons has recently expanded its effort on dCBOR, or deterministic CBOR, which is a deterministically encoded version of the IETF RFC 8949: CBOR - Concise Binary Object Representation data format <http://cbor.io/>. As part of our new work, we’ve released dCBOR codecs in Rust and Swift for community <https://github.com/BlockchainCommons/Community/blob/master/release-path-standards.md#community-review>review as well as a command-line app to verify encodings: - Rust dCBOR Codec: https://github.com/BlockchainCommons/bc-dcbor-rust - Swift dCBOR Codec: https://github.com/BlockchainCommons/BCSwiftDCBOR - dCBOR-CLI: https://github.com/BlockchainCommons/dcbor-cli We’ve been leveraging the CBOR standard as a data format because it’s a binary format that’s concise, compact, self-describing, good in constrained environments, platform/language agnostic, and standardized as an IETF RFC. (see our “Why CBOR?” article & video for more details: https://www.blockchaincommons.com/introduction/Why-CBOR/) But, we also have a need for our data format to be deterministic <https://en.wikipedia.org/wiki/Deterministic_algorithm>. That’s because our Gordian Envelope <https://www.blockchaincommons.com/introduction/Envelope-Intro/> Smart Document format uses Merkle-tree hashes to assure the consistency of an Envelope that’s been elided or encrypted, or produced from identical data by disparate agents. For the hashes to remain consistent, the same data must always be encoded in the same way, which is to say, deterministically! We are sure that we’re not the only ones who can benefit from deterministic encoding! There are many international and other standards groups that are moving toward using CBOR for emerging security standards, including W3C, ISO, and more. The IETF CBOR RFC includes a section (§4.2) on deterministically encoding CBOR: https://datatracker.ietf.org/doc/html/rfc8949#name-deterministically-encoded-c That section of the RFC includes several requirements for deterministic CBOR, but it lists other things as “considerations” which might be implemented differently by different encoders. This includes how to encode numbers like -0 and NaN. There are also some ambiguities in the “shortest-form” rules which state that floating point numbers and large integers (“BigNums”) should be encoded in their shortest form. According to the RFC, all of these considerations are “opt-in” for codec implementers, and have not been prioritized in most existing implementations, putting the cognitive burden for correct deterministic serialization and validation of deterministic compliance when deserializing directly onto the application engineer. To support a fully codified version of dCBOR, we have proposed a new IETF internet-draft (I-D) that focuses on a more strict definition of deterministically encoded CBOR: https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/ Here is the editor’s copy, including all the latest corrections and clarifications. Issues and PRs should be posted there. https://blockchaincommons.github.io/WIPs-IETF-draft-deterministic-cbor/draft-mcnally-deterministic-cbor.html This I-D defines two major domains of responsibility for successfully implementing dCBOR: those which are properly the responsibility of the codec because they deal with the implementation details of serialization, and those which application developers depending on dCBOR must undertake. This I-D is being discussed in the IETF CBOR discussion lists at https://mailarchive.ietf.org/arch/browse/cbor/ We feel that fully defining dCBOR as an international standard is crucial to its advancement so that data is deterministically encoded in the same way by different encoders, creating a truly interoperable standard. As we say in our dCBOR internet-draft: It is important to stress that dCBOR is *not* a new dialect of CBOR, and that all dCBOR is well-formed CBOR that can be read by existing CBOR codecs. The goal of our I-D is to provide norms and practices that standardize binary serialization so that different implementers can produce formatted data that is identical to the point that it will always hash the same when created from the same originating data. We also wish to remove as much cognitive load as possible from adopters of dCBOR, putting responsibility for implementation details on codec implementers; for example, the form in which particular numeric values must be serialized. So far, one of the largest points of discussion in our current I-D has to do with the serialization of floating point numbers with no fractional component, e.g. “10.0”. We believe that the “shortest-form” requirement of RFC 8949 requires us, in principle, to convert floating point numbers to integers when doing so results in no loss of accuracy. This, for example, reduces the three (up to seven!) RFC-accepted ways of encoding the concept of zero down to one: the byte 0x00. The same consideration applies to codecs that support BigNums <https://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic>, or that might support future numeric standards such as quadruple precision <https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format>. This also has benefits in conciseness, which will help in deployment to constrained environments, such as JavaCard devices, TPMs, SEs, or even implemented in silicon logic. However, we are, of course, open to rough consensus input on this topic, as the ultimate goal is to create standards for everyone who interacts with numeric and other types of CBOR values. Thanks for your interest! We hope to be able to quickly develop these guidelines for using dCBOR so that we can all be sure that we are approaching (and encoding!) dCBOR in the same way. -- Christopher Allen - Blockchain Commons
- [Cbor] New IETF Internet-Draft draft-mcnally-dete… Christopher Allen
- Re: [Cbor] New IETF Internet-Draft draft-mcnally-… Carsten Bormann
- Re: [Cbor] New IETF Internet-Draft draft-mcnally-… Anders Rundgren