Re: [Cbor] Deterministic CBOR as a possible DISPATCH item

[Laurence Lundblade <[e-mail]>]

> On Mar 5, 2023, at 12:58 PM, Carsten Bormann <[e-mail]> wrote:
> 
> On 2023-03-05, at 21:10, Laurence Lundblade <[e-mail]> wrote:
>> 
>> 
>>> On Mar 5, 2023, at 5:29 AM, Carsten Bormann <[e-mail]> wrote:
>>> 
>>>> Determinism is one such good practice when it comes to creating cryptographically enhanced documents.
>>> 
>>> Determinism is mostly unneeded in interchanged data items.
>>> (It is needed for signing inputs, which are therefore best designed to make determinism inexpensive.)
>>> It still is nice to have, e.g., for test scripts.
>> 
>> I don’t think determinism is required for signing.
> 
> That sentence only works for a weird definition of “determinism”.  
> I think you are confusing this term with “Deterministic Encoding”.
> RFC 8949 uses the terms "Deterministically Encoded” or "Deterministic Encoding” for a reason.  
> There is a limited area of application where that is a useful thing.  
> COSE signing needs the trivial subset of this (Section 4.1), but generally not the full generality of the requirements in Section 4.2, which need more work during implementation and possibly at runtime.
> 
>> When the signed data is transmitted with the signature, then the receiver has the exact bytes that were signed and can verify the signature. For example, determinism is not needed for COSE_Sign payloads.
> 
> I’d rather say “deterministic encoding is trivial” for where it is needed COSE.
> (COSE only uses arrays for building the signing inputs; these need to be deterministically encoded, which is trivial for most encoders as it is identical to preferred encoding; see Section 9 of RFC 9052, which is just a recap of the subset needed.)
> 
>> There are a few rare cases where determinism is needed for signing:
>> 
>> 1) The signed data data is not transmitted. It is instead constructed separately by the sender when signing and the receiver when verifying. The Sig_structure internal to COSE is an example of this.
> 
> The Sig_structure is built from the transmitted data in the following way:
> 
>   Sig_structure = [
>       context : "Signature" / "Signature1" / "CounterSignature",
>       body_protected : empty_or_serialized_map,
>       ? sign_protected : empty_or_serialized_map,
>       external_aad : bstr,
>       payload : bstr
>   ]
> 
> The creation of the encoded data item from this structure is (trivially) deterministic, and needs to be that.
> 
>> 2) The data is transformed in transmit. An example of this email signing like S/MIME because the old email protocols are text-based and change line endings and such. Both the sender and receiver have to be able to construct the same canonical representation because you can’t rely on the transport to not change it. I don’t think this happens when CBOR is transmitted.
> 
> That happens below the representation format layers we are concerned with here.
> 
>> So to me, CBOR determinism is technically only needed in rare circumstances.


Here’s a simple protocol to show that deterministic CBOR serialization is not required for signing.

In this protocol, a CBOR-encoded integer temperature reading is signed by a sensor and sent to a server. Some temperature sensors always serialize to a 64-bit integer, others a 32-bit integer and others use preferred encoding. There is no requirement that the integer be sent with a particular CBOR serialization (but there is a requirement the server be able to decode all integer forms).

The signature scheme here could be anything that can carry a payload — COSE, CMS, PGP, TLS or other — it doesn’t matter.

The signature verification is always possible because the receiver has the exact bytes that were signed. The signature verification would succeed even if the CBOR was not well formed.


That said, it looks to me that the Gordian Envelope is much much more complicated than just signing some transmitted payload. I would say something like “Gordian Envelope is a complicated cryptographic protocol that involves data in transit, data at rest plus modification and manipulation of CBOR-represented data. To make it work deterministic encoding is needed” — very different from the blanket statement that “deterministic CBOR encoding is required for (all) signing (in the known world)”.


> 
> For your definition of CBOR determinism…
> See above.
> 
>> All that said, I can still see why it might be good to have something like dCBOR standardized in a way that 8949 doesn’t. Sometimes it is nice to be able to do a binary comparison of two encoded structures.
> 
> If you want to do this, you indeed need deterministic encoding.
> (And, as I said, that is quite useful in test scripts.
> So if you don’t need the performance of simple preferred encoding, go ahead and make everything deterministic.)
> 
>> The variability in CBOR serialization does seem to cause a lot of discussion and confusion (e.g., in work on EAT). Because of the variability, there isn’t guaranteed interoperability between CBOR implementations.
> 
> You keep saying that...
> 
>> I’d speculate that this is slowing the adoption of CBOR.
> 
> There is way less variability than, say, for JSON.

Yes, but that doesn’t mean my speculation is incorrect.


…


>> P.S. I am not opposed to fixing QCBOR encoding of subnormals.
> 
> What is wrong with that encoding?
> Your IEEE 754 floating point hardware is likely to already give you all you need to get this right.
> 
>> I just have been working on other things. Note also that QCBOR doesn’t use floating point for half-precision, so the fix is a bit more work.
> 
> Can’t parse this.  How can you not use floating point for half-precision floating point?
> Maybe you are trying to say you didn’t implement half-precision floating point?
> There is code in the RFC you can copy.

This isn’t really of relevance to this discussion. Anders just brought it up. As you said, this can be fixed.

(I chose to implement the conversion between half, single and double-precision float with shifts and masks rather than floating point hardware or a floating-point library. This makes QCBOR more portable. Before I added conversion from bignum, decimal fraction, big float to regular float, it didn’t need any floating-point support at all other than float and double in the ABI. There’s a few corner cases with subnormals that I didn’t complete. I use the code in the RFC for regression tests.)