Re: [Cbor] Deterministic CBOR as a possible DISPATCH item

[Carsten Bormann <[e-mail]>]

On 2023-03-05, at 21:10, Laurence Lundblade <[e-mail]> wrote:
> 
> 
>> On Mar 5, 2023, at 5:29 AM, Carsten Bormann <[e-mail]> wrote:
>> 
>>> Determinism is one such good practice when it comes to creating cryptographically enhanced documents.
>> 
>> Determinism is mostly unneeded in interchanged data items.
>> (It is needed for signing inputs, which are therefore best designed to make determinism inexpensive.)
>> It still is nice to have, e.g., for test scripts.
> 
> I don’t think determinism is required for signing.

That sentence only works for a weird definition of “determinism”.  
I think you are confusing this term with “Deterministic Encoding”.
RFC 8949 uses the terms "Deterministically Encoded” or "Deterministic Encoding” for a reason.  
There is a limited area of application where that is a useful thing.  
COSE signing needs the trivial subset of this (Section 4.1), but generally not the full generality of the requirements in Section 4.2, which need more work during implementation and possibly at runtime.

> When the signed data is transmitted with the signature, then the receiver has the exact bytes that were signed and can verify the signature. For example, determinism is not needed for COSE_Sign payloads.

I’d rather say “deterministic encoding is trivial” for where it is needed COSE.
(COSE only uses arrays for building the signing inputs; these need to be deterministically encoded, which is trivial for most encoders as it is identical to preferred encoding; see Section 9 of RFC 9052, which is just a recap of the subset needed.)

> There are a few rare cases where determinism is needed for signing:
> 
> 1) The signed data data is not transmitted. It is instead constructed separately by the sender when signing and the receiver when verifying. The Sig_structure internal to COSE is an example of this.

The Sig_structure is built from the transmitted data in the following way:

   Sig_structure = [
       context : "Signature" / "Signature1" / "CounterSignature",
       body_protected : empty_or_serialized_map,
       ? sign_protected : empty_or_serialized_map,
       external_aad : bstr,
       payload : bstr
   ]

The creation of the encoded data item from this structure is (trivially) deterministic, and needs to be that.

> 2) The data is transformed in transmit. An example of this email signing like S/MIME because the old email protocols are text-based and change line endings and such. Both the sender and receiver have to be able to construct the same canonical representation because you can’t rely on the transport to not change it. I don’t think this happens when CBOR is transmitted.

That happens below the representation format layers we are concerned with here.

> So to me, CBOR determinism is technically only needed in rare circumstances.

For your definition of CBOR determinism…
See above.

> All that said, I can still see why it might be good to have something like dCBOR standardized in a way that 8949 doesn’t. Sometimes it is nice to be able to do a binary comparison of two encoded structures.

If you want to do this, you indeed need deterministic encoding.
(And, as I said, that is quite useful in test scripts.
So if you don’t need the performance of simple preferred encoding, go ahead and make everything deterministic.)

> The variability in CBOR serialization does seem to cause a lot of discussion and confusion (e.g., in work on EAT). Because of the variability, there isn’t guaranteed interoperability between CBOR implementations.

You keep saying that...

> I’d speculate that this is slowing the adoption of CBOR.

There is way less variability than, say, for JSON.

> I recall always preferring DER to BER, but that was mostly because the difference was confusing, I needed to get my code working quickly and I didn’t want to take the time to understand what it was about and what I’d need to do to support BER in my code.

DER is BER.
(Not all BER is DER, though.)

> Another thing here is that the use cases where the CBOR serialization variations are needed are rare and implementing deterministic encoding is not that hard, though sorting map keys can require some memory.
> 
> Also, I agree that we want to be strict these days — Postel was wrong.

Postel was right, it’s just that people have misunderstood and misapplied his principle.
draft-iab-protocol-maintenance-12

> P.S. I am not opposed to fixing QCBOR encoding of subnormals.

What is wrong with that encoding?
Your IEEE 754 floating point hardware is likely to already give you all you need to get this right.

> I just have been working on other things. Note also that QCBOR doesn’t use floating point for half-precision, so the fix is a bit more work.

Can’t parse this.  How can you not use floating point for half-precision floating point?
Maybe you are trying to say you didn’t implement half-precision floating point?
There is code in the RFC you can copy.

Grüße, Carsten