Re: [Cbor] Unusual map labels, dCBOR and interop

[Wolf McNally <[e-mail]>]

Orie,

No, dCBOR does *not* serialize maps differently from CDE, and all dCBOR is absolutely `application/cbor`.

dCBOR is a “profile” of CDE, which is a profile of CBOR. Hence, all dCBOR is valid CDE and all CDE is valid CBOR. All CBOR decoders can decode dCBOR without even knowing it *is* dCBOR. All CBOR encoders can be used to encode dCBOR as long as the data presented to the encoder conforms to the dCBOR rules.

Now: not all CBOR is dCBOR. This is expected because CBOR does not have determinism as a goal, while dCBOR does. CBOR lets you encode non-preferred numerical encodings, have map keys not in canonical order, have NaN values with payloads, and numerous other things that RFC CBOR decoders *must* accept as valid CBOR. dCBOR decoders on the other hand *must* enforce dCBOR validation: preferred numerical encoding (including numeric reduction of floating point values) *must* be used. Map keys *must* be in canonical order. There is exactly *one* valid value for NaN. The only simple values accepted are `true`, `false`, and `null`, etc.

If you want a protocol that allows for either CBOR or dCBOR, then a tag can be assigned for that purpose, but you don’t even really need one: we already have a tag for Gordian Envelope: #6.200, and Gordian Envelope requires dCBOR. Therefore, in its simplest expression, you have 200(42(“Hello”)), where #6.200 is “Gordian Envelope”, #6.24 is “Encoded CBOR Data Item”, and “Hello” could be any dCBOR at all:

D8 C8               # tag(200) Envelope
   D8 2A            # tag(42) CBOR Data Item
      65            # text(5) Any dCBOR
         48656C6C6F # "Hello"

Hence there is absolutely no ambiguity.

~ Wolf

> On Mar 27, 2024, at 8:36 PM, Orie Steele <[e-mail]> wrote:
> 
> (with no hat)
> 
> Set size comparison, and equivalence is key to distinguishing types in mathematics.
> 
> The set of valid application/cbor instances is strictly larger than the set of application/dcbor (or whatever) instances.
> 
> This exact property is what makes dcbor useful especially in the context of unique digests for a given serialized data structure.
> 
> If undefined were encountered when deserializing dcbor to JavaScript, an error should be thrown?
> 
> SCITT Receipts and CWTs are built on the assumption that no error would be thrown.
> 
> I must have not understood the comment made regarding map labels, because I thought you were saying that dcbor serializes maps differently from vanilla CDE... If it doesn't, why the need for any new serialization?
> 
> I don't think dcbor can be called application/cbor any more than json without booleans can be called application/json.
> 
> Processors not expecting Booleans in JSON would explode, and processors of dcbor expecting it to be meaningfully different than application/cbor will also explode right?
> 
> Feeding unexpected content to a parser is the normal expectation for security formats.
> 
> We've seen real problems arise from JSON-LD, being valid json, but not quite valid "JSON-LD"... It can lead to ambiguous processing, where some middle boxes throw errors and others don't.
> 
> Distinguishing dbcor from cbor seems a prerequisite to making any progress on its application in a security context.
> 
> OS
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Wed, Mar 27, 2024, 9:42 PM Wolf McNally <[e-mail] <mailto:[e-mail]>> wrote:
>> Orie,
>> 
>> > On Mar 27, 2024, at 6:57 PM, Orie Steele <[e-mail]> wrote:
>> > 
>> > Using deterministic encoding to serialize map keys is interesting.
>> > 
>> > I wonder about security issues with distinguishability.
>> > 
>> > Consider the case where a map has keys that are serialized with and without deterministic encoding.
>> 
>> Map keys in dCBOR are themselves part of the dCBOR serialization and must therefore also conform to dCBOR.
>> 
>> > 
>> > Restricting map keys, reminds me of another flavor of CBOR, I think it was called dag-cbor, it required all map keys to be tstr... Whereas dCBOR requires them all to be bstr?
>> 
>> No, the handling of map keys as serialized dCBOR strings internally is a detail of our reference implementation. dCBOR map keys are simply serialized as dCBOR items like every other dCBOR item— they are not segregated into typed byte strings. So the map:
>> 
>> {[1, 2, 3]: "Hello”}
>> 
>> Is serialized as:
>> 
>> A1               # map(1)
>>    83            # array(3)
>>       01         # unsigned(1)
>>       02         # unsigned(2)
>>       03         # unsigned(3)
>>    65            # text(5)
>>       48656C6C6F # "Hello"
>> 
>> Not as:
>> 
>> {h'83010203': "Hello”}
>> 
>> A1               # map(1)
>>    44            # bytes(4)
>>       83010203   # "\x83\u0001\u0002\u0003"
>>    65            # text(5)
>>       48656C6C6F # "Hello"
>> 
>> > Is it critical to assign unique media types in order for deserialization to succeed?
>> 
>> No, for the reasons stated above.
>> 
>> > I also wonder about interactions with COSE and related protocols especially regarding unprotected headers.
>> 
>> COSE is not necessarily dCBOR, although you can certainly encode COSE in compliance with dCBOR. And for particular applications you can certainly state that any of the signed byte strings in the protected header or payload must conform to dCBOR, which would help ensure determinism. For example, the payload could be a serialized Gordian Envelope, which is a format that supports verifiable claims based on dCBOR. We at Blockchain Commons are considering proposing the inclusion of Gordian Envelope as a Verifiable Data Structure for inclusion in the COSE Receipts registries.
>> 
>> https://datatracker.ietf.org/doc/draft-mcnally-envelope/
>> 
>> ~ Wolf