[Cbor] dCBOR moving from numerically-typeless systems

[Wolf McNally <[e-mail]>]

Consider the case of a dCBOR codec written in Javascript: a language that only has a “number” type and therefore makes no programmer or API-level distinction between integers and floating point values. I have the array:

var a = [10, 10.0, 0, -0.0, 0.0];

Let’s also say our application protocol specifies that positions 1, 3, and 4 in `a` MUST be encoded as floating point values and the others MUST be encoded as integers (yes this is a toy example, but someone working first in a language with a type distinction between integers and floating point values might very well make this kind of choice, which becomes binding on its adopters.)

Remember: we’re dealing with *deterministic* CBOR, where different agents encoding the same data should automatically converge on the exact same serialized representation, so if the protocol specifier says “this element is floating point,” it MUST be floating point.

Now, how shall we serialize this to dCBOR in Javascript? First of all, as soon as we execute the above line of code we have already lost the distinction between “integers” and “floating point values with no fractional part” (Javascript preserves the sign bit for zero, apparently):

console.log(a);
[10, 10, 0, -0, 0]

What if we start with JSON?

JSON.stringify(a);
[10,10,0,0,0]

Now we’ve lost everything redundant and have the canonical numeric representation I’m proposing for dCBOR.

But back to the original question. Let’s say we've insisted that that “official” dCBOR keeps CBOR’s type distinction and must therefore validate not just the numeric value but also the type of encoding (or again, it’s not deterministic.) The canonical serialization of `a` will then b:

85         # array(5)
   0A      # unsigned(10)
   F9 4900 # primitive(18688)
   00      # unsigned(0)
   F9 8000 # primitive(32768)
   F9 0000 # primitive(0)

(As an aside given the input, to me this looks like the antithesis of “deterministic.” As a further aside, it is only one byte smaller than the JSON.stringify() version.)

To create a JavaScript dCBOR API that can output this serialization it would need to afford something like this:

let cborA = new CBORArray();
cborA.appendInt(a[0]);
cborA.appendFloat(a[1]);
cborA.appendInt(a[2]);
cborA.appendFloat(a[3]);
cborA.appendFloat(a[4]);
let data = cborA.serialize();

Each element in such a `CBORArray` struct would need to be a type that keeps track of whether it is to be serialized as a floating point value or an integer.

The problem is even worse on deserialization, as validation would require passing the decoder some sort of schema, or require manual validation that the elements of `a` were properly encoded as integers or floats.

On the other hand, if we encode the canonical representation, which both JSON.stringify() and our I-D agree upon, we get:

85    # array(5)
   0A # unsigned(10)
   0A # unsigned(10)
   00 # unsigned(0)
   00 # unsigned(0)
   00 # unsigned(0)
 
And to produce this result in Javascript, the API is minimal:

let data = CBOR.serialize(a);

This would be portable to any other CBOR or conformant dCBOR implementation, where every element of `a` could be successfully extracted as Double regardless of how it was serialized.

~ Wolf