Re: [Cbor] dCBOR moving from numerically-typeless systems

[Wolf McNally <[e-mail]>]

> On Mar 12, 2023, at 6:46 AM, Carsten Bormann <[e-mail]> wrote:
> 
> On 2023-03-12, at 12:49, Wolf McNally <[e-mail] <mailto:[e-mail]>> wrote:
>> 
>> Carsten,
>> 
>>> On Mar 12, 2023, at 3:26 AM, Carsten Bormann <[e-mail]> wrote:
>>> ...maybe you can explain your “no `null` in map entry values” rule next.  Isn’t that really very much about how the application processes these values?
>> 
>> Just like numeric values have multiple equivalent CBOR representations, map entries with “no value” also have two semantically-equivalent representations:
>> 
>> • omit the entry entirely, or
>> • encode the entry, but give it a `null` value.
>> 
>> If these representations are semantically equivalent,
> 
> Big assumption.

You *say* that, but I’d like to see an actual argument that it’s not in fact true.

> I think you are assuming that all map entries have default values, and that all default values are `null`.

No, not all map entries have default values. Some are required. Some specific combinations may be disallowed.

I have no idea what you mean by “and that all default values are `null`,” as this is equivalent to saying, “and that all default values are no value.” `null` cannot be a default value as semantically `null` expresses the absence of a value.

As I state in the I-D, `null` still has uses in other positions, such as in indexed structures like arrays where some of the elements are optional:

example = {
	a: [number / null, number / null, number / null]
}

In this example, the array MUST have three elements, any of which may be a number or `null`. What `null` means for each element is up to the specifier.

> What if your default value is 10?
> 
> If course, you can use `null` as a secondary representation of that default value, unless the map entry actually has defined semantics for (non-default) `null`; you may choose to exclude such application data models, and you are.
> 
> However, if 10 is the actual default value, it is the application that needs to ensure it never expresses this value (as default values must never be expressed in deterministic representation). 
> 
> You are essentially doing half of the work in the encoder (the part that suppresses `null` if it is the default value), and leaving the other half to the application (suppressing 10).  This is at the cost of not supporting data models at all where `null` is not the default value.
> 
> I find this weird.

Let’s focus on a more concrete example. Consider the following CDDL:

font = {
    name: tstr,
    ? size: (number .gt 0) .default 12
}

As written, the encoder does not have the option of making `size` `null`-valued. It clearly specifies that the field is either to be entirely included or entirely omitted, and that if it is included it must have a numeric value. Therefore, the codec would be right to reject a `null` value here.

I think you’re trying to say that there are conceivable and useful situations in which the codec would be *wrong* to reject it, but you haven’t yet given any examples of where that would be the case *in a deterministic setting.*

To restate the dCBOR requirement:

> Protocols that depend on dCBOR MUST specify the circumstances under which particular optional fields MUST or MUST NOT be present. Protocols that specify fields using key-value paired structures like CBOR maps, where some fields have default values MUST choose and document one of the following strategies:
> 
> • they MUST specify that the absence of the field means choosing the default. This allows the default to be changed later, or
> • they MUST encode the field regardless of whether the current default is chosen. This locks in the current value of the default.

In the above example, the CDDL is telling us that the field’s absence means choosing the default 12, and further that the default will not change, as it’s a fixed part of the published specification. And yes, in a deterministic setting the encoder MUST NOT encode the default value. In fact, I’ve written a lot of code for both JSON and CBOR that simply does not encode a key if it would have the default value; this is only parsimonious. But under dCBOR it is REQUIRED.

On the decoder side, null-valued map entries are semantically equivalent to an absent map entry (I again assert that the burden is on you to demonstrate otherwise) and therefore not well-formed dCBOR. So the “doing half the work in the encoder” as you put it is a natural division of labor between the encoder and the application, and is nothing remarkable to me. The decoder’s work is low-level well-formedness, and the application’s work is higher-level semantic validation.

Back to the dCBOR requirements for defaults: if the specifier wanted to choose the second policy, the CDDL would be written thus:

font = {
    name: tstr,
    size: (float .gt 0) ; Recommended default: 12
}

Now the encoder MUST encode some `size` field. If they want to use the default, the specifier has said it’s 12, but the implementer has to affirmatively encode it.

Now, I mention a possibility of the default “changing” in the case of an optional field. For clarity let's look at a practical example of this:

set-alarm = {
    name: tstr,
    ? expiry: time ; Defaults to current time + 1 hour
}

In this case, omitting the `expiry` field means the receiver will construe it to be 1 hour from the time it processes the request. The protocol specifier will also need to specify what happens if the `expiry` date is present but out of allowable range, and the application will have to enforce this constraint.

Again: in no case is a `null` value for the `expiry` field necessary, useful, or even interesting.

~ Wolf