Re: [Cbor] dCBOR moving from numerically-typeless systems

[Wolf McNally <[e-mail]>]

Carsten,

> On Mar 12, 2023, at 3:26 AM, Carsten Bormann <[e-mail] <mailto:[e-mail]>> wrote:
> ...maybe you can explain your “no `null` in map entry values” rule next.  Isn’t that really very much about how the application processes these values?

Just like numeric values have multiple equivalent CBOR representations, map entries with “no value” also have two semantically-equivalent representations:

• omit the entry entirely, or
• encode the entry, but give it a `null` value.

If these representations are semantically equivalent, then like numeric values, this calls for reduction to a canonical case because we don’t want disparate agents making different choices about how to encode the same semantics.

Hypothetically, the protocol specification for a map could have hundreds of possible documented keys, most or all of which are optional. If we are to represent this optionality deterministically, we have two choices:

• include possibly hundreds of keys, many with a `null` value, or
• only include keys with non-`null` values.

As I said, it is not a choice to let the protocol adopter decide, as this breaks determinism. Should we let the protocol specifier decide? This would require a compelling case that there could be a meaningful difference in semantics between a `null`-valued map entry and the omission of the entry, and I have yet to see such an argument.

Even having ruled out null-valued map entries, this still leaves protocol specifiers with a choice about the semantics of an omitted entry. I deal with this in section 4.1 of the I-D:

> Protocols that depend on dCBOR MUST specify the circumstances under which particular optional fields MUST or MUST NOT be present. Protocols that specify fields using key-value paired structures like CBOR maps, where some fields have default values MUST choose and document one of the following strategies:
> 
> • they MUST specify that the absence of the field means choosing the default. This allows the default to be changed later, or
> • they MUST encode the field regardless of whether the current default is chosen. This locks in the current value of the default.

https://blockchaincommons.github.io/WIPs-IETF-draft-deterministic-cbor/draft-mcnally-deterministic-cbor.html#name-optional-default-values

Neither of the above choices requires a `null`-valued map entry.

So there is definitely a responsibility of the specifier to determine how the absence of an optional key is processed.

Again: the principle is that as much cognitive burden as possible should be placed on the codec. Obviously, the codec can only reject cases as invalid that it can detect. Since the codec can mechanically detect the presence of a key with a `null` value, and since there exists an equivalent way of representing the same semantics (the absence of the key), we therefore specify that the absence of the key is canonical and that the codec MUST reject the case it can detect: a `null`-valued entry.

After the codec validates a map, then it is again up to the application developer to perform any additional validations, such as whether all required keys are present, whether the combination of optional keys present is valid, that no additional unsupported keys are present, and whether all present keys have values in their allowable ranges.

~ Wolf