Re: [Cbor] Decoding numbers and compliance verification in dCBOR

[Wolf McNally <[e-mail]>]

Laurence,

Thanks for your comments. Blockchain Commons is targeting dCBOR at a number of applications where security is a high priority. So if you’re in an environment where you *know* that your agent will never decode a document that is malformed, or into which malicious content may have been smuggled, then you could relax the validation at decode time considerably both for the codec and at the application level. But this situation is not the default case we are considering, hence the use of MUST.

If performance becomes an issue then it will of course be up to the developer to determine whether they are increasing the potential attack surface by reducing validation requirements. And if you want to use a decoder that is lax where we specify stringency, then pretty much any other existing decoder will do. It is a non-goal of our specification to have the tightest, smallest codec.

In addition, throughout the I-D we’re emphasizing that while as much validation work as possible must be placed on the codec, it can’t get you all the way there: the protocol specifier/application developer MUST perform further validations, for example to ensure that a map contains no additional unauthorized data in undocumented entries. This is something the next version of the I-D will make even clearer, and is why I am treating codec-level validation, API best practices, and application developer responsibilities with equal emphasis: determinism isn’t something that can be achieved in a completely automated way (at least not with the tools at hand.)

BTW, if you’re interested I recommend following along with the editor’s copy of the I-D; it’s where I’m putting all the incremental changes that will lead to the 01 revision:

https://blockchaincommons.github.io/WIPs-IETF-draft-deterministic-cbor/draft-mcnally-deterministic-cbor.html

Of course, we also welcome issues and PRs in the repo itself:

https://github.com/BlockchainCommons/WIPs-IETF-draft-deterministic-cbor
BlockchainCommons/WIPs-IETF-draft-deterministic-cbor: Norms and practices for encoding and validating deterministic CBOR (dCBOR).
github.com

~ Wolf

> On Mar 11, 2023, at 12:47 PM, Laurence Lundblade <[e-mail]> wrote:
> 
> On Mar 11, 2023, at 12:27 AM, Wolf McNally <[e-mail]> wrote:
>> 
>> For us, minimizing absolute size of serialization of a numeric value is less of a goal than having a single, deterministic serialization for a given value. In addition, dCBOR codec implementors should be able to forego floating point or bignum support, and still be able to expect the same canonical serialization for all the integers representable by major types 0 and 1.
>> 
> 
> Yes, that generally makes sense, but a few comments.
> 
> You have a requirement that all decoders MUST validate compliance with the dCBOR spec — check sort order, check that 0 is not encoded as a float or bignum. This check is to re enforce hygiene in the dCBOR ecosyste, not a necessity for correct functioning of the decoder.
> 
> This results in extra code on the decode side, not the encode side. The decode side is always bigger and more complex already just for dealing with regular input validation.
> 
> I can see that for highly constrained protocols this hygiene check is much smaller, maybe zero for integers only, no maps….
> 
> So is it really is a MUST, not a really strong SHOULD? Something like “while dCBOR doesn’t absolutely require full compliance (e.g. map is sorted correctly) on decode, this exception is only intended for use in very constrained environments that just can’t afford the extra code size for the check. These are only environments where a few hundred bytes of extra code would affect the cost of the device.”
> 
> This is not a critical issue for me in the end. I mostly want to make sure we’re clear about the implications of the requirement, because it is essentially a prophylaxis for the ecosystem, not something needed for correct function of the decoder.
> 
> LL
> 
> 
> 
>> ~ Wolf
>> Lead Researcher, Blockchain Commons
>> 
>>> On Mar 10, 2023, at 11:06 PM, Anders Rundgren <[e-mail]> wrote:
>>> 
>>> If shortest possible number representation is an absolute goal, you already have a problem with "pure" integers.  An integer value of 1099511627775 (0xffffffffff) would actually yield two bytes less(!) using the Bignums type.
>>> 
>>> It would (IMO) be unwise trying to fix this in dCBOR.
>>> 
>>> Anders
>>> 
>>> On 2023-03-10 20:14, Carsten Bormann wrote:
>>>> Hi Laurence,
>>>> I think your arguments are important.
>>>> But for a receiver of information, there are also benefits from knowing what to expect.
>>>> In particular map processing can be simpler if only a specific sequence of the entries needs to be accepted.
>>>> Whether floating point values are important for an application may also influence whether it is worth to expend some additional processing.  The simplest devices often get by without any floating point operations.  Once floating point becomes important for the functioning of a device, processors such as those of ARM’s Cortex M4 series can help reduce the power-hungry on-time of the device by efficiently performing the floating point computations.  With these processors (and certainly with processors of the smartphone, laptop, and server classes), the requirements of dCBOR become almost trivial.
>>>> I don’t want to take a particular side here, just point out that not all applications are the same.  Having a go-to profile of CBOR that covers an interesting subset of applications appears to be a net win to me.
>>>> Additional CDDL support such as that provided in draft-bormann-cbor-cddl-more-control with .cbordet and .cborseqdet may be desirable.  There is currently no way in CDDL to express the rules about number representation that dCBOR has adopted; that may be one interesting gap.
>>>> Little aside: When I started to think about this, I started to wonder: What is the dCBOR way to represent 65536000000.0?
>>>> 5 bytes:
>>>>>> 65536000000.0.to_cbor.hexs
>>>> => "fa 51 74 24 00"
>>>> 9 bytes:
>>>>>> 65536000000.to_cbor.hexs
>>>> => "1b 00 00 00 0f 42 40 00 00”
>>>> Here, the floating point representation is shorter…
>>>> But wait, even if float32 cannot be exact (e.g., for 65536000001), try:
>>>> 7 bytes:
>>>>>> CBOR.decode("c2 45 0f 42 40 00 01".xeh)
>>>> => 65536000001
>>>> 9 bytes:
>>>>>> CBOR.decode("c2 45 0f 42 40 00 01".xeh).to_cbor.hexs
>>>> => "1b 00 00 00 0f 42 40 00 01”
>>>> (Number systems are always an unwieldy part of representation formats.)
>>>> Grüße, Carsten
>>>> _______________________________________________
>>>> CBOR mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/cbor
>>> 
>> 
>