Re: [Cbor] Decoding numbers and compliance verification in dCBOR

[Anders Rundgren <[e-mail]>]

AFAICT, properly defined deterministic encoding rules can actually *simplify* decoders.

Map sorting gets trivial, you just keep a copy of the preceding key and verify that the new key is bigger.

However, not sticking to Rule 2 in section 4.2.2 of RFC 8949 is asking for trouble.  Really constrained systems probably do not even use floating point, making the space argument seems pretty weak.

Although not an IETF document the following specification holds a condensed description of what I consider a "reasonable" dCBOR scheme:
https://cyberphone.github.io/javaapi/org/webpki/cbor/package-summary.html#deterministic-serialization

Anders

On 2023-03-11 21:47, Laurence Lundblade wrote:
> On Mar 11, 2023, at 12:27 AM, Wolf McNally <[e-mail]> wrote:
>>
>> For us, minimizing absolute size of serialization of a numeric value is less of a goal than having a single, deterministic serialization for a given value. In addition, dCBOR codec implementors should be able to forego floating point or bignum support, and still be able to expect the same canonical serialization for all the integers representable by major types 0 and 1.
>>
> 
> Yes, that generally makes sense, but a few comments.
> 
> You have a requirement that all decoders MUST validate compliance with the dCBOR spec — check sort order, check that 0 is not encoded as a float or bignum. This check is to re enforce hygiene in the dCBOR ecosyste, not a necessity for correct functioning of the decoder.
> 
> This results in extra code on the decode side, not the encode side. The decode side is always bigger and more complex already just for dealing with regular input validation.
> 
> I can see that for highly constrained protocols this hygiene check is much smaller, maybe zero for integers only, no maps….
> 
> So is it really is a MUST, not a really strong SHOULD? Something like “while dCBOR doesn’t absolutely require full compliance (e.g. map is sorted correctly) on decode, this exception is only intended for use in very constrained environments that just can’t afford the extra code size for the check. These are only environments where a few hundred bytes of extra code would affect the cost of the device.”
> 
> This is not a critical issue for me in the end. I mostly want to make sure we’re clear about the implications of the requirement, because it is essentially a prophylaxis for the ecosystem, not something needed for correct function of the decoder.
> 
> LL
> 
> 
> 
>> ~ Wolf
>> Lead Researcher, Blockchain Commons
>>
>>> On Mar 10, 2023, at 11:06 PM, Anders Rundgren <[e-mail]> wrote:
>>>
>>> If shortest possible number representation is an absolute goal, you already have a problem with "pure" integers.  An integer value of 1099511627775 (0xffffffffff) would actually yield two bytes less(!) using the Bignums type.
>>>
>>> It would (IMO) be unwise trying to fix this in dCBOR.
>>>
>>> Anders
>>>
>>> On 2023-03-10 20:14, Carsten Bormann wrote:
>>>> Hi Laurence,
>>>> I think your arguments are important.
>>>> But for a receiver of information, there are also benefits from knowing what to expect.
>>>> In particular map processing can be simpler if only a specific sequence of the entries needs to be accepted.
>>>> Whether floating point values are important for an application may also influence whether it is worth to expend some additional processing.  The simplest devices often get by without any floating point operations.  Once floating point becomes important for the functioning of a device, processors such as those of ARM’s Cortex M4 series can help reduce the power-hungry on-time of the device by efficiently performing the floating point computations.  With these processors (and certainly with processors of the smartphone, laptop, and server classes), the requirements of dCBOR become almost trivial.
>>>> I don’t want to take a particular side here, just point out that not all applications are the same.  Having a go-to profile of CBOR that covers an interesting subset of applications appears to be a net win to me.
>>>> Additional CDDL support such as that provided in draft-bormann-cbor-cddl-more-control with .cbordet and .cborseqdet may be desirable.  There is currently no way in CDDL to express the rules about number representation that dCBOR has adopted; that may be one interesting gap.
>>>> Little aside: When I started to think about this, I started to wonder: What is the dCBOR way to represent 65536000000.0?
>>>> 5 bytes:
>>>>>> 65536000000.0.to_cbor.hexs
>>>> => "fa 51 74 24 00"
>>>> 9 bytes:
>>>>>> 65536000000.to_cbor.hexs
>>>> => "1b 00 00 00 0f 42 40 00 00”
>>>> Here, the floating point representation is shorter…
>>>> But wait, even if float32 cannot be exact (e.g., for 65536000001), try:
>>>> 7 bytes:
>>>>>> CBOR.decode("c2 45 0f 42 40 00 01".xeh)
>>>> => 65536000001
>>>> 9 bytes:
>>>>>> CBOR.decode("c2 45 0f 42 40 00 01".xeh).to_cbor.hexs
>>>> => "1b 00 00 00 0f 42 40 00 01”
>>>> (Number systems are always an unwieldy part of representation formats.)
>>>> Grüße, Carsten
>>>> _______________________________________________
>>>> CBOR mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/cbor
>>>
>>
>