Re: [Cbor] dCBOR moving from numerically-typeless systems

[Wolf McNally <[e-mail]>]

Carsten,

In the case of  [uint, 1.0..100.0, -14..68, float, float16] as a data model, the codec will ensure that the serialization is canonical, and the application must validate that each value is in the correct range.

Anything beyond what the CBOR codec can validate is up to the application to validate. That includes the range of values allowable in each position of the array. This is what I mean when I say that the codec can’t get you all the way there: all it can do is ensure that the canonical encoding is used for each numeric value; it can’t ensure that each element falls into a particular range unless you add some kind of scheme validation feature, like a built-in CDDL-based validator. That is out of scope for our current endeavor.

So as far as I’m concerned, nothing about our approach to dCBOR changes between my hand-crafted example, and your hand-crafted example.

Whether BigInt precision is necessary is up to the protocol specifier. This will in turn dictate the features of codecs that can work with the protocol. The same is also true for floating point: many protocols will not require the representation of floating point values, and will therefore work with codecs that don’t support them. None of this changes whether a given codec can be considered dCBOR compliant, and I made this clear in the I-D.

As I said before: it is up to the protocol specifier to specify the *maximum* precision of a numeric value. It is the job of the codec to find the canonical representation of it for encoding, and to validate that canonical representation upon decoding. The developer who wants to adopt that protocol must choose a codec that can handle its numerical precision needs, and perform any further validation of things like numeric ranges when decoding.

~ Wolf

> On Mar 12, 2023, at 1:00 AM, Carsten Bormann <[e-mail]> wrote:
> 
> Hi Wolf,
> 
> (This is a nice specimen for what I call the fallacy of hand-crafted examples.)
> 
> So it seems your data model is, say:
> 
> a = [uint, 1.0..100.0, -14..68, float, float16]
> 
> (I’m guessing from your example, using uint and int as well as a floating point range just to point out that there may be other application-level type requirements than int vs. float.)
> 
> You already assume that, when extracting these values, you know the data model so you can hand the right type to the application (u64, f64, i8, f64, f16).  So you might as well employ that information for encoding.
> 
> You picked the example so that it actually has a benefit for the mixed integer/float encoding.
> But where your data model has floating point values, you are likely to have actual values all over the place, and the percentage of integral values (no fractional part) that you can represent as integers is low (depending on range — if you have a value from 1024.0 to 2047.0 with 10-bit precision, you are going to be more lucky of course :-), (but then there is no size difference in that particular range).
> 
> If you are stuck with the JavaScript number system, you’ll want to represent the full-range uint in position 0 as a BigInt (see RFC 7493): you can’t just push all numbers into binary64 (float64), losing exactness.
> 
> Again, I very much sympathize with the choices dCBOR makes (it was pretty much how I looked at things in 2013), I just wanted to point out where your hand-crafted example is missing out on additional considerations that just don’t pop up in that example, but do in my hand-crafted one :-).
> 
> Grüße, Carsten
>