Re: [Cbor] Updated Drafts for dCBOR I-D and Gordian Envelope Structured Data Format I-D & IANA Tag Registration

[Jeremy O'Donoghue <[e-mail]>]

Mostly agree.

Important to understand that BigNum implies challenges that extend beyond the CBOR codec in most cases. For many implementation platforms, a non-core library is required to make meaningful use of bignum values. For constrained implementations, this is something an implementer would likely wish to avoid. Many bignum libraries are optimised for crypto use-cases rather than general purpose arithmetic, and can be fiddly to use.

Carsten has mentioned that there may be good reason to wish to standardize CBOR encodings of other low-precision numeric formats (see https://mailarchive.ietf.org/arch/msg/cbor/lFY2jvRQhBzog_EpUXsoor-x8FA/) which are useful for specific use-cases. I support this in principle – it extends the applicability of CBOR as a serialization format.

This does mean that any attempt to standardize a deterministic encoding format risks locking us into a particular view of the world that may prove impossible to change in a backwards compatible manner.

I agree with Wolf that smallest encoding should be a non-goal. In descending order of priority, I believe the goals should be:


  *   Representation of values without loss of precision. In many applications it is not reasonable to represent floating numbers with their truncated integer equivalents. We already have good mechanisms for this which are implemented at least in QCBOR and my CBOR codec, and probably many others.
  *   Prefer primitive value representations over non-primitive. This means that tagged representations such as Bignum and Decimal Fractions are always less preferred than major types.
  *   Prefer smaller representations over larger.

The idea of a canonical hierarchy of types is one that I support in principle, although it may be difficult to come to consensus once we support additional lower-precision formats. I suspect that it may be impossible to define dCBOR without a set of profiles (or “configurations”, or whatever we choose to call them) which define required features for support – thus a dCBOR codec without float support would necessarily encode floating point values by truncating (or rounding?) to integer values.

Best regards
Jeremy

On 31/05/2023, 09:22, "Wolf McNally" <[e-mail]> wrote:


WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
Christopher and I will be a today’s meeting.

The main concern I have over using the heuristic of “shortest encoding wins” is exactly the issue that, as has been pointed out by others, some BigNum-encoded representations are in fact shorter than regular CBOR uint/nint representations. This means that to encode these values deterministically, a dCBOR codec *must* support BigNum, and I don’t want this to be a requirement.

The proposal we have forwarded is “least precise wins.” Since all fixed-size types can be considered "less precise" than BigNum, BigNum is considered less-preferred, and will then only be used when all the less-precise types fail to represent a presented value. For applications that don’t require values that require BigNum representation, the dCBOR codecs used don’t need to support it.

The same principle also works on a smaller scale for floating point values: A constrained dCBOR codec wouldn’t even have to implement floating point support if it doesn’t need it, because the integer types are less precise than the floating point types.

One could quibble about the exact meaning of “precision” in this context when, for instance, comparing the precision of a float type to an int type, but our proposal comes down to establishing a canonical hierarchy of numeric types (not encodings!) from most-to-least preferred, and finding the single most-preferred type that can represent a presented value without loss of accuracy. So if it can be represented in one byte, it will be. If a codec supports BigNum, and a value is presented to the codec API as a BigNum, and can’t be represented by any more preferred (less-precise) type, then BigNum will be used for the encoding.

The primary goal of dCBOR is determinism. Shortest-possible encoded form is nice when possible, but a non-goal.

~ Wolf


On May 10, 2023, at 10:55 AM, Laurence Lundblade <[e-mail]> wrote:

On May 10, 2023, at 2:44 AM, Jeremy O'Donoghue <[e-mail]<mailto:[e-mail]>> wrote:

To give a very practical use-case, Rust has a native i128 type, for which the most “natural” encoding of Carsten’s example would be as a CBOR negative integer. Haskell uses “unlimited” length integers as (perhaps more usefully) does Python.

I would prefer to see DCBOR require encoding of integer types on the smallest/most natural CBOR type. Only when outside the range of the CBOR positive and negative integer ranges should bigint be used. We don’t have to constrain everything else because of the limitations of the C and C++ builtin integer types.

Best regards
Jeremy

+1 for this.

DCBOR implementations and applications are going to vary depending on whether floating point is needed/available and on the precision of the float and on the range of integers needed/available. It’s not possible to expect every implementation to support every number possible, nor will every use case need every value.

It might be helpful (necessary to pre-define a few number number range profiles so a use case can pick from a menu and so there are example profiles.

For example:
·         Basic 64-bit integer profile: Integer values between  MIN_INT64 to MAX_INT64 are allowed. All others error out.
·         Basic float profile: All values that can be represented by an IEEE 754 double are allowed. All others error out. Bignum support is not required.
·         Basic float profile with lignum: All values that can be represented by an IEEE 754 double are allowed. All others error out. Bignum representation are allowed. Implementations must round off big numbers that have greater precision than can be represented by an IEEE 754 double.
LL