Re: [COSE] [Rats] RAM requirements for COSE/CWT

[Jeremy O'Donoghue <[e-mail]>]

Hi Carsten, Laurence,

On 01/03/2022, 05:40, "COSE" <[e-mail]> wrote:


WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

On Feb 23, 2022, at 1:08 PM, Carsten Bormann <[e-mail]<mailto:[e-mail]>> wrote:

Hi Laurence,



I know in reality most decoders will handle non-preferred serialization, but I don’t see anything in section 5.2 that says that they must. (non-preferred is still well formed).

What part of the first sentence of section 5.2:


A generic CBOR decoder can decode all well-formed encoded CBOR data items and present the data items to an application.

...could we have phrased better?

Well, um, maybe.  :-)

It’s a small sentence with potentially big implications.

If your generic decoder is like the pseudo code the implications aren’t big, but for decoders like QCBOR that do a lot more work for you, the implications are large — deep combined nesting of interleaved definite length and indefinite lengths maps and arrays, huge indefinite length strings, multiple levels of tag nesting for a data item that is a map label, a nested set of maps that is used as a map label,…

I haven’t checked, but there might not be very many truly conformant generic decoders. QCBOR definitely isn’t, because it has limits like a max of 4GB for encoded input, max levels of map, array and tag nesting and such to be efficient with code size and memory.


[JOD] In strong agreement with Laurence here. I’ve written two CBOR codecs that I believe to be complete, and which try to do quite a lot of the work for the caller: one in Rust (WIP Open Source) and one in Haskell (unpublished).

Thanks to the magic of the Garbage collector, the Haskell decoder can offer an exceptionally ergonomic interface and breezes through most of the problem areas listed, up to the limit of the memory available to a process on the device (I tested it with an auto-generated 2GB token with pretty complex nesting). It is also very large and utterly unsuited to any embedded target I can think of – code plus dependencies plus runtime is about 4MB on my machine.

The Rust version is almost the opposite – it is designed for constrained embedded devices and necessarily has compile-time limits on the size of the input token, limits on nesting and many of the other limitations mentioned above. It’s also about 11kB of code on the same machine (running Rust no_std, so the runtime is almost non-existent – a panic handler and an entry point).

A truly conforming generic decoder is likely to have more in common with the Haskell implementation than the Rust one, but the applicability is necessarily entirely different. Constrained targets will require their CBOR implementations to make implementation decisions that constrain the acceptable input.

This does suggest to me that EAT profiles (and indeed any specification that uses CBOR) might be wise to indicate minimum capability in a decoder in some way. This is most simply expressed in terms of a token size that MUST be accepted, but in the real-world there might be further limitations on nesting depth and the like.

As an interesting aside, during experiments in code-generation from CDDL, one of the challenges is that the structure of the CDDL AST leads to generated code which potentially nests very deeply – I am still looking for ways to handle this efficiently.

That said, probably most decoders can handle non-preferred integers, the topic at hand, just fine because it is trivial.

[JOD] Indeed – non-preferred integers require literally no code in most cases. One thing I would really like to see, and haven’t found so far, is a reliable mechanism to determine the preferred encoding for floats. Currently my implementations require the caller to decide on the float encoding (float16, float32, float64) as I haven’t been able to find a reference that explains how to straightforwardly determine the most efficient *encoding* for a float without loss of precision.

Jeremy

Yes you are of course right that 5.2 that defines generic decoders says non-preferred serialization must be supported.

LL