[Iot-directorate] Segmented strings (Re: [Rats] [Last-Call] EAT profiles (was Re: Iotdir last call review of draft-ietf-rats-eat-13))

[Carsten Bormann <[e-mail]>]

On 2022-06-08, at 21:26, Laurence Lundblade <[e-mail]> wrote:
> 
> The decoding of indefinite length strings requires a memory allocator or some strategy to coalesce string chunks. The COSE payload, which must be hashed, could be many independent string chunks. COSE header parameter values could be lots of string chunks. However you do the coalescing or processing, it is going to be more code and it is something people may not always implement. My t_cose implementation of COSE doesn’t support indefinite length strings yet.

OK, you are focusing on indefinite length encoded strings.
(I couldn’t imagine why you would want to discuss indefinite length encoded arrays or maps.)
Let’s simplify this term to “segmented strings”.

Segmented strings were not part of the original CBOR design, but were added after loud arguments from the “streaming” part of the community.
For an implementation, segmented strings are important when a string is not all available
— at the same place in memory (“scatter-gather”), or
— at the same time (“streaming”).

They are a trade-off:
They were added to allow generators (encoders) to generate encoded CBOR from segmented inputs without having to require to ship that encoded CBOR internally as a segmented list of strings (e.g., in “ropes”), and more importantly without having the entire segmented input available to compute the string lengths.
Why would one want to do that:

— when the input is too large to fit in memory, and length information is not known ahead of time (“streaming”, performance critical environments, constrained devices);
— as a convenience when the input is already in segmented form (constrained devices, removing the need for allocation and copying).

This convenience for the generator turns to an inconvenience for the consumer.
A generic encoder would have to offer the decoded data in a segmented form (“scatter-gather”) or, as you say, copy (gather) it into separately allocated space.

As you notice, some applications of COSE could be in a space where “streaming” (segmented processing) is natural or even the only way to do things.  So it probably doesn’t make sense to disallow segmented strings for COSE in general.

For EAT, you may want to make the decision that you don’t support segmented processing, alleviating the intrusion of segmentation into the decoder API.

That is indeed a valid decision you may want to make.  But note that this is less of an interoperability decision than a decision to simplify some parts of your systems as a tradeoff to decreased performance or capability in other parts of the system — which decrease you may want to consider insignificant.  It is a bit like deciding that floating point values cannot be used in an EAT, except that this decision is not visible at the data model level.

Grüße, Carsten