Re: [Cbor] tag 24, CBOR sequence and non-CBOR sequence

[Laurence Lundblade <[e-mail]>]

Well glad I asked. 

> On Jun 5, 2020, at 11:50 AM, Carsten Bormann <[e-mail]> wrote:
> 
> Hi Laurence,
> 
>> On 2020-06-05, at 17:13, Laurence Lundblade <[e-mail]> wrote:
>> 
>> Here’s some example of CBOR sequence (e.g., application/cbor-seq) and CBOR that is not a sequence (e.g., application/cbor)
>> 
>> 0x00 
>> This is a legal sequence or non-sequence
> 
> “CBOR data item” (or “encoded CBOR data item”) is my preferred wording for the latter.
> 
>> 
>> 0x00 0x00
>> This is a legal sequence, but not non-sequence. A non-sequence can contain only one data item, this contains two.
> 
> Right.
> 
>> 0x82 0x00 0x00
>> This is a legal sequence or non-sequence. It is how you encode two zeros as a non-sequence.
> 
> Right.
> 
>> Looks to me that tag 24 cannot contain a CBOR sequence.
> 
> Indeed, no.

How about stating it like this? While tag 24 can contain a sequence of length one, it cannot contain most sequences as defined in 8742. 

Personally, I’d prefer to just say tag 24 can’t contain sequences. That way you’d have tag 24 for non-sequences and new tag XX for sequences. You don’t want people designing a protocol that uses sequences and using tag 24 when the sequence happens to be length 1 and tag XX when it is larger. That’s just confusing and more code.

Basically a protocol designer should either design a protocol that uses sequences or one that doesn’t and declare that up front. If some messages in a sequence-based protocol happen to be just one item, they are still part of a sequence-based protocol.

> 
>> It says “a data item”. The use of “a” implies one data item and thus not a sequence. That means a decoder should always consider a sequence in the contents of a tag 24 invalid.
> 
> (A sequence that doesn’t happen to contain exactly one data item.)
> 
>> This seems overly restrictive and unfortunate to me.  Would it be wrong for a generic decoder to allow this?
> 
> Yes.
> (Well, it might not be checking the validity.)
> 
> It seems we forgot to define an equivalent to tag 24 in RFC 8742.
> But a registration for such a tag can easily be made now.
> 
>> The wording for tag 55799 avoids the issue of sequence or not.
>> 
>> 
>> Technically, it seems that something labeled application/cbor or in a file named foo.cbor can’t contain the CBOR 0x00 0x00 because it is not a legal non-sequence.
> 
> Correct.
> 
>> That might implies a generic decoder should have a sequence mode and a non-sequence mode.
> 
> Yes.  You can always use the sequence mode, though (just check that you got exactly one data item back).
> 
> My preferred API asks the generic decoder to decode one CBOR data item from a position given and returns the position of the first input byte beyond that CBOR data item.
> A sequence decoder is built trivially from this.
> But you can also use it for cases where a CBOR data item is the “header” for some blob that is not decoded as CBOR.

QCBOR is a more powerful generic decoder than the one implied by the API you describe. It aims to relieve the protocol implementor of a lot of these details and hard-to-understand aspects of CBOR. It aims to make the work of the protocol implementor a lot easier. As a shared library it reduces overall object code because it handles things common to many protocols (e.g. duplicate detection (coming soon)).

It’s of course fine to have all different sorts of decoders.

I find the distinction between sequences and non-sequence subtle and it took until this conversation about tag 24 (started by mcr) to really get it sharp for me. It was definitely one of the things that was confusing to me when I started working with CBOR a few years ago. Easy to gloss over in a way, but a niggling issue that finally came to light. Seems like it wasn’t just me.

One of the reasons I try to do more in QCBOR is to help others get these kinds of details right. They are hard to to understand from the RFCs. The average protocol implementor won’t spend the five plus years reading them we all have here.

> 
>> In practice this doesn’t seem like a very useful restriction. What is useful is whether the encoded CBOR is legal for the protocol being implemented.
> 
> If the protocol wants one CBOR data item and gets two (or zero), that doesn’t help.

Protocol implementors have to deal with lots of errors.

> 
>> From what I’ve experienced, it seems easier if everything is just a CBOR sequence and no distinction is made. Individual CBOR protocols can decide on their own structure either deciding to bind things together in an array or not. Am I missing something?
> 
> I think the important observation is that it took us a couple of years to rationalize the weird text about “streaming decoders” in RFC 7049 into a clear concept of CBOR sequences.
> So there is a lot of code out that that doesn’t cope very well with CBOR sequences, just with single CBOR data items — and that is often also exactly what’s needed.

QCBOR is fine with sequences or not as input. The CBOR playground doesn’t allow sequences.


Personally, I still think it would be better to relax the distinction and allow tag 24 to carry sequences rather than define a new tag. It is a broadening, not a narrowing so it doesn’t break anything. There’s no backwards compatibility issue just like there was no backwards compatibility issue for 7049 when 8742 was published. 

If not relaxing the distinction and a new tag is defined for 8742, maybe 3.4.5.1 should say that 8742 CBOR sequences are invalid as content for tag 24. It is OK for CBORbis to make normative reference to 8742.

LL