Re: [Cbor] correctness of implied top level array?

[Laurence Lundblade <[e-mail]>]

> On Feb 28, 2019, at 12:46 PM, Carsten Bormann <[e-mail]> wrote:
> 
>> On Feb 28, 2019, at 20:23, Joe Hildebrand <[e-mail]> wrote:
>> 
>>> On Feb 27, 2019, at 5:53 PM, Michael Richardson <[e-mail]> wrote:
>>> 
>>> An alternative way to do the desired action (and bring this inside of cbor)
>>> would be an (indefinite?) array type that was specified to concatenate.
>>> 
>>> I'm not really arguing for this, but I think it's worth knowing why this
>>> would be less good a thing.
>> 
>> It's an interesting idea.  One of the reasons this didn't feel right to me was that my initial take on indefinite-length arrays was to read the whole array, growing memory as needed.  I've moved on from that, but would expect others to find themselves in a similar spot.
>> 
>> More interestingly though, there are times when I might want to use an optional external length-framing approach, like embedding a single CBOR data item in a WebSocket message.
> 
> It is hard for a format like CBOR to take in external length information — data items need to be self-delimiting *within* the tree, so taking in that information at the top level would mean that a different encoding would be needed there.
> 
> CBOR sequences are actually exactly that, for the specific case of a top-level array.
> So this is your “array type that was specified to concatenate” — except that this cannot be used within a CBOR data item ([[1, 2], [3, 4]] ≠ [[1, 2, 3, 4]]), only right on top.
> 
> Whether a single data item or a CBOR sequence is expected is indicated by meta information, such as the Content-Type.  We could define more things like CBOR maps, CBOR strings (hey, we already have ct 0 for text and 42 for bytes).  This is becoming complicated quickly.
> 
> CBOR sequences stand out because they are often exactly what is needed for streaming, and for flexible storage of partial streams in containers that can then simply be concatenated.
> 
> Next stop: Getting a CDDL spec to define top-level sequences…  (Right now, we simply describe a top-level array and add English-language information that this is to be encoded as a sequence.  Quite similar to how .cborseq works inside the spec.)

When a decoder receives a CBOR data item, it always knows when it as decoded the whole thing. Through the decoding is knows how many bytes it needs.  So one way to frame this up is by asking what should a decoder do when it has got to the end of a data item. 

With application/cbor, it is an error for there to be more bytes.

With application/cbor-seq, further bytes are expected to be another data item. 

Another way to say this is that application/cbor is truly and fully self-delimiting and application/cbor-seq is not. With application/cbor-seq you rely on the unavailability of more bytes to know it is the end.

Maybe this answer’s Michaels question. If there was an implied indefinite length array around all CBOR, then all CBOR would have to either  a) rely on something external like the end of a transmission or b) have a CBOR break present to know it is the end. I don’t think we want that.

The net of it then seems that a protocol using CBOR should be clear about whether it is application/cbor or application/cbor-seq. (Doesn’t have to use MIME and MIME type and all that, just indicate some how).

LL