Re: [Cbor] tag 24 and 55799 (was Re: my (WGLC re-)views on error processing in RFC7049bis and future-proofing)

[Laurence Lundblade <[e-mail]>]

Really appreciate the discussion here (despite my incorrect statements).

> On May 26, 2020, at 2:43 PM, Carsten Bormann <[e-mail]> wrote:
> 
>> On 2020-05-26, at 23:04, Laurence Lundblade <[e-mail]> wrote:
>> 
>>> On May 15, 2020, at 6:40 PM, Michael Richardson <[e-mail]> wrote:
>>> 
>>>>> I note that AFAIK, we do not use tag#24 (Encoded CBOR data item) for
>>>>> the signed object, in COSE.  Should we?  What's the difference between
>>>>> #24 and #55799.
>>> 
>>>> 55799 is a tag that can have any CBOR data item as tag content 24 is a
>>>> tag that can only be on byte strings.  The byte string then *encodes*
>>>> another CBOR data item.  (The main use here is to keep the decoder from
>>>> decoding, to provide easy skip-ability or because we need exact bytes
>>>> as in COSE.)  As often with tags, there is no need for tag 24 on a byte
>>>> string when it is clear from context that the byte string contains
>>>> encoded CBOR; this is the case in COSE.
>>> 
>>> Understood.
>> 
>> My answer on the difference is that you use 55799 when the surrounding data / file / protocol is not CBOR and 24 when it is CBOR. 55799 is intended to work as a magic number, 24 is not because it is not unique enough.
>> 
>> From a decoder point of view, they should be handled exactly the same
> 
> Actually, no.
> 
> 55799 has any CBOR data item as tag content and essentially has the semantics of that CBOR data item.

This is well-formed and valid CBOR by RFC 7049 & CBORbis:

D9 D9F7          # tag(55799)
   81            # array(1)
      D9 D9F7    # tag(55799)
         D9 D9F7 # tag(55799)
            01   # unsigned(1)


One is tempted to say that 55799, as a true CBOR no-op, must always be ignored by generic decoders. However, I think most implementations of CBOR protocols (e.g., CWT, COSE) today would not tolerate 55799’s showing up anywhere in the encoded CBOR like in my example above because ignoring it universally has never been the rule.

So you really need to mean it when you put tag 55799 in, the same as any other tag. Its use needs to be mandated as part of the protocol specification so implementors know it will be occurring and are ready to ignore it.

Probably it is not even CBOR protocol specifications that says to use 55799. It is the protocol or file format that carries the encoded CBOR that would indicate it is in use.

Probably 55799 should never occur except at the beginning of some encoded CBOR.


> 
> 24 has a byte string as tag content.  That byte string is identified by the tag as encoded CBOR. 
> A byte string with embedded encoded CBOR is a data item, but it is different from the data item that was encoded and embedded. Decoders will differ in their tag 24 handling: they might go ahead and decode that CBOR or just hand the byte string and tag to the application.  In either case, the decoded CBOR is not simply “in place”, making the tag and the byte string vanish.

Yes of course (I should have known that).

A generic decoder could have a mode that skips the tag24-plus-byte-string the same it skips 55799, but it shouldn’t be the default because protocols usually assign some semantics to it such as having it hashed or distinguishing it from JSON (protocols should never assign such semantics to 55799).


> This is kind of twisted, but seems legal.
>>  Encoding: 
>>    - start with an RFC 3339 date string
>>    - base 64 encode it and tag it as so with tag 34
> 
> Why would one want to do that?
> 
>>    - tag it with tag 24 (55799 ) or to say it is CBOR
> 
> Ignoring the 55799 case, you are missing one encoding step (tag 24 requires a byte string with encoded CBOR, not a tagged text string).
> 
>>    - tag it with tag 0 to say it is a date string
> 
> Not allowed.  This only takes major type 3, no tags.
> 
>>    - tag it with 22 to say it should be b64’d if re-encoded later

I’m after a deeper understanding of how base64, tag 24 and 55799 interact or don’t interact. My date string example was poor (sorry).

Base64 is really a content transfer encoding, not a data type.  The original data before base64 encoding is of some type or other. Whom ever processes it after the base64 encoding is removed has to come to know what type it is.

However, CBOR generally treats base64 as a data type itself and as it is has no means for indicating the type of the original data before base 64 encoding was added (by contrast, MIME can describe both type and transfer encoding).

So how can the receiver of base64-tagged data know the type of original data? It pretty much has to be in the definition of the CBOR protocol. It might say it is always an X.509 cert or an elf executable or such. It might say it is looked up by magic number (in which case 55799 might apply). Lots more options...

It is also possible to define tags whose content type can be bstr, base64, base64-url. If it was really common to base64 encode X.509 certs (maybe because of JOSE) and a tag was defined to indicate an X.509 cert, it might be useful to say the X.509 tag content could be bstr, base64 or base64-url.

No suggestion for change and nothing at issue so far, just thinking through how it fits together.

What would really be helpful to me is some detailed example of when you’d use the base64 tags. Why would you ever use them when CBOR carries binary data just fine? Why wouldn’t every CBOR-based protocol just say you always strip the base64 before CBOR encoding? And why wouldn’t every CBOR bstr always be base64 encoded when translating to JSON?


> 
> Well, not “it”, but any byte string in the CBOR data item.
> So, here it would base-64 encode for JSON conversion the byte string in tag 24 (the tag 24 itself would presumably be stripped in any JSON conversion, but I’m already confused — you can do “JSON in JSON” as well, just not tag it).
> 
>> Decoding:
>>    - remove the base 64 encoding because of the tag 34
> 
> That is inside, so you see it last.
> 
>>    - feed it back to the CBOR encoder because of tag 24
>>    - interpret it as an RFC 3339 date because of tag 0
> 
> Lost track.
> 
>> Base64 encoding / decoding is not that much code or that difficult, so a generic decoder might actually do this.
> 
> I wouldn’t touch that with a ten-foot pole — it might convert that pole into base64 without asking me.

I wasn’t suggesting any base64 decoding conversion that wasn’t indicating by a tag in an input stream or any base 64 encoding that wasn’t requested through an explicit API request. Section 3.4.5.3 seems to suggest the decoding is a good thing to do:

3.4.5.3 <https://tools.ietf.org/html/draft-ietf-cbor-7049bis-13#section-3.4.5.3>.  Encoded Text

   Some text strings hold data that have formats widely used on the
   Internet, and sometimes those formats can be validated and presented
   to the application in appropriate form by the decoder.  There are
   tags for some of these formats.  As with tag numbers 21 to 23, if
   these tags are applied to an item other than a text string, they
   apply to all text string data items it contains.

Is there anything wrong with a generic decoder validating and removing bas64 encoding when encountering tag 34?

Also see this issue <https://github.com/cbor-wg/CBORbis/issues/194> I filed against this text and Table 5.

Thanks very much.

LL