Re: [Cbor] Record proposal

[Kris Zyp <[e-mail]>]

To follow up, I did update the proposal/spec to use the tag id range
of 57342-57599. Let me know if you would prefer different ids.
Thanks,
Kris

On Wed, Nov 17, 2021 at 10:52 PM Kris Zyp <[e-mail]> wrote:
>
> That sounds reasonable to me.
>
> To clarify a little bit of the rationale for this:
> The purpose of a global registry, as I understand it, is to define the
> global tag ids that need to be known by independent encoders and
> decoders prior to their encoding and decoding interaction. However,
> ids that can be communicated within a message for the scope of that
> message, don't require global allocation, if they are only used within
> a specified scope. This is just like in a programming language: a
> variable from an inner block may shadow another variable from an outer
> block or a global scope temporarily within its block without affecting
> the outer or global scope (this doesn't require any coordination with
> a registry of globals). And a decoder that really implements this
> proposal would, by definition, be implementing this behavior of
> temporary tag reassignment/shadowing, if it really is a true record
> implementation (i.e. a real record implementation wouldn't see
> gobbledygook if it is conformant).
>
> Also allowing dynamic/temporary allocation permits potential use of
> shorter/more efficient tag ids based on known use of other tags, and
> in the case of record references, these may occur much more frequently
> in a data item/document compared record definitions, and be much more
> sensitive to size differences. Furthermore, this also sidesteps issues
> with having to figure out the appropriate size of a range of ids to
> globally allocate. How many ids should be allocated? In terms of
> percentage of available space, registering/using 256 ids from the tag
> 1+2 range seems like it is effectively equivalent as registering 1 id
> from the tag 1+1 range (and tag 1+4 would definitely be undesirable
> due to the high frequency and sensitivity to size). And I have had
> users of my implementations say they need a few hundred structures to
> be defined and referenceable, so even 256 may be somewhat limiting to
> some users.
>
> But that being said, if you are thinking that it may be too onerous
> for (record) implementators to support this type of potentially
> dynamic/temporary tag reassignment with possibly conflicting ids (as
> opposed to using a more stable table of tags or tag ranges), I can
> understand that concern. What size of chunk of tag ids (above 32768)
> do you think would be appropriate to allocate? Would a chunk of 256 be
> reasonable?
>
> Anyway, thank you for your help and feedback!
> Thanks,
> Kris
>
>
>
> On Wed, Nov 17, 2021 at 1:52 PM Carsten Bormann <[e-mail]> wrote:
> >
> > Hi Kris,
> >
> > thank you for the update.
> > I still need to take a closer look, but I have one immediate reaction:
> > You probably shouldn’t induce implementations to “hijack” a tag number.
> > The handling of a tag number that has actually been allocated for something else may be deeply embedded into the CBOR decoder; a record implementation then would only see gobbledygook.
> > So I think we should allocate some space for tag-allocated tags — I don’t see a big problem with allocating a chunk above 32768 just for the record proposal.
> >
> > Grüße, Carsten
> >
> >
> > > On 2021-11-17, at 16:16, Kris Zyp <[e-mail]> wrote:
> > >
> > > I have updated my tag registration proposal/submission at
> > > https://github.com/kriszyp/cbor-records to use 1+2 tag entries, which
> > > hopefully makes this a much less intrusive registry entry. I have also
> > > updated the proposed tag definitions to also support up-front
> > > declaration of a set of record structure definitions for a data item
> > > ("around" that data item, like the packed approach, as you suggested),
> > > in addition to the current inline record definitions (which can be
> > > scoped with record definitions and should follow a well-specified
> > > order for when they can be referenced by subsequent references). I
> > > hope this offers flexibility for encoders that have all structures
> > > known a-priori and streaming encoders (that do not), while still
> > > maintaining nearly the same mechanics for decoders. Let me know if you
> > > think this looks reasonable.
> > > Thank you!
> > > Kris
> > >
> > > On Thu, Nov 11, 2021 at 9:15 PM Kris Zyp <[e-mail]> wrote:
> > >>
> > >>> Actually, stats would be very interesting.
> > >>> I was assuming that the 1+1 setup comes with a number of 1+2 referencing records, the hit from going to 1+2 there as well would be relatively insignificant.
> > >>> Number are better than assumptions!
> > >>
> > >> You are definitely right, using 1+2 tag for defining records is pretty
> > >> insignificant (around a quarter of percent in my tests). Anyway, I put
> > >> together some tests comparing CBOR packed, record structures, and
> > >> combinations with a couple test data structures, with my library, for
> > >> the sake of further comparisons:
> > >> https://gist.github.com/kriszyp/b623b85d2dc25ac9e3b07d8f39df9307
> > >>
> > >> Anyway, seeing this, I am happy to go ahead and update my proposal to
> > >> use a 1+2 tag for defining records. And thinking about this, I don't
> > >> think my proposal necessarily even needs to mandate the tag ids used
> > >> for referencing records since those are dynamically assigned and
> > >> explicitly specified by the encoder itself (encoder obviously must not
> > >> conflict and use tag ids that will be used for other purposes), albeit
> > >> can encourage a certain range (presumably from the first-come
> > >> first-serve range).
> > >>
> > >> Do you have any preference for a tag id to use? It looks like 279 is
> > >> the next in the contiguous block, but sounds like choosing aesthetic
> > >> characters is the new preference (29299/"rs" perhaps).
> > >>
> > >> Anyway, thanks again for the helpful feedback, really appreciate it!
> > >>
> > >> Thanks,
> > >> Kris
> > >
> > > _______________________________________________
> > > CBOR mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/cbor
> >