Re: [Cbor] Packed CBOR and dictionaries

[Jim Schaad <[e-mail]>]

-----Original Message-----
From: Michael Richardson <[e-mail]> 
Sent: Friday, August 28, 2020 11:21 AM
To: Jim Schaad <[e-mail]>; [e-mail];
[e-mail]
Subject: Re: [Cbor] Packed CBOR and dictionaries


I feel that the WG should adopt the document already.

[JLS] The WG was promised an updated draft that dealt with a couple of
issues before the adoption call was made.

Jim Schaad <[e-mail]> wrote:
    > * Should the dictionary expansion be separate or part of this draft?
I am
    > not sure how I want to address this.  If you have a dictionary w/
50,000
    > entries in it, that is going to change how things should be done.  It
may
    > also be that one might want to use a dictionary entry for something
that
    > might otherwise be encoded as a prefix and the prefix might not be
needed
    > anymore.

Once the dictionary is larger than 512, I guess 131072 is the next size.
That uses four-byte references, and so the dictionary ought to provide at
least four-byte substitutions, right?
Otherwise, we'd be expanding rather then compressing.
It seems that a (C,Python,etc.) array is probably always appropriate as the
internal datastructure to perform lookups into the dictionary. It shouldn't
require a sparse array, ever, should it?

[JLS] That raises a different question.  At one point when I was emailing
Klaus about the fact that dictionaries in CoRAL are only defined to use
positive integers, there was some discussion that the negative values would
be for a more application specific version of a dictionary.   That would
make it 256 and 65,XXX.

It might be interesting to have "sparse" dictionaries.  Consider that you
might want all of the URLs without a fragment to occur at a smaller section
that the same set of all URLs with fragments.  This means that you can get
smaller compressions for CoRAL vocabulary that does not include all of the
possible values using just the base URL.

Jim



    > Being able to do packed is going to be of importance for doing CoRAL,
but
    > just as important is being able to do dictionaries.  Dictionaries do
have
    > the downside that if they are not referenced internal to the structure
then
    > from a security point of view they can be problematic as the
    > signed/encrypted CBOR byte stream is no longer self-contained.  This
is not
    > a problem for packed CBOR as long as the packing does not cross the
security
    > boundary.

Agreed.


--
Michael Richardson <[e-mail]>, Sandelman Software Works  -=
IPv6 IoT consulting =-