[Cbor] Re: Bundle protocol RFC 9171 design problems

[Brian Sipos <[e-mail]>]

Thank you for this analysis. I can explain the rationale for the Wireshark
implementation is twofold: the BPv7 addition just used existing crc
functions (which are one shot, not incremental), and accessing contiguous
data from a tvbuff_t structure will usually involve a buffer copy anyway
because these PDUs will usually be the result of segmentation reassembly
from multiple capture frames.

If there were a pressing need, the Wireshark CRC API could be made to act
on a tvbuff hierarchy directly and be zero-copy. But that is a substantial
change to that API.

Brian S.

On Sat, Jul 11, 2026, 18:47 Vadim Goncharov <[e-mail]> wrote:

> On Thu, 9 Jul 2026 12:29:57 -0700
> Laurence Lundblade <[e-mail]> wrote:
>
> > I go on too long; summary:
> >  - Brian is right. Modifying the decode input buffer is not needed with
> > incremental (sequential, chunked…) hash/crc implementations
> >  - Seems possible to support Bundle with some generally useful and
> long-term
> > maintainable APIs
> >  - Byte string wrapping would have allowed implementation with every CBOR
> > library, even simple-minded ones
> >
> > For decode, what is needed is the pointer and length of the CBOR items
> that
> > are input to the hash/crc. In the case of the array holding a bundle
> block,
> > that means decoding everything in the array. An API that returns the
> start
> > (a pointer) and length of a full CBOR item like an array with all its
> member
> > items seems like a general and useful design. This seems OK as a
> long-term
> > supported general feature for QCBOR.
>
> Actually, as you will see below, this is enough for Bundle *when* there is
> no
> 0xFF after CRC field (and even then, you just have to branch on two
> possible
> values), for which may be errata needed, I don't sure. This is
> continuation of
> https://mailarchive.ietf.org/arch/msg/cbor/_NdRMMh6LMB4GA60ahJ0IV7tjz8/
> message with research about CRC and Bundle implementations.
>
> > This API should work for CBOR::Core too.
> >
> > But, Bundle has one more issue. The hash/crc calculation can’t be done
> over
> > all the members in the array because the CRC, which has to be substituted
> > with zeros, is in the array. The Bundle design recognized this and always
> > puts the CRC last, so you can just subtract 2 or 4 from the full length.
>
> It can, if you're about decoder, see below.
>
> > If byte string wrapping were used, the recursive decode is not needed
> > because the length is (redundantly) present. The byte string can be
> > efficiently decoded and hashed with any simple-minded CBOR library
> without
> > any extra API. (Score card: fxamaker has the advanced API, QCBOR will add
> > it, Python cbor2 doesn’t)
> >
> > I expect to support any API I put in my library forever with full
> backwards
> > compatibility, regression tests, and documentation. I expect Bundle
> > implementations to want that from libraries they use and want to provide
> > that to their consumers.  I don’t want future features to be pinned in by
> > backwards compatibility for dumb APIs that weren’t thought through.
> That’s
> > why I’m fussing more here and why byte string wrapping is beneficial.
> >
> > On the encode side:
> >     AddRewritableBytes(QCBOREncodeContext *pCtx, UsefulBufC Bytes,
> > QCBORRewriteHandle *Handle); RewriteBytes(QCBOREncodeContext *pCtx,
> > QCBORRewriteHandle Handle, UsefulBufC Bytes);
> >
> > The Handle and Rewrite API are to provide pointer safety and a way to
> manage
> > future features that might interact where just returning a target pointer
> > and length wouldn’t.
>
> So, continuing message about Wireshark and other, I've played with CRC,
> and I
> now can say you *can* check whether CRC32c is correct because it is last
> field.
>
> Bundle refers to RFC 7143 for CRC32c (which has same definition that CRC is
> put in last 4 bytes which are filled with 0's during calculation) and it
> (p. 231) states:
>
>    - The CRC bits are mapped into the digest word.  The x**31
>      coefficient is mapped to bit 7 of the lowest numbered byte of the
>      digest, and the mapping continues with successive coefficients and
>      bits so that the x**24 coefficient is mapped to bit 0 of the lowest
>      numbered byte.  The mapping continues further with the x**23
>      coefficient mapped to bit 7 of the next byte in the digest until
>      the x**0 coefficient is mapped to bit 0 of the highest numbered
>      byte of the digest.
>
>    - Computing the CRC over any segment (data or header) extended to
>      include the CRC built using the generator 0x11edc6f41 will always
>      get the value 0x1c2d19ed as its final remainder (R(x)).  This value
>      is given here in its polynomial form (i.e., not mapped as the
>      digest word).
>
> [... p.260]
>
> A.4.  CRC Examples
>
>    Note: All values are hexadecimal.
>
>    32 bytes of zeroes:
>
>       Byte:        0  1  2  3
>
>          0:       00 00 00 00
>        ...
>         28:       00 00 00 00
>
>        CRC:       aa 36 91 8a
>
>    32 bytes of ones:
>
>       Byte:        0  1  2  3
>
>          0:       ff ff ff ff
>        ...
>         28:       ff ff ff ff
>
>        CRC:       43 ab a8 62
>
> That is, if CRC is calculated over data with CRC put into it, where prior
> CRC
> was of that data plus four zero bytes, then such calculation will always
> result in SAME number iff CRC is correct (data not corrupted)
>
> There is some play with mapping of CRC word (above definition),
> little-endian
> vs big-endian, so playing with Perl module which allows different CRCs by
> proper configuring, have following results:
>
> $ perl -MDigest::CRC -e '
>   my $ctx1 = Digest::CRC->new(width => 32, poly => 0x1EDC6F41, init =>
> 0xFFFFFFFF, xorout => 0xFFFFFFFF, refin => 1, refout => 1);
> $ctx1->add("\x00"
> x 32); print sprintf("Step 1 (CRC for 32 zero): %08x\n", $ctx1->digest);
> '
>
> Step 1 (CRC for 32 zero): 8a9136aa
>
> $ perl -MDigest::CRC -e '
>   my $ctx1 = Digest::CRC->new(width => 32, poly => 0x1EDC6F41, init =>
> 0xFFFFFFFF, xorout => 0xFFFFFFFF, refin => 1, refout => 1);
> $ctx1->add("\xff"
> x 32); print sprintf("Step 1 (CRC for 32 1): %08x\n", $ctx1->digest);
> '
>
> Step 1 (CRC for 32 1): 62a8ab43
>
> --> this is what you get for RFC 7143 first two test vectors, so here we
> know
>     that initialization parameters were correct.
>
> $ perl -MDigest::CRC -e '
>   my $ctx1 = Digest::CRC->new(width => 32, poly => 0x1EDC6F41, init =>
> 0xFFFFFFFF, xorout => 0xFFFFFFFF, refin => 1, refout => 1);
> $ctx1->add("FreeBSD"); print sprintf("Step 1 (CRC for FreeBSD): %08x\n",
> $ctx1->digest); '
>
> Step 1 (CRC for FreeBSD): f0883564
>
> $ perl -MDigest::CRC -e '
>   my $ctx = Digest::CRC->new(width => 32, poly => 0x1EDC6F41, init =>
> 0xFFFFFFFF, xorout => 0xFFFFFFFF, refin => 1, refout => 1);
>
>   # Add four bytes of CRC in reversed order
>   my $data = "FreeBSD" . pack("C*", 0x64, 0x35, 0x88, 0xf0);
>
>   $ctx->add($data);
>   print sprintf("End-to-end verification result: %08x\n", $ctx->digest);
> '
>
> End-to-end verification result: 48674bc7
>
> $ perl -MDigest::CRC -e '
>   my $ctx1 = Digest::CRC->new(width => 32, poly => 0x1EDC6F41, init =>
> 0xFFFFFFFF, xorout => 0xFFFFFFFF, refin => 1, refout => 1); $ctx1->add("yet
> another test"); print sprintf("Step 1 (CRC for yet another test): %08x\n",
> $ctx1->digest);
> '
>
> Step 1 (CRC for yet another test): 7b165e79
>
> $ perl -MDigest::CRC -e '
>   my $ctx = Digest::CRC->new(width => 32, poly => 0x1EDC6F41, init =>
> 0xFFFFFFFF, xorout => 0xFFFFFFFF, refin => 1, refout => 1);
>   # Add four bytes of CRC in reversed order
>   my $data = "yet another test" . pack("C*", 0x79, 0x5e, 0x16, 0x7b);
>
>   $ctx->add($data);
>   print sprintf("End-to-end verification result: %08x\n", $ctx->digest);
> '
>
> End-to-end verification result: 48674bc7
>
> As you can see, it is always 48674bc7 !
>
> What is 48674bc7 ?
>
> See, original from RFC 7143 = 0x1c2d19ed which is
>
>    0001 1100 0010 1101 0001 1001 1110 1101
>
> mirror reflection of it (see "mapping digest word") is
>
>    1011 0111 1001 1000 1011 0100 0011 1000
>
> or 0xb798b438, and
>
>    0xb798b438 XOR 0xFFFFFFFF = 0x48674bc7
>
>
> To conclude, IF the CRC field in Bundle is definite-length array, then all
> the
> API you have to have in decoder is just to feed entire array's
> representation,
> even incremental adding is not needed. But IF if is INDEFINITE-length
> array,
> then you'll also get stable number - given that of state of the
> accumulating
> register and single FF byte, which is always same in such situation.
>
> I am not sure if an Errata should filed against 9171 stating that array of
> block must be of definite-length, so that optimization using this CRC
> property
> for non-copy could be used without branching. Two implementations (Go and
> Wireshark) I've digged in previous message - do not use it and instead copy
> memory.
>
> --
> WBR, @nuclight
>