Re: [Cbor] List of not-well-formed CBOR and test vectors

[Thiago Macieira <[e-mail]>]

On Monday, 29 July 2019 11:49:18 PDT Laurence Lundblade wrote:
> I think I’ve made a comprehensive list of all things that are not well
> formed by going through the latest draft and my decoder. There are about a
> dozen of them. I’ve also created 110 test vectors that cover them pretty
> thoroughly.
> 
> Everything is here
> <https://github.com/laurencelundblade/QCBOR/blob/not_well_formed/test/not_w
> ell_formed_cbor.h> in a C header. The dozen types of non-well-formedness are
> listed as comments in the header file. The test vectors are in an array
> that can be used for testing. It is BSD-3 license.
> 
> I’ve turned up one bug in the RFC’s pseudo code. It doesn’t catch an
> indefinite length string as a segment in another indefinite length string.
> 
> I’d like to get some review, some folks to try it out and such to see if
> I’ve missed anything and all is right. When that is done I’ll make a pull
> request for the draft out it. Probably in about two weeks.

Hello Laurence

I'll add your test vectors to TinyCBOR soon and see if there's anything I 
didn't catch. You're welcome to do the same with my test data, see:
https://github.com/intel/tinycbor/blob/dev/tests/parser/tst_parser.cpp#L1538-L1767

There are 194 entries in that list, though some of them are slightly duplicate 
of one another. There are a couple that also test implementation limits, like 
strings bigger than half your machine's address space. Some others aren't 
testing invalid CBOR, but common parsing mistakes like reading a size of -1 or 
overflowing counters / pointers.

Another that I came across but isn't seen in the test list was that my buffer 
was always followed by a NUL byte, which masked one read-past-the-end. The 
trick for that test is in
https://github.com/intel/tinycbor/blob/dev/tests/parser/tst_parser.cpp#L159-L192
which always places the data to be parsed at the end of a page, followed by a 
page with no read access.

And if you look at https://github.com/intel/tinycbor/blob/dev/tests/parser/
tst_parser.cpp#L1796-L2155. you'll see an extensive list of strict, canonical 
mode and JSON-compatibility testing.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products