[Cbor] Failure Testcases (was: RE: Simple Value Example)

Richter, Jörg <Joerg.Richter@pdv-FS.de> Wed, 30 January 2019 12:55 UTC

Return-Path: <Joerg.Richter@pdv-FS.de>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E7D34130F7F for <cbor@ietfa.amsl.com>; Wed, 30 Jan 2019 04:55:23 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.921
X-Spam-Level:
X-Spam-Status: No, score=-0.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FROM_EXCESS_BASE64=0.979] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nLC6UcMA2jZp for <cbor@ietfa.amsl.com>; Wed, 30 Jan 2019 04:55:21 -0800 (PST)
Received: from mail.pdv-fs.de (mail.pdv-fs.de [213.208.220.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4520112D4E7 for <cbor@ietf.org>; Wed, 30 Jan 2019 04:55:21 -0800 (PST)
Received: from EXCHDB1.pdv-fs.de (192.168.180.94) by EXCHDB2.pdv-fs.de (192.168.180.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Wed, 30 Jan 2019 13:55:19 +0100
Received: from EXCHDB1.pdv-fs.de ([fe80::6c4a:8b1b:60f4:4437]) by EXCHDB1.pdv-fs.de ([fe80::6c4a:8b1b:60f4:4437%15]) with mapi id 15.01.1466.012; Wed, 30 Jan 2019 13:55:19 +0100
From: "Richter, Jörg" <Joerg.Richter@pdv-FS.de>
To: Carsten Bormann <cabo@tzi.org>
CC: "cbor@ietf.org" <cbor@ietf.org>
Thread-Topic: Failure Testcases (was: RE: [Cbor] Simple Value Example)
Thread-Index: AdS4l1VPJq5DtGkdQeyJSip0eYM1Aw==
Date: Wed, 30 Jan 2019 12:55:19 +0000
Message-ID: <3da886c762564d20a75796733ec73e23@pdv-FS.de>
Accept-Language: de-DE, en-US
Content-Language: de-DE
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-originating-ip: [192.168.180.93]
Content-Type: multipart/mixed; boundary="_003_3da886c762564d20a75796733ec73e23pdvFSde_"
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/wwrHDFJyNNxfxthN3LaFcrwphLw>
Subject: [Cbor] Failure Testcases (was: RE: Simple Value Example)
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Jan 2019 12:55:24 -0000

> > BTW: Is there a collection of “corrupt” CBOR messages that must produce
> > a decode error?
> 
> We have talked about this:
> 
> https://github.com/cbor-wg/CBORbis/issues/16
> 
> But so far nobody has started collecting instances.
> Do you have some we could start with?
> I was planning to scan some of the open source CBOR packages for tests of
> failing decoding.

Okay. I've been thinking about it for a while.

One set of test cases can be easily constructed from valid test cases. 
Just decode a prefix of a valid test case and the decoder should fail with 
an "end of message" error.  Just generate them during normal testing.

I think a second set of failing test cases can be constructed by just 
considering which CBOR prefixes are invalid. This test cases should of course 
fail with a different error than "end of message".

I wrote and attached a simple Lua script to generate all invalid 1 and 2 byte 
prefixes.

This script outputs 578 test cases.  Also attached.

Another set of test cases can be constructed when using tags.  But I think
this is more involved and not clearly defined.  But I might be wrong.
Consider for example a bigfloat. Is it allowed that the array elements themselves
are using a tag?  

Or what if a tag is followed by an element that is not allowed for this tag.  
Like following a tag for a set (tag=258) with a string instead of an array?  Should this 
cause a decode error?

Or is it allowed that Tag 55799 (0xd9d9f7) is the only content of a CBOR file?

- Jörg