Re: [Suit] self-describing format vs fixed/binary manifest structure - pull parser

Martin Pagel <Martin.Pagel@microsoft.com> Thu, 20 December 2018 02:05 UTC

Return-Path: <Martin.Pagel@microsoft.com>
X-Original-To: suit@ietfa.amsl.com
Delivered-To: suit@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FAEE130DE7 for <suit@ietfa.amsl.com>; Wed, 19 Dec 2018 18:05:21 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.065
X-Spam-Level:
X-Spam-Status: No, score=-2.065 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.065, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8DZNIJwsTah2 for <suit@ietfa.amsl.com>; Wed, 19 Dec 2018 18:05:16 -0800 (PST)
Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-eopbgr810130.outbound.protection.outlook.com [40.107.81.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6C260130DDA for <suit@ietf.org>; Wed, 19 Dec 2018 18:05:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I7C5A3zHX080493Xg5Ai0h1CWA9VkUK6I6AtoUrsAlI=; b=AdZ3aFynx6AijH9u97Y34b91jBFU93tI+LZP13PSKBEKsfbCmRanmbDez1mvx/1M0xiyMDr/MP27fYGtjUVjpXTCNxDQK4bRVcyGFtlVm3FCSV5ZCyAcl7AUcuZotn7VkLQ+KpwKThItZafGXJOpgTEFk3QCK6Uuhj0EEtioBz4=
Received: from DM5PR21MB0698.namprd21.prod.outlook.com (10.175.112.13) by DM5PR21MB0746.namprd21.prod.outlook.com (10.173.172.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1471.7; Thu, 20 Dec 2018 02:05:15 +0000
Received: from DM5PR21MB0698.namprd21.prod.outlook.com ([fe80::3807:fbfd:938f:b1c5]) by DM5PR21MB0698.namprd21.prod.outlook.com ([fe80::3807:fbfd:938f:b1c5%10]) with mapi id 15.20.1471.011; Thu, 20 Dec 2018 02:05:15 +0000
From: Martin Pagel <Martin.Pagel@microsoft.com>
To: David Brown <david.brown@linaro.org>
CC: Michael Richardson <mcr+ietf@sandelman.ca>, "suit@ietf.org" <suit@ietf.org>, "dev-mcuboot@lists.runtime.co" <dev-mcuboot@lists.runtime.co>
Thread-Topic: [Suit] self-describing format vs fixed/binary manifest structure - pull parser
Thread-Index: AdSYCHGNvZso6Id+TN2TeD47dEfJkQ==
Date: Thu, 20 Dec 2018 02:05:14 +0000
Message-ID: <DM5PR21MB06984CC3CF3075F362FB410A9DBF0@DM5PR21MB0698.namprd21.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Owner=mapagel@microsoft.com; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2018-12-20T02:05:13.5439224Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=General; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Application=Microsoft Azure Information Protection; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Extended_MSFT_Method=Automatic; Sensitivity=General
x-originating-ip: [2001:4898:80e8:8:325d:971b:ba65:d4db]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; DM5PR21MB0746; 6:xAJtxez+IOT1zQxJQ3mNRsW1+sVHckU473iQ8IVStwsbknPZUcQoBwPP9JmsJY90PKrupmlZT9MABtBmLPiZQHAejHwo3/yB1jI+oKj2SSnruHiVtDKAOdGTMwD9BhKE//HoLQ6ug5SAHg1U5kEosmUlyLKPxSxwVeF7djpH+N9XzLwNcNW/rUgxcyRNDKQETOnlLFV9eeXpKcCNaBRZwbxV/aRm5aLX3VEDwW7F03HASoE/eJdX/4Dbgd0WIsfPSmiGR740Zv6lbv+a+PUj+9PLPWqp8ItcEHt4DNTYuO5hZw1N3qN1+Iu4+W4JvSKDMk3Bf98effWDPQy8QfEbmsoC4AtZdiGmKKOGTz5oeDDcyb+RwVw5hQeeNSS4Qkxlj6bCpYrGEkRoOASDHcnLl6pEqmSYTIzEkUt5szpFmEftiptqjb7S3UbFl+hI24J7884df5SVDyQn3kMJPOBxvw==; 5:HAwYNFqbSpPcD6igEgf9Sg+UxxfElXdx29LayGj4uWRZ3t/2ZsqzwcYahKHfPfkwhugF+GR3uz7N58forzyZDWM5/uHFeCt3g9MnLlCjod1W/TvwBl9CJEHZPeo/enmXvsqhD6vxogWLk6DFZgz5qmSWY9RSow54RVjld4VrOCA=; 7:uNVRsQYmFOey6BmPfC5jvY4o5BsFMyX4BcQt4pDMk4/Pc6y1pRF0Y/pi8WnRQ6aarriQ0svY2IN+C3X9Z0mJP26zReMDfE8N+m3lSICPdAIKsihxicrT9QlX+/4UJXIMEC6/yq1RAdkx7rzgKOZ3wg==
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-correlation-id: 4a01659d-e24d-41eb-6db1-08d6661f9527
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7193020); SRVR:DM5PR21MB0746;
x-ms-traffictypediagnostic: DM5PR21MB0746:
x-ms-exchange-purlcount: 1
authentication-results: spf=none (sender IP is ) smtp.mailfrom=Martin.Pagel@microsoft.com;
x-microsoft-antispam-prvs: <DM5PR21MB07460EC003BE351FD57777F19DBF0@DM5PR21MB0746.namprd21.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(999002)(5005026)(6040522)(8220039)(2401047)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231475)(944501520)(2018427008)(6055026)(149066)(150057)(6041310)(20161123562045)(20161123560045)(20161123558120)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:DM5PR21MB0746; BCL:0; PCL:0; RULEID:; SRVR:DM5PR21MB0746;
x-forefront-prvs: 0892FA9A88
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(39860400002)(396003)(376002)(136003)(366004)(13464003)(189003)(199004)(9686003)(97736004)(99286004)(105586002)(106356001)(6246003)(256004)(14444005)(10090500001)(4326008)(33656002)(53936002)(6436002)(68736007)(6306002)(7736002)(8990500004)(305945005)(55016002)(74316002)(186003)(6506007)(25786009)(8676002)(486006)(966005)(72206003)(10290500003)(478600001)(476003)(8936002)(53546011)(229853002)(81156014)(81166006)(71190400001)(71200400001)(6916009)(316002)(86612001)(7696005)(5660300001)(22452003)(14454004)(46003)(102836004)(2906002)(86362001)(6116002)(54906003); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR21MB0746; H:DM5PR21MB0698.namprd21.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts)
x-microsoft-antispam-message-info: hjCq8lK4sXXM/1rYxZoNVYenAQTpobuuQRICpjxVemZuok8JyUe2IWnh89bADXJg1N2dkqCFzNNIAB2/2ZAXb8+v76BvwaziNMGpaZcSmukQ17iyDDB6KTFX2QZMXXCAnaqHmY1XbpfP3kHLXdctJCTHYvh/q60E37AIXW7JuxcbfNllTsDGdiDIKesvm8jH3hG86drrLQGtmt0utGY4wc6p0iNxH1Ay3ReUGWS5AZVh0G0SbLUm+BJtc8oHnqXbCb5jJpJy5G2McqdYK+7Sa5nSJS1+89xJRjX10VNYhHXCunomcVaKqrBPxTucQ/XY
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4a01659d-e24d-41eb-6db1-08d6661f9527
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Dec 2018 02:05:15.0100 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR21MB0746
Archived-At: <https://mailarchive.ietf.org/arch/msg/suit/BmejRs28cEQu6rq3tZ5fA9xX_3I>
Subject: Re: [Suit] self-describing format vs fixed/binary manifest structure - pull parser
X-BeenThere: suit@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Software Updates for Internet of Things <suit.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/suit>, <mailto:suit-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/suit/>
List-Post: <mailto:suit@ietf.org>
List-Help: <mailto:suit-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/suit>, <mailto:suit-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Dec 2018 02:05:21 -0000

Thanks, David, 
Yes, I am familiar with pull parsers, thanks to Brendan's example, but I'm not sure what the advantage of CBOR encoding provides if you build a custom parser for a particular fixed CBOR encoding/schema. Seems more complicated (and therefore error-prone) to me than using a packed binary structure which apparently MCUboot uses. If I understand correctly, you believe that such encoding wouldn't be appropriate as a manifest. I proposed to use such binary encoding in https://tools.ietf.org/html/draft-pagel-suit-manifest-00. 

I agree that there will be powerful and flexible MCUs which have capabilities where a CBOR  based flexibility would provide benefits, but for simple constrained devices, I think there is a benefit to use the same simple lean packed encoding for both the boot and update process, or am I missing some important aspect? If so, can you provide an example?
Best
Martin


-----Original Message-----
From: David Brown <david.brown@linaro.org> 
Sent: Wednesday, December 19, 2018 9:07 AM
To: Martin Pagel <Martin.Pagel@microsoft.com>
Cc: Michael Richardson <mcr+ietf@sandelman.ca>; suit@ietf.org; dev-mcuboot@lists.runtime.co
Subject: Re: [Suit] self-describing format vs fixed/binary manifest structure

On Thu, Dec 13, 2018 at 11:22:07PM +0000, Martin Pagel wrote:

>In general I actually agree with your assessment. If you have many 
>options, a well defined format like CBOR is much better than custom C 
>structures as you can use a standard, already vetted parsing library.
>The standard parser library turns the CBOR data into a data structure, 
>in IoT usually a pretty complex C structure. As the generic parser does 
>not understand the semantics of the fields (only the syntax as defined 
>by the CDDL), the developer still has to write code to verify a complex 
>data structure and enforce semantics and you still have to test/verify 
>such code.

There has been some good discussion on list about pull vs push parsing.  What you are describing is known as a push parser.  The parser parses the CBOR and converts it into a fairly rich data structure that describes the contents of the CBOR.  This is a common approach, and is quite useful on "powerful" computers, such as host-side tooling to process the data.

However, in a highly-resource-constrained device, this is not typically how it is done.  Usually a "pull" parser is used, which will have entry points to effectively expect a specific (and likely
limited) variant of the manifest, and simply abort if it see what is unexpected.  The data is not converted to another structure, but directly parsed/used in place.  For this kind of use, CBOR works very well.  Draft 3 of the manifest document has an accompanied example parser that works this way.

As a comparitive example, MCUboot uses a custom encoding format for the data in its manifest.  It consists of a small amount of data in a fixed structure (with defined byte ordering and packing), and a set of tag-length-value items (effectively a single CBOR map, with integer keys and all values as bstrs).  This is parsed with a similar custom parser for the data expected.  I expect the parser for the SUIT manifest to be somewhat larger than this, but within a similar magnitude.  Even this simple case of a single map, there is enough variation that a plain struct would not be appropriate to represent this information.

David