Re: [Suit] Which type of devices?

[Brendan Moran <[e-mail]>]

Hi Dave,

I’ve seen lots of open source projects that rely on pragma pack. Some of them, such as LwIP do so in a safe way, by using pool-allocated memory that is confined to a safe region in memory.

However, I don’t think that relying on pragma pack for general purpose code is a good idea. Here are a few reasons you might want to avoid pragma pack (there may be more!):

1. It’s not part of the standard, so it’s up to you, as a developer, to validate the handling of pragma pack on each target and to ensure that they are compatible. Read the docs, or you might be surprised!

2. As an example of the above, minGW on Windows ignores pragma pack unless you issue -mno-ms-bitfields to the compiler invocation.

3. pragma pack issues different instructions on some architectures: when used with MIPS, some compilers issue unaligned load/store instructions rather than the regular load/store. Architectures that do not support unaligned access will need to issue code for multiple loads, stores, shifts, masks, and ors.

4. Some compilers do not support pragma pack, frequently those targeting proprietary, low end processors of exactly the type that are targeted by this binary format.

5. Unless you can guarantee alignment in keeping with the architecture’s ABI, some accesses will incur an unaligned access overhead, which is not obvious, nor easy to reason about without profiling. (Performance is lower than it should be, and this is architecture-dependent)

6. On some architectures, it can lead to rare and difficult to trace bugs: on ARM v7-m, it’s an exception to issue an unaligned access across memory regions, even if those regions are contiguous. This means that, if you have a buffer allocated across a region boundary and you issue an unaligned access to that buffer, you will trigger an exception, even if that access is legal everywhere else in RAM. I have seen this bug occur in production code.


I don’t like depending on the behaviour of a non-standard extension, only available on some compilers, which is prone to introducing subtle and rarely triggered bugs, in order to make arguments about the performance and efficiency of a proposed standard.

Regardless, I think the important thing is really what the resource utilisation comparison is between a CBOR-encoded manifest and a packed-structure manifest, irrespective of how the packed-structure manifest is interpreted. To make this an easier comparison, I’m working on a example of a special purpose parser, not for CBOR, but for a minimal subset of the manifest described in draft-moran-suit-manifest-03, so that the working group can have numbers available to make an informed decision.

Best Regards,
Brendan Moran

On 9 Nov 2018, at 02:00, Dave Thaler <[e-mail]<mailto:[e-mail]>> wrote:

If you do a web search on “pragma pack” you’ll see an example of what Martin refers to as packing.
It tells the compiler not to insert any padding between fields, so the developer is always in control of the exact structure much like a packet format diagram in an RFC.

This sort of compiler construct is what is often used to implement binary protocols.  It’s a fairly common practice.

It may not be part of C99 but it’s generally supported by compilers in their own way (so if you want cross-platform code, you have to use ifdefs appropriately, but OSS projects often do exactly that).

Dave

From: Suit <[e-mail]<mailto:[e-mail]>> On Behalf Of Martin Pagel
Sent: Friday, November 9, 2018 5:30 AM
To: Brendan Moran <[e-mail]<mailto:[e-mail]>>; Martin Pagel <Martin.Pagel=[e-mail]<mailto:Martin.Pagel=[e-mail]>>
Cc: Hannes Tschofenig <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; Matthias Waehlisch <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Suit] Which type of devices?

Hi Brendan,
Yes, the C99 compiler spec contains certain disclaimers as different processor architectures may pad or store certain data structures in different ways, for example how a compiler/processor implements “Int” or pack Boolean arrays in different ways.
The draft circumvents such issues by only using UInt and by aligning on 16bit boundaries. That way the data structure can be mapped directly by any compiler.
Again, for firmware development, this is pretty common practice.
Martin

From: Suit <[e-mail]<mailto:[e-mail]>> On Behalf Of Brendan Moran
Sent: Thursday, November 8, 2018 4:47 AM
To: Martin Pagel <Martin.Pagel=[e-mail]<mailto:Martin.Pagel=[e-mail]>>
Cc: Hannes Tschofenig <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; Matthias Waehlisch <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Suit] Which type of devices?

Hi Martin,

Perhaps I have misunderstood something. The C99 standard (ISO/IEC 9899:1999) says:

6.7.2.1 Structure and union specifiers
…
12 Each non-bit-field member of a structure or union object is aligned in an implementation defined manner appropriate to its type.

13 Within a structure object, the non-bit-field members and the units in which bit-fields reside have addresses that increase in the order in which they are declared. A pointer to a structure object, suitably converted, points to its initial member (or if that member is a bit-field, then to the unit in which it resides), and vice versa. There may be unnamed padding within a structure object, but not at its beginning.

From my reading, this says that:
1) Alignment of members is implementation defined.
2) Padding in the structure is implementation defined.
3) The only guarantee you have is monotonically increasing address of structure members.

If my reading is correct, then a “proper struct definition” does not change whether or not the compiler is allowed to take liberties with the alignment or padding of a structure. There is no guarantee of consistency between compilers, nor between versions of the same compiler. There may be incidental consistency, but this is not adequate to make an argument about wire format definition, nor about complexity of parsers.

If even just alignment of members is implementation defined (6.7.2.1.12 clearly says it is) then a structure is not adequate to define nor to deserialise a wire format.

Perhaps someone can correct me if I have misinterpreted ISO/IEC 9899:1999.

Best Regards,
Brendan

On 8 Nov 2018, at 05:05, Martin Pagel <Martin.Pagel=[e-mail]<mailto:Martin.Pagel=[e-mail]>> wrote:

Hannes and Matthias,
I think there is an increasingly large spectrum of MCUs with different capabilities, some now include crypto accelerators, then the crypto code overhead is quite small.

We currently use the proposed format in a quite constrained MCU implementation, for both software update and secure boot. Crypto operations are done via crypto accelerator. We use a simple C Struct to access the manifest elements, no need for schema validation. With the proper Struct definition this is compiler safe, Brendan.

Brendan talked about a life cycle on how to drop data elements once they are not needed anymore. This is complexity we would like to avoid on constrained MCUs with a simple memory mapped binary structure.
Martin


-----Original Message-----
From: Suit <[e-mail]<mailto:[e-mail]>> On Behalf Of Hannes Tschofenig
Sent: Wednesday, November 7, 2018 8:35 PM
To: Matthias Waehlisch <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: [Suit] Which type of devices?

Hi Matthias,

The contribution by Martin with his custom binary format (as an alternative binary format to the CBOR/COSE standardized format) is based on his impression that
* a custom binary format allows to reduce the size of the manifest by some (yet unknown) number of bytes, and
* CBOR and COSE libraries are not used in industrial IoT deployments.

I personally think that this discussion is unrelated to the device classes since the overhead really comes from the crypto, as several speakers on the microphone noted.

Ciao
Hannes

-----Original Message-----
From: Matthias Waehlisch <[e-mail]<mailto:[e-mail]>>
Sent: Thursday, November 8, 2018 11:02 AM
To: Hannes Tschofenig <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: RE: [Suit] Which type of devices?

Hi Hannes,

 after Brendan's presentation there were some remarks on this. The discussion made slightly the impression that the group needs to argue why the SUIT solution should also fit into Class 1 devices.

 Based on the charter, the SUIT solution needs to suit Class 1, which has implications on design decisions and on which topic the WG invests time mostly.

 Supporting Class 1 devices is the default, anything else is nice to have.


Cheers
 matthias

On Thu, 8 Nov 2018, Hannes Tschofenig wrote:

Hi Matthias,

Which discussion during the meeting today did you give you the
impression that we are not aiming for this goal?

Ciao
Hannes

-----Original Message-----
From: Suit <[e-mail]<mailto:[e-mail]>> On Behalf Of Matthias Waehlisch
Sent: Thursday, November 8, 2018 10:10 AM
To: [e-mail]<mailto:[e-mail]>
Subject: [Suit] Which type of devices?

Hi,

 I didn't get the discussion today. Just as a reminder, the SUIT WG Charter says explicitly: "This group will focus on defining a firmware update solution (taking into account past learnings from RFC 4108 and other firmware update solutions) that will be usable on Class 1 (as defined in RFC 7228) devices, i.e., devices with ~10 KiB RAM and ~100 KiB flash. "



Cheers
 matthias

--
Matthias Waehlisch
..  Freie Universitaet Berlin, Computer Science ...
https://na01.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.cs.f
u-berlin.de<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fu-berlin.de%2F&data=02%7C01%7Cdthaler%40microsoft.com%7Cbc43b4494264448bded108d645c165bf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636773094283336350&sdata=cpRvt98Vc8z3sBcoGgdmLMXuK5fVf96U%2BgG6l9zs4hk%3D&reserved=0>%2F~waehl&amp;data=02%7C01%7Cmartin.pagel%40microsoft.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=02%7C01%7Cdthaler%40microsoft.com%7Cbc43b4494264448bded108d645c165bf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636773094283346358&sdata=mWNM4gPbtIxzWtJ8OQGEXJUieDlVr%2FWN5cy2Pe6bQtQ%3D&reserved=0>%7
Ca39ff4cf1fd74f0dba5d08d645339c7e%7C72f988bf86f141af91ab2d7cd011db47%7
C1%7C0%7C636772485305445335&amp;sdata=ct5txfAAcRM%2BgHw1GoOImrr3bhU8ou
ByV2kMW%2BjGbHw%3D&amp;reserved=0

_______________________________________________
Suit mailing list
[e-mail]<mailto:[e-mail]>
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.i
etf.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fetf.org%2F&data=02%7C01%7Cdthaler%40microsoft.com%7Cbc43b4494264448bded108d645c165bf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636773094283356367&sdata=MdSXUriIkdMs2g4RtovNxOvknpt3sYDOfvk%2FFyoefN4%3D&reserved=0>%2Fmailman%2Flistinfo%2Fsuit&amp;data=02%7C01%7Cmartin.pagel%40
microsoft.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmicrosoft.com%2F&data=02%7C01%7Cdthaler%40microsoft.com%7Cbc43b4494264448bded108d645c165bf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636773094283356367&sdata=sgjSnW12a6gw1kZSvXvBBn7HC4Cc7%2B4KtjxucYz567o%3D&reserved=0>%7Ca39ff4cf1fd74f0dba5d08d645339c7e%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C636772485305445335&amp;sdata=%2FjHzLtzk04XkWqc
uOAAsYjD9Wz%2FK7xxvsqILxP6Rr%2Fc%3D&amp;reserved=0
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


--
Matthias Waehlisch
...  Freie Universitaet Berlin, Computer Science ... https://na01.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.cs.fu-berlin.de%2F~waehl&amp;data=02%7C01%7Cmartin.pagel%40microsoft.com%7Ca39ff4cf1fd74f0dba5d08d645339c7e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636772485305445335&amp;sdata=ct5txfAAcRM%2BgHw1GoOImrr3bhU8ouByV2kMW%2BjGbHw%3D&amp;reserved=0<https://na01.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.cs.fu-berlin.de%2F~waehl&data=02%7C01%7Cdthaler%40microsoft.com%7Cbc43b4494264448bded108d645c165bf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636773094283366375&sdata=u%2B32lS0NCtsmcG00JUGLtIyjndNDNqLd49zxohC%2FrRM%3D&reserved=0>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

_______________________________________________
Suit mailing list
[e-mail]<mailto:[e-mail]>
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fsuit&amp;data=02%7C01%7Cmartin.pagel%40microsoft.com%7Ca39ff4cf1fd74f0dba5d08d645339c7e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636772485305445335&amp;sdata=%2FjHzLtzk04XkWqcuOAAsYjD9Wz%2FK7xxvsqILxP6Rr%2Fc%3D&amp;reserved=0<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fsuit&data=02%7C01%7Cdthaler%40microsoft.com%7Cbc43b4494264448bded108d645c165bf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636773094283376383&sdata=%2Fdg2L3tRelnUIyKVJDgp44iSq7OwcAaKuqZN0gAPnBc%3D&reserved=0>

_______________________________________________
Suit mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/suit<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fsuit&data=02%7C01%7Cdthaler%40microsoft.com%7Cbc43b4494264448bded108d645c165bf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636773094283376383&sdata=%2Fdg2L3tRelnUIyKVJDgp44iSq7OwcAaKuqZN0gAPnBc%3D&reserved=0>

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.