Re: [Suit] Transactional updates

[Phillip Hallam-Baker <[e-mail]>]

On Tue, Oct 24, 2017 at 10:44 AM, Hannes Tschofenig
<[e-mail]> wrote:
> Hi PHP
>
> Thanks for the contribution.
>
> I am trying to find out what problem you are primarily trying to solve.  You list lots of topics below. Let me start with one item where you write:
>
> "I am fed up of the time it takes for software updates on my desktop. First the update has to be downloaded and then it has to be installed. Why can't this be instantaneous?"
>
> Clearly, you cannot install software / firmware that has not been downloaded yet. You are not complaining about the slow transfer rate (although this might be a serious issue with many low power radio technologies). You aren't talking about differential updates or updates of individual libraries. Finally, you are not asking for a hitless update either.


Lets look at the sequence of events

1) Download the software update
2) Unpack and write files to directories
3) Stop and restart processes or reboot.

Each of these need to occur obviously and in many cases user
permission is required at at least one point in the process. But the
user experience for the desktop right now is:

1) Informed there is an update
2) Pestered into downloading the update, promised it will be applied
automatically
3) Open machine to discover that the update could not be applied
because software I don't care about was running.
4) Apply the update manually
5) Wait 15 minutes
6) Use machine.

Another vendor has a different approach. I turn the machine on first
thing in the morning and it tells me it needs to start applying
updates.

Neither of these approaches is going to be remotely acceptable for IoT
because the whole point is to save time and you can't save time if you
are giving people systems administration tasks. IoT is the harder use
case with lower capability devices.

It has to be possible for the process to take place automatically and
transparently with the only input from the user (if any) being to
authorize roll forward to the new software or to force a rollback to
the old. So my preferred implementation would be

1) Detect and download update. This is done under prior user permission.
1a) Authenticate and validate.
2) Request user permission (if required, may be granted remotely)
3) Apply update atomically (may be scheduled)

4) Test (optional) Rollback if failed.


> I feel like I am reading that you want to write the downloaded code directly over the code that is being executed. That cannot be true either.

No. What I am saying is that the device acquires the updates and
stores them somewhere. Then it compiles whatever local index is
required to access them and switches from one index to the other as an
atomic operation.

We seem to be fixed in the 1970s notion of files in directories. This
is merely an abstraction what we have in storage is a sequence of
bits. If all we require is read access, we can store data in a zip
file and read it as efficiently as if it was a set of separate files
broken out on disk.

We have to run the reader code on the constrained device of course.
But file creation can take place on a real machine with real
resources.



> So, could you explain your first point in more detail?
>
> Ciao
> Hannes
>
> -----Original Message-----
> From: Suit [mailto:[e-mail]] On Behalf Of Phillip Hallam-Baker
> Sent: 24 October 2017 15:31
> To: [e-mail]
> Subject: [Suit] Transactional updates
>
> I don't know about you, but I am fed up of the time it takes for software updates on my desktop. First the update has to be downloaded and then it has to be installed. Why can't this be instantaneous?
>
> What I would like is to download the software update and then tell the O/S to simply overlay the update on top of the file system as an atomic operation. So installing a software update takes a millisecond, no more.
>
> Rolling back a software update is just a matter of telling the O/S to stop applying the overlay.
>
> It seems to me that this sort of capability would be very useful at the device level as well. Even more so as the chief concern in any software update scheme is that you end up bricking a device. Building the ability to roll updates forwards and backwards is powerful.
>
> I would also like this to be integrated into software signing and in an intelligent fashion so that the signature encompasses both the code and the data and it is possible to validate components independently.
>
>
> There is of course going to be a cost and it is the need for storage.
> The traditional file system organization is a response to highly constrained storage. I have not yet fully considered space optimizations because I have been thinking about the constrained device but I think my approach could be modified for a fairly constrained device. My approach is not going to be appropriate for your PIC controllers but it is definitely feasible at the Arduino level and above.
>
> My approach is based on a Merkle Tree based container format. This may sound intimidating but it shouldn't be. I designed, documented and implemented the scheme in under a week:
>
> https://tools.ietf.org/html/draft-hallambaker-jbcd-container-00
>
> or in HTML with diagrams at
>
> http://prismproof.org/Documents/draft-hallambaker-jbcd-container.html
>
>
> The draft describes a container format that consists of a sequence of variable length chunks that has the property that the sequence can be traversed with equal efficiency in the forward and reverse directions.
> This can optionally be indexed via a binary tree to allow random access to any frame in log(n) time. Chain and (Merkle) tree digests may be constructed which may be signed.
>
> Each frame consists of a header and an optional payload. The payload may be encrypted and/or signed using JOSE. The only fixed requirement for the header encoding is that frame 0 has to have a header encoded in JSON (for interop). The code supports use of JSON, JSON-B, JSON-C and ASN.1 for header encoding, the container uses JSON-B frame encoding because it has to support the read in either direction property.
>
> The original concept here was to provide an append only container format that could be used as the basis for synchronizing Mesh portal logs between nodes and also as an encryption format for end-to-end encrypted Web sites. Static content is straightforward, but people expect to be able to support applications like Web forums. The idea was to see if we could support a Web forum run on a cloud server that does not have the ability to read any of the content (yes).
>
> The authentication scope is intentionally limited to the payloads, and the signature parameters. The headers are not authenticated which allows them to be mutable. A device may store the whole update as a single file or break it apart into attachments. The latter approach making garbage collection of unused attachments possible without a separate sweep/collect cycle.
>
>
> I am not sure that the encryption capability is needed for software update but it might turn out to be useful allowing devices to be shipped with code for a wide range of applications and only enabling the specific modules required. I do not like devices to have more code than is necessary for their purpose. Authentication is obviously necessary.
>
> The way I would see this applied to software update is that the device/desktop would download and verify the incremental updates as a background task (when bandwidth permits). It would then request user permission to apply the update (if required) and set a flag internally telling the loader to use the new version in place of the old. Any processes that needed to be restarted would be restarted.
>
> This could be used to effect updates across multiple devices at the same time. It could also allow support for features such as testing the configuration and performing an automatic rollback if the test failed.
>
>
> Besides providing a more secure process, this provides a better user experience.
>
> The code required to support the container format is comparable to that required to implement a SHA-2 or the like. implementation in a TPM or such would be entirely feasible.
>
> _______________________________________________
> Suit mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/suit
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.