Re: [Suit] On device resets during manifest processing

["Rønningstad, Øyvind" <[e-mail]>]

Hi Brendan, thanks for the detailed reply. Note that I finished this email before your second email with the proposed manifest text. I will reply to that separately.

I think this is actually moderately simple to do and it's my preferred alternative. The rules are:
1.       Condition Failures are recorded because they alter the flow of the manifest
2.       Nonvolatile state-altering commands (Copy, Fetch, Swap) are recorded because we need to know which failed if there is more than one between other records
These are stored in a journal, like the SUIT Report, and used to guide the manifest parser until it runs out of journal entries. Because a Copy/Fetch with a volatile target is a potential source of problems in this approach, we may need to perform those again. This might become a challenge. I think it might take some more consideration to work through this question.

I agree. Hopefully there is a simple ruleset that can be used.

Progress tracking in non-volatile memory
===============================

Essentially, this is a journal approach. The manifest parser uses the journal (a SUIT Report, for example) along with the manifest to recreate the parser state immediately before failure.

On startup, the manifest processor checks for a SUIT Report matching the manifest in its report storage area. If there is not one, it creates one and starts processing the manifest.

If there is already a SUIT Report matching the manifest, then it begins processing the manifest, using the SUIT Report to determine what state it should hold-since that is the purpose of the SUIT Report-effectively treating each command as a "dry run" if it has a corresponding entry in the SUIT Report.

When it comes to the end of the SUIT Report, it knows that all subsequent commands did not execute to completion. There's a small hiccup here: it's not currently mandatory for commands that modify non-volatile system state to append to the SUIT Report. We can fix that by ensuring that they have the correct reporting policy when constructing the manifest (https://datatracker.ietf.org/doc/html/draft-ietf-suit-manifest-17#section-8.4.7<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-suit-manifest-17%23section-8.4.7&data=05%7C01%7COyvind.Ronningstad%40nordicsemi.no%7Cb88835e0fdcf4427f24808da5f9f9b7f%7C28e5afa2bf6f419a8cf6b31c6e9e5e8d%7C0%7C0%7C637927436364714700%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GHrM%2FGl6qp2M%2FlMwGd55EmtLUkPyjWQ76knsGzS7BK8%3D&reserved=0>) or we can override the default behaviour of reporting policy for those commands in the Manifest Processor.

Since this is an application-specific mitigation for power loss/reset, I would prefer to have the Manifest Processor override the specified reporting policy for any commands that alter non-volatile system state.

I agree a SUIT Report should be a good way to perform the caching. I believe we should take a few steps to ensure that using reports for the sake of reset resilience doesn't interfere with other uses for the reporting machinery. Sometimes, a command needs to be reported for the sake of auditing (or something else), but needs to NOT be reported for the sake of reset resilience. And sometimes the opposite. If we were to add additional bits to suit-reporting-bits specifically for the purpose of reset resilience, that would be more or less analogous to the separate directive I proposed. I think that could be a good way to make this type of reporting independent of the other types.

QUESTION:
Should we add some language around using a SUIT Report for manifest resumption in scenarios where the manifests may have enough complexity that simply re-running the manifest will not result in a correct installation?

I think we should have language explaining the problem and proposing a solution. If using SUIT Reports becomes our proposed solution, then the proposed mechanism should be fully explained.

This doesn't have to be a SUIT Report, but it is a convenient way to express the idea.

I agree

Application-Specific Command-Resumption
=================================

We have discussed this in connection with MCUBoot support for image swap in the past. MCUBoot uses a resumable image swap algorithm that will continue even if it is interrupted. This could potentially be applied to other operations that modify non-volatile state, such as applying a differential update. I'm not sure how applicable it is to non-diff, non-swap operations.

This also applies to copy, if it supports overlapping source and destination (think memmove() vs memcpy()). I think this should not be in the scope of copy, since it will rarely be necessary, and when it is, it can usually be reproduced via multiple non-overlapping copy operations.

PROPOSED:
A command that alters system state in a way that prevents a repeated execution of the command from completing correctly MUST provide a journal that it can use to resume execution correctly.

I agree, just want to clarify that this is about atomicity: The command has to EITHER revert to its before-state (this is entirely possible with swapping at least), OR it must be completed. After that, other resumption logic can run. I.e. there's (at least) two levels of resumption logic, first complete or roll back the interrupted command (if needed), then complete or roll back the command sequence.

Partial Completion Detection in the Manifest
=================================

Using the Try-Each block we can build a number of constructs that allow us to resume in the correct state. For example, in the sequence you provided:
1.       Copy from Component ID 100 to 200
2.       Copy from Component ID 300 to 100

This can be replaced in a reasonably straight-forward way:
1.       If not Image Match Component ID 200
1.       Copy from Component ID 100 to 200
2.       If not Image Match Component ID 100
1.       Copy from Component ID 300 to 100

This uses the existing manifest structure to check for a partial completion in a repeatable way.

This does indeed solve the problem nicely. It makes me think that maybe there should be an alternative to copy, which merges the copy and the image match, essentially serving as:


  1.  Try-each:

     *   (sequence a)

                                                   i.      Image match

                                                 ii.      Copy

                                               iii.      Image match

     *   nil



This second example, however is more challenging.
1.       Copy from component Foo (non-volatile) to component Bar (volatile)
2.       Copy from somewhere to Foo
3.       Copy from Bar to Baz (non-volatile)
I understand this conceptually, however I'm having difficulty seeing whether we should consider it as a use case that dictates how we construct the specification. The actions themselves are inherently risky, regardless of whether or not SUIT is involved.

Yes, this was just to demonstrate that it is possible to construct these sequences "by mistake". I fully believe there's always better ways to construct these. There should be some text discouraging or prohibiting these sequences. One rule could be:

"If a component serves as a copy source for a volatile component, it cannot itself be modified before that volatile component has been unlinked."

Maybe we need a tool for correctly generating manifests under these conditions?

That would be very helpful, given we are able to formulate the ruleset. I feel that, depending on the complexity of the ruleset, it either works better as a tool during manifest generation, or as part of the manifest processor.

I think this is a great example of reconstruction using a SUIT Report (progress tracking in non-volatile memory). By replaying the manifest, but using the SUIT Report to guide the result of any conditions, and using the SUIT Report to skip any non-volatile state altering commands, we can perfectly reconstruct the state of the parser, prior to any resets or power failures, in-situ, which allows us to continue processing wherever the previous run left off.

I agree, but we do still need to figure out which commands should go into the report, and either explicitly choose them during manifest creation, or automatically infer them during manifest processing, or both.

Yes, I think this makes sense. Perhaps we should define the two overarching options: journaling and resilient manifest construction.

Sound good to me.
There's already a requirement in the Architecture document, but perhaps a Threat in the Threat Model would help.

Devices must not fail when

   a disruption, such as a power failure or network interruption, occurs

   during the update process.
https://datatracker.ietf.org/doc/html/rfc9019#section-3<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc9019%23section-3&data=05%7C01%7COyvind.Ronningstad%40nordicsemi.no%7Cb88835e0fdcf4427f24808da5f9f9b7f%7C28e5afa2bf6f419a8cf6b31c6e9e5e8d%7C0%7C0%7C637927436364714700%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=H4u4iTFyFAKqwHTcJ7FV5D2OsYLRk2cwRHKro0K8Umc%3D&reserved=0>

Thanks, I had a feeling I'd missed something about it.

Best regards, Øyvind