[Suit] Internal handover of updates inside a device

[Rønningstad, Øyvind <[e-mail]>]

We have been exploring how updates would be passed between implementations ("parties") inside a system, and how the manifest format can help with that. Below is a manifest format together with a short description of the update procedure and some restrictions on the format not captured by the CDDL.

At the bottom are a number of user stories as well as how the format and restrictions fulfil the user stories.

Highlights:

  *   We propose moving metadata about the preprocessing of payloads and manifests out of the parent manifest. This is to facilitate handing responsibilities between parties, and it has the added benefits of making the format simpler, and not tying it to COSE.
  *   After thinking quite a bit about it, we think we found a satisfactory way to support multiple payloads. Each payload will have its own timestamp and set of conditions to be met before they can be installed.
  *   Aliases has been promoted to resourceLocations, which holds all URIs. Dependencies and payloads contain only the hash, used to look up the URI in the list. This simplifies the incorporation of aliases.
  *   Conditions have been divided into PreConditions (all of which are checked at the very beginning) and PostConditions (all of which are checked at the very end), because the start and finish of the update are well-defined points in time, that are critical for the robustness of the update.

Thanks for the great work on the documents, we tried to adapt, but also wanted to send this in time for the hackathon. Do you plan to release a serialization format as well, or to keep it out of scope? Anyway, hopefully this is helpful.

Unfortunately we cannot participate in the hackathon in person, but we will try to join online as much as possible.

Best regards,
Øyvind and Frank


Manifest format (CDDL) [alt5]

TimeStamp           = uint .size 8
Uuid                = bstr .size 16
StorageIdentifier   = bstr
ComponentIdentifier = bstr
UriList             = { + int => tstr }
UriList             = { + int => tstr }

DigestInfo = [
    size:            uint,
    digestAlgorithm: uint,
    digest:          bstr
]

IdCondition     = [vendor: 1 / class: 2 / device: 3,         id:         Uuid]
TimeCondition   = [installAfter: 4 / bestBefore: 5,          time:       Timestamp]
ImageCondition  = [currentContent: 6 / notCurrentContent: 7, digest:     DigestInfo / nil, location: StorageIdentifier]
CustomCondition = [nint,                                     parameters: bstr]

PreCondition  = IdCondition / TimeCondition / ImageCondition / CustomCondition
PostCondition = ImageCondition / CustomCondition

ResourceLocation = [
    resourceLocationVersion: 1,
    digestInfo: DigestInfo,
    encoding:   bstr,
    location:   UriList
]

PayloadInfo = [
    subject:    StorageIdentifier,
    timestamp:  Timestamp,
    digest:     bstr
]

ManifestInfo = [
    component:  ComponentIdentifier,
    digest:     bstr
]

Manifest = [
    manifestVersion:   1,
    resourceLocations: [ * ResourceLocation ],
    preConditions:     [ * PreCondition ],
    dependencies:      [ * ManifestInfo ],
    payloads:          [ * PayloadInfo ],
    postConditions:    [ * PostCondition ],
    text:              { * int => tstr } / nil,
    extensions:        { * int => bstr } / nil
]


The update procedure

All manifests must be authenticated (signed), e.g. by being wrapped in a COSE_Sign or COSE_Mac. The format does not mandate COSE-specific types.

A received manifest or payload might be encoded in multiple steps like signature, encryption, compression etc. Signing, encryption, compression and other encodings are implementation-dependent, but when encrypting and/or signing a manifest, it must happen in the following order (any step can be skipped) during creation:

  1.  Compress/diff/other encoding
  2.  Encrypt
  3.  Sign
which then happen in reverse during parsing:

  1.  Check signature
  2.  Decrypt
  3.  Decompress/patch/other decoding

Operations like signing, encryption, compression etc. can be applied to both manifests and payloads. Any encoding (e.g. delta patching) can be supported either through ResourceInfo::encoding or through a custom wrapper type (analogous to the COSE types) for the payload/manifest.

It is recommended that the update process involves multiple passes of the manifest, e.g.:

  1.  For each manifest, validate the serialization and check the validity of StorageIdentifiers and ComponentIdentifiers. Check timestamps and preConditions. This can hopefully be accomplished without writing to flash storage.
  2.  Retrieve payloads and validate them. This does not involve irreversible change.
  3.  Install payloads
  4.  Check postConditions.

I.e. strive to validate as much as possible before any flash writes happen, and as much as possible before irreversible change happens.

If any check fails, or if any procedure cannot be completed, the whole update must be aborted, and changes reverted if possible.


Restrictions and Clarifications about the types

PreCondition:

  *   preConditions are always checked against the state before the entire update started.

PostCondition:

  *   All postConditions lists included in the update must pass before the update is complete.
  *   All postConditions lists included in the update are checked against the state after all manifests are installed.

Manifest

  *   Dependencies must be installed in the order they appear in the dependencies list.
  *   Payloads must be installed in the order they appear in the payloads list.
  *   All dependencies must be installed before any payloads are installed
  *   A dependency is installed when the referenced Manifest is completely installed. I.e. the dependency tree is traversed depth-first.
  *   A Manifest is installed when all its payloads are installed.

ResourceLocation

  *   If a single list of ResourceLocations contains two entries with the same digestInfo::digest, their entire digestInfo and encoding entries must also be identical. This ensures that digests referred to in PayloadInfo and ManifestInfo are unambiguous.
  *   The parser can use ResourceLocation::encoding to understand the encoding of the referenced manifest or payload.
  *   The format of ResourceLocation::encoding is implementation-dependent.
  *   When a resource is successfully received or retrieved, its digest must be checked.

PayloadInfo

  *   The PayloadInfo::digest can be used to look up an entry in the resourceLocations list by matching it with ResourceLocation::digestInfo.digest.
  *   The value in PayloadInfo::timestamp must be at least one greater than the stored timestamp for this StorageIdentifier. This must be checked immediately before the payload is installed.
  *   After the PayloadInfo has been installed, the stored timestamp for this StorageIdentifier is overwritten with the value in PayloadInfo::timestamp. This must happen before the next payload's timestamp is checked.

StorageIdentifier

  *   The format of StorageIdentifier is implementation-dependent, but must give enough information to unambiguously point to a certain string of bytes, including its size, inside the image. Multiple StorageIdentifiers can point to the same byte string, or overlapping byte strings, with no restrictions. I.e.:
     *   Imagine an update was stored with StorageIdentifier "app/5/1/0/store_metadata". Note that StorageIdentifiers don't have to be text, and don't have to look like paths. In an ImageCondition, the StorageIdentifier "app", "app/5", "app/5/1", "app/5/1/0", or "app/5/1/0/load_metadata" might all resolve to the same content received in the update. ANY aliasing scheme is allowed.
     *   Updates to different StorageIdentifiers can cause incrementation of the same stored timestamp. Any mapping from StorageIdentifier to stored timestamp is allowed.

IdCondition

  *   A device's vendorID, classID, and deviceID should be static within the same ComponentIdentifier.

TimeCondition

  *   These conditions require a secure clock providing a current time.
  *   These conditions are unrelated to PayloadInfo::timestamp.
  *   installAfter fails if the current time is less than the required time.
  *   bestBefore fails if the current time is more than the required time.

ImageCondition

  *   The condition refers to the state of the image at the time it is checked.
  *   ImageCondition::digest represents the hash of the expected content.
  *   DigestInfo::size represents the expected length of the content.
  *   If ImageCondition::digest is nil only the existence of the location is checked, not the content.
  *   The total size of the byte string pointed to by the StorageIdentifier must equal the content length reported in the ImageCondition. To check only a part of the byte string content, define a new StorageIdentifier that points to the relevant part.
  *   notCurrentContent is the inverse of currentContent, i.e.:
     *   A notCurrentContent condition will pass for any state currentContent would fail for.
     *   A notCurrentContent condition will fail for any state currentContent would pass for.

ComponentIdentifier

  *   The format of ComponentIdentifier is implementation-dependent but is intended to refer to components that the current party does not have direct write access to. "party" here means an implementation of the standard.

ManifestInfo

  *   The ManifestInfo::digest can be used to look up an entry in the resourceLocations list by matching it with ResourceInfo::digestInfo.digest.
  *   If the manifest referred to by the ComponentIdentifier will be installed by a different party, the handover will involve some or all of the following:
     *   The transmitting party transfers the digest and encoding from the relevant ManifestInfo, together with an instruction such as "Validate", "Install", or "Abort". Which instructions are available will be implementation-dependent, but "Install" or something to that effect is needed.
     *   The receiving party must immediately start carrying out the instruction and report back any success or failure. The receiving party needs to retrieve or receive the manifest referenced by the digest and encoding it received, as well as any dependency manifests or payloads. This can happen through any channel, e.g. the transmitting part retrieves it on behalf of the receiving party.
     *   The transmitting party transfers a list of ResourceLocations, either preemptively or when asked. The receiving party may:
        *   Use the list to find the URI associated with the digest received in the first point.
        *   Use the list to supplement the lists in any manifest(s) it resolves.
        *   Pass the list on to other parties.

Appendix: User Stories


  1.  I want to use a different signing wrapper.
  2.  I want to use a different encryption wrapper.
  3.  I want to use a custom wrapper for encoding.
  4.  I want the signing, encryption, and decoding metadata to be parsed by another party than parses the rest of the manifest.
  5.  I want to prevalidate as much as I can before writing anything to flash, to prevent unnecessary flash wear.
  6.  I want to prevalidate as much as I can before doing irreversible change.
  7.  I want to be able to easily concatenate lists of aliases to have only a single lookup table for digest -> URI.
  8.  I want to explicitly validate the post-update state e.g. support fallback or rollback.
  9.  I want to keep multiple parallel monotonic counters to support the device containing multiple independent firmwares.
  10. I want to be able to check for existence of a firmware without checking the contents.
  11. I want to be able to check that something has been successfully removed/changed.
  12. I want to be able to merge dependencies if they are signed by the same entity. To reduce complexity and space used for manifests.
  13. I want to independently identify and reference external components (components managed by other parties).
  14. I want to do easily do rudimentary version/compatibility checks more complex than the monotonic counter.


Appendix: Resolutions of user stories


  1.  Resolved by moving signing, encryption and encoding out of scope of the specification. It is assumed that the user will implement it as nested wrappers like COSE_Sign<COSE_Encrypt<Payload_blob>>.
  2.  Resolved as for 1.
  3.  Resolved as for 1.
  4.  Resolved as for 1.
  5.  Resolved by the fact that PreConditions are always checked against the state before the update started.
  6.  Resolved as for 5.
  7.  Resolved by Manifest::resourceLocations being the only list containing URIs that are looked up by digest.
  8.  Resolved by PostConditions and the fact that they are all applied to the state after all payloads.
  9.  Resolved by the PayloadInfo::timestamp being tied to the StorageIdentifier.
  10. Resolved by ImageCondition::digest being nil.
  11. Resolved by ImageCondition::notCurrentContent.
  12. Resolved by having a list of payloads instead of a single payload.
  13. Resolved by the ComponentIdentifier.
  14. Resolved by aliasing of StorageIdentifiers.