[Sml] Proposal: define a new Structured-Email header field

[Phillip Tao <[e-mail]>]

Hi all,

I would like to propose that a new header field be defined in the core draft. The reason is that MUAs may want to be able to make choices about how and when to download a given message, based on the presence of structured data inside the message.

Proposed format:

    Structured-Email: representation=machine-readable; content=vacation-notice;

The body of the header would be a tag-value list as defined by DKIM, with two tags: representation and content.

The "representation" tag can have four values: "machine-readable", "full", "partial", or "non".

These map to the types of representation defined in the core doc. In my reading of the core draft, machine-readable messages are strictly a subset of "full representation", while non-machine-readable messages may be any of the three representation types. Therefore, I've chosen to collapse "machine-readable" into one of the four representation types here, rather than express that as a separate tag. Note that "non" specifies the "non-representation" case, not the absence of structured data.

The "content" tag would be a comma-separated list of values identifying the type of structured content in the message. A new IANA registry would be created to register these types.

I would propose that the header with at least a representation tag be required, while the content tag would be optional.

FAQ

Q: Why do we need this in addition to the message flags defined in section 6 of the core draft?
A: Two main reasons:
    1. The IMAP flags rely on mailbox provider implementation. For a MUA designed to interoperate with a variety of MBPs, a reliable signal is needed for whether a message has structured content or not. Because the sender is the one attaching the structured data, they're the natural party to also attach metadata about the presence of the structured data.
    2. The currently defined flags do not convey the type of structured data carried in the message. This is important because the MUA may not understand all possible structured data that can be in an email message, and may only care about certain types. Even among the types that it does care about, the representation type and content type may inform when and how it wants to download the message.
    For example, for a machine-readable one-time-code, a MUA might want to download it immediately, even when it would otherwise hold off on downloading message bodies due to power or network usage concerns. It may also just download the structured data part and not any other MIME part. For a message with a boarding pass or ticket, it may want to download just that part for now, and download the rest of the body parts according to its normal body download scheduling.

Q: Should this replace the message flags defined in section 6?
A: I lean toward yes, but I can also see some potential use cases for the message flags (like, maybe being better supported for IMAP SEARCH)?

Q: Why does this need to be a header? Can the same information not be derived from the BODYSTRUCTURE?
A: Currently, structured data is defined as being carried exclusively in application/ld+json or application/jose parts. However, the presence of MIME parts of those types alone are not sufficient to determine that they are actually structured data, as opposed to some an attachment which happens to use the same format. An alternative could be to define a new attribute on the Content-Type or Content-Disposition.
    More importantly, however, there has been discussion of allowing some cases of structured data being allowed to be carried in the text/html part. In that case, it is impossible to tell a message with structured data apart from any other HTML message.

Q: Why do we need a new IANA registry?
A: The current IANA registry defined by the core draft is for vocabularies. This new registry would be for content types. Some content types may share vocabularies. One content type may also use multiple vocabularies. Many content types may not need any vocabulary defined in the IANA registry, and may simply use schema.org <http://schema.org/>.
    The broader question of why the content-type is needed at all is addressed in question 1.

Open to any feedback, questions, or comments.

- Phillip