[openpgp] Ambiguity about signature vs literal packet encoding formats

[Andrew Gallagher <[e-mail]>]

Hi, all.

It has come to my attention that it is possible (and apparently permitted!) to generate a text (0x01) signature over a binary literal data packet. The test suite does not currently cover this scenario well, seemingly because of lack of support for the required SOP commands:

https://tests.sequoia-pgp.org/#Signed_messages

https://gitlab.com/sequoia-pgp/openpgp-interoperability-test-suite/-/issues/17
https://gitlab.com/sequoia-pgp/openpgp-interoperability-test-suite/-/issues/63
https://gitlab.com/sequoia-pgp/openpgp-interoperability-test-suite/-/issues/73

But it would now appear that this is a real interop issue:

https://github.com/rpgp/rpgp/pull/263

tl;dr: A text document can be encoded using non-normalised line endings in a binary literal data packet, and have a text signature over it that is correctly calculated over a normalised (but ephemeral) copy of that document. If this signature is detached, and then verified over the detached document, it is obvious that the document must be normalised before verification, and the verification should therefore succeed. But it does not appear to be obvious that the same is true of that document before detachment, since at least two implementations do not currently perform this normalisation.

Does the group feel this a case of the receiving implementations not being sufficiently lenient in what they consume, or the generating implementations not being sufficiently strict in what they produce? ;-)

Crypto-refresh appears to be contradictory. It allows generating applications to put text data in a binary-marked literal data packet, even if the application knows for sure it will be text-signed:

> If the implementation is certain that the data is textual and is encoded with UTF-8 (for example, if it will follow this literal data packet with a signature packet of type 0x01 (see Section 5.2.1), it MAY set the format octet to u. Otherwise, it MUST set the format octet to b.

But it tells a receiving application that `b` means binary data, without exception:

> The body of this packet consists of:
> • A one-octet field that describes how the data is formatted.
> If it is a b (0x62), then the Literal packet contains binary data.

https://datatracker.ietf.org/doc/html/draft-ietf-openpgp-crypto-refresh-13#name-literal-data-packet-type-id

If this field does not contain authoritative information about the the literal data packet encoding, would it not make more sense to default it to NUL, or some other meaningless value, rather than a potentially misleading one? Otherwise, surely the literal data packet encoding MUST match the signature type? Alternatively a receiving application MUST ignore this field and infer it from the signature type when verifying that signature.

A

PS: Before DKG panics, this is aimed at the future semantics clarification document, which appears more and more necessary as time passes :-)