Re: [openpgp] test vectors for unknown signatures [was: Re: Implementers: does your OpenPGP tool gracefully discard signature packets of unknown version?]

[Daniel Kahn Gillmor <[e-mail]>]

Hi Peter, all--

On Sun 2022-05-01 10:53:03 +0000, Peter Gutmann wrote:
> Daniel Kahn Gillmor <[e-mail]> writes:
>
>>To raise the profile of this specific issue and facilitate testing, i've
>>extracted the artifacts for the relevant tests into their own repository
>
> A couple of issues with this...
>
> * The 4+23 signature is reported by pgpdump as being another 4+4, not a 4+23,
> so either pgpdump is getting confused or it's not actually a 4+23.

thanks for catching this.  I also ran into it, and pushed fixes to git,
but I apologize for not posting about it earlier.  I think i had a
copy/paste error when pulling from the interop test suite.  I've made a
few updates to this corpus and i've pushed them to git (at
https://gitlab.com/openpgp-wg/unknown-signature-versions)  If anyone
has trouble retreiving them from git, feel free to contact me (off-list
is fine), and i'll push another tarball to the list.

> * The version number in the tests should really be 5 or at most 6, not 23.
> When performing version checks my code allows future versions one or two ahead
> of the current one, but a jump of nineteen versions implies corrupted data,
> not a time-warp leap of around 200 years worth of standardisation (currently
> about ten years per rev).  So I'd say code is quite justified in rejecting
> what looks like a gibberish version number.

I'm not convinced of this argument, but if this is the consensus of the
WG, i'm willing to go along with it.  What do other folks think?

My understanding is that OpenPGP's packet grammar is well-defined.
Encountering a new version 20 versions in advance should be no more
confusing than encountering a version 2 versions beyond what is known.

If any subsequent packet isn't a packet type that makes sense in the
context of the grammar that the implementation knows about, yes, i can
see rejecting the packet stream.  But a version that isn't understood
seems identical to an algorithm that isn't understood.  Hopefully those
aren't rejected either, but rather silently ignored, same as you would
if you didn't have access to the key for a known-version,
known-algorithm signature packet.  An implementation that deliberately
fails solely because of an unknown version seems to be inviting interop
failures.

As for the "corrupted data" interpretation: when encountering a
signature, if a particular signature is corrupt, it should not validate.
That doesn't mean that the stream itself should be rejected.

> * The test vectors should include a file with just a single version n+1
> (currently 5) signature to see what happens when there are no sigs present
> that can be processed.

We can produce those as well, of course, but it would be great to ensure
now that all the major implementations can safely skip over
unknown-version signatures.  If that's not the case, we'll have to
repeat this dance any time a new signature version is defined.

In https://gitlab.com/openpgp-wg/unknown-signature-versions i've added a
single sig23.sig detached signature on its own, which implementations
are not expected to validate.

> * Because of the way OpenPGP formats messages, it's unclear what an
> implementation should do when it hits a version it can't process because
> there's no way to tell whether any more data that it can process will appear
> later.

I don't think this is correct.  The packet headers clearly delimit the
size of the packets.  packets that indicate they contain unknown data
can be skipped cleanly.  There hasn't been new framing in 20 years, and
implementations that take care to use the same stable grammar ought to
be able to rely on other implementations handling that grammar.

> Consider a streaming implementation that sees a header, the payload,
> and a signature with a version it can't process.  Because of OpenPGP's
> concatenation-of-packets format rather than encapsulation format it can't tell
> whether there's more signatures to follow that it can process,

I don't follow this argument.  Since RFC 2440 in 1998, we start with
one-pass signature (OPS) packets or full preceding signatures.  It's
clear that OPS packets and preceding signatures all come *before* the
message, and the trailing signatures correspond to the OPS signature
packets.

   https://datatracker.ietf.org/doc/html/rfc2440#section-10.2

If one OPS signature packet or one signature packet is unknown or
uninterpretable, that shouldn't have any effect on the other signature
packets in the stream.

> so what should it do with the one signature it's seen?  And if the
> answer is "buffer it in the hope that more signatures will turn up",
> how many should it buffer while waiting for one it can process?  10?
> 20?  100?  1000?  Or should it report the first signature as a
> signature-failed and hope more turn up?  I'm not actually sure what my
> code should do in this situation, so perhaps the new spec could make
> some comments on how to handle this.

I agree that the spec is currently silent about this, with the
implication that the packet stream could be infinite in length.  That
said, the same is true for signatures that *can* be processed.  (should
you keep on reading 10000000 signatures hoping you'll find a literal
data packet somewhere?)

There are other forms of infinite regress that the specification
implies, such as encryption-in-encryption-…, not to mention the
compression layer.

The interop test suite contains a short test for recursion depth using
compression:

   https://tests.sequoia-pgp.org/#Maximum_recursion_depth

And it implies that 32 layers deep is probably too deep, though that's
not in the specification at all.

If anyone wants to suggest text about reasonable limits for inclusion in
the specification, i'd be happy to consider it.  But i think that's
orthogonal to the question about what to do with unknown versions.

> And based on all this a followup question:
>
> * How badly do we really need a new incompatible-format signature?

A new version of signature is not an incompatible format as written.  It
is deliberately compatible with the packet framing and versioning
invariants established well over two decades ago.

What we're trying to figure out in this discussion is whether we need to
treat a new version as practically incompatible, the way that TLS has
done for TLS's failed version negotiation mechanism in its handshake.
(see "legacy_version" and the "supported versions" extension in
https://datatracker.ietf.org/doc/html/rfc8446#section-4.2.1 for that
sordid history).

It would be nice (and the protocol would be cleaner) if we can get
consensus among most implementers that unknown versions should be safely
skipped, the way that other unknown signatures are also skipped.

    --dkg