[openpgp] Re: pure vs. pre-hash in FIPS 204 and 205

[Justus Winter <[e-mail]>]

Hi Falko :)

Falko Strenzke <[e-mail]> writes:

> I attribute this at least in part to a misunderstanding about the
> variants “pure” vs. “pre-hash” in RFC 8032 (EdDSA) as well as FIPS 204
> and 205. The idea is to use the pre-hash variant whenever the message
> is first hashed and then input to the signature algorithm. As I had
> mentioned on this list as well, this is done – formally – wrong in RFC
> 9580, which specifies to use the pure variant of EdDSA to sign the
> hash of the message.

I continue to be puzzled by this assertion, but I want to make another
effort to understand it.

RFC8032 defines two variants: PureEdDSA and HashEdDSA.  They differ in
the prehash parameter:

   11.  A "prehash" function PH.  PureEdDSA means EdDSA where PH is the
        identity function, i.e., PH(M) = M.  HashEdDSA means EdDSA where
        PH generates a short output, no matter how long the message is;
        for example, PH(M) = SHA-512(M).

I'm convinced that OpenPGP uses the PureEdDSA variant, i.e.:

  sig = PureEdDSA(Hash(OpenPGP_Hash_Stream))
                  ^ This happens in OpenPGP

The evidence I have gathered for that is:

- Sequoia uses https://docs.rs/nettle/latest/nettle/ed25519/fn.sign.html
- This primitive is tested against the test vectors from
  https://datatracker.ietf.org/doc/html/rfc8032#ref-ED25519-TEST-VECTORS
- These test vectors are part of the test vectors for "Ed25519"
  https://datatracker.ietf.org/doc/html/rfc8032#section-7.1
- Ed25519 is EdDSA instantiated with: [...]
  |   PH(x)   | x (i.e., the identity function)                       |
  https://datatracker.ietf.org/doc/html/rfc8032#section-5.1
-> "Ed25519" is PureEdDSA
-> Sequoia uses PureEdDSA
- Sequoia's v6 implementation agrees with all the other v6
  implementations when using Ed25519-based binding signatures and data
  signatures
  https://tests.sequoia-pgp.org/v6.html#Detached_Sign-Verify_roundtrip_with_minimal_key_from_RFC9580
-> OpenPGP as specified in RFC9580 uses PureEdDSA

On the other hand, if OpenPGP were to use the HashEdDSA variant, we'd
compute:

  sig = PureEdDSA(PH(Hash(OpenPGP_Hash_Stream)))
                     ^ This happens in OpenPGP
                  ^ This is the prehash in HashEdDSA

Which seems not ideal.

Concluding, I'm pretty sure that OpenPGP uses PureEdDSA.  Since there is
obviously some confusion, maybe it is not about what we are doing but
what we are calling it.

- Are you saying since arguably OpenPGP does pre-hashing, the overall
  construction looks more like HashEdDSA, and therefore we should have
  called it that?

- There is no "pre-hash" variant defined in RFC8032, so maybe that is a
  FIPS term and the confusion arises because of that?


Best,
Justus