[CFRG] Re: Where should test vectors live?

[Phillip Hallam-Baker <[e-mail]>]

Surprised nobody mentioned the NIST AVCP project here.

They have test vectors and code. Wow do they have a lot of test vectors.
https://github.com/usnistgov/ACVP-Server/tree/master

They also have a pile of Internet Drafts:
https://pages.nist.gov/ACVP/draft-celi-acvp-symmetric.html

More information:
https://www.nccoe.nist.gov/automation-nist-cryptographic-module-validation-program


I converted my back end to use their JSON file formats for my test system
so I am slightly biased here. But that only took me a day and I could have
done that faster if I was only looking to do ML-KEM and ML-DSA. And anyone
who is getting their module validated has to use these formats.

There is definitely value in progressing at minimum a draft describing the
meta format to standard. And we should probably do the lot.

The big caveat here is that right now, there are so many test vector sets,
it is difficult to know which one is the right one to use. Some
algorithms are easy, SHA2, SHA3, etc. Others are pretty difficult. I still
haven't found a test vector set for ECDH key exchange with P256/P384/P521
for use in JOSE/COSE. There are a dozen that look relevant but none seems
to be the one.

A small caveat is that while I don't accept the legitimacy of groups like
JOSE and COSE saying they MUST support the algorithm of choice when they
were meeting, that is an application choice, it is certainly legitimate for
a group to say MUST NOT. And so we probably want to look into testing to
see that MUST NOT is managed as we would want. We definitely want to
continue to erase SHA-1, etc.


We might want to extend the approach somewhat to go beyond algorithms to
protocol test sets. But it seems capable enough.