[Cfrg] Trusting government certifications of cryptography

["D. J. Bernstein" <[e-mail]>]

Torsten Schuetze writes:
> Smart cards, i.e., its coprocessors and its ECC software
> implementations, are always certified by Common Criteria, usually with
> rather high assurance level. Don't you trust this certification in
> principle? 

http://smartfacts.cr.yp.to explains how we factored 184 RSA-1024 keys
from Taiwan's national "Citizen Digital Certificate" database. As you
can see from http://smartfacts.cr.yp.to/cert.html, these keys were
generated by government-issued smart cards that advertised all of the
usual Common Criteria and FIPS certifications from BSI, NIST, and CSE.

http://smartfacts.cr.yp.to/analysis.html analyzes five contributing
factors to this security disaster---low-quality hardware RNG, inadequate
sanity checks, no run-time sanity checks, no postprocessing, and
inadequate initial response---and the complete failure of certification
to prevent the disaster from occurring.

Another interesting example to consider is the FIPS certification for
branches of OpenSSL starting with openssl-fips-1.1.2.tar.gz (2007). The
pursuit of FIPS certification was the reason that OpenSSL added support
for Dual EC with NSA's P and Q. The certification procedure failed to
catch the fact that OpenSSL's Dual EC implementation didn't work at all:

   https://www.mail-archive.com/[e-mail]/msg00127.html

As Steve Marquess put it: "Frankly the FIPS 140-2 validation testing
isn't very useful for catching 'real world' problems."

After the public learned that Dual EC with this P and Q was an NSA back
door (see, e.g., https://projectbullrun.org/dual-ec), the slowness of
recertification stopped openssl-fips from dropping Dual EC support for
_nine months_:

   https://www.mail-archive.com/[e-mail]/msg75061.html

It's clear that OpenSSL's certification had various other effects, such
as diverting quite a bit of OpenSSL funding into InfoGard Laboratories
Inc. How much positive security impact did this have to outweigh its
obvious negative impacts? For example, how many of OpenSSL's security
holes, whether in the FIPS version or generally, were discovered through
the FIPS validation process?

Security review is of course important, and I've heard several stories
of security problems being caught by Common Criteria testing labs. But
the security problems that _aren't_ caught show that certification is
not true "assurance" of security---even in relatively simple settings
such as the Taiwanese Citizen Digital Certificates, never mind the much
more complex environment of Internet cryptography.

I understand that words such as "certification" and "validation" and
"assurance" have an important impact on users in some markets. But for
experts it should be clear that more skepticism is warranted. You can't
reasonably ask CFRG to "trust this certification in principle".

---Dan