Re: [saag] Algorithms/modes requested by users/customers

["Santosh Chokhani" <[e-mail]>]

On the issue of it takes too long, the FIPS process is the envy of other
schemes.  One could always be faster.

On the issue of restart, that is misunderstanding on Paul's part.  There
is no restart.  Generally, problem is fixed and you pick up that point.
This is generally a vendor problem.  Depending on the nature of
problem/failure, other aspects of the testing can continue without loss
of calendar time.

Subjective tests seems to be misunderstanding on Paul's part.  Having
been in the Orange Book, CC and FIPS process, I as validator, tester,
and certifier as well as vendor generally rue about rigidness of the
standard and lack of subjective judgment on the part of the tester or
lack of subjective latitude to vendors.

FIPS 140 has specific guidelines on how to deal with minor incremental
changes that helps reduce cost and calendar time.

The standard is NOT a protocol standard.  It does verify the algorithm
implementations and thus, FIPS validated algorithms can interoperate.

In terms of low end devices, 20-30K for level 1 testing amortized over
devices does not seem too onerous.

I do not understand what Paul refers to as silly modes.  The module
being FIPS validated must use FIPS validated or recognized algorithms
for a given crypto service.  That seems like a good thing to me.



-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of
Paul Hoffman
Sent: Tuesday, February 19, 2008 11:18 AM
To: Stephen Kent
Cc: [e-mail]; Randall Atkinson
Subject: Re: [saag] Algorithms/modes requested by users/customers

At 10:15 AM -0500 2/19/08, Stephen Kent wrote:
>Can you share the reasons cited by vendors to support the notion 
>that the FIPS 140 process is broken.

Sure.

- It takes way too long from submission of system to validation, even 
if no problems are found.

- If problems are found during evaluation, the restart time is too long.

- Some of the tests are fairly subjective, and it becomes a game of 
fixing code to please the testing service, not to make the product 
more secure.

- An evaluated product cannot have its core firmware updated without 
losing the validation. For example, if a customer asks for a new 
feature that might touch the crypto, the vendor has to choose between 
losing the validation (and paying for a new one, and waiting for the 
results) or keeping the customer happy.

- The validation doesn't even check for on-the-wire interoperability, 
which is what the customers care about most.

- The test process is too expensive for many low-end devices that 
would be very useful in USGovt offices.

- The system introduces silly modes that make the systems more
complicated.

These are the top complaints I hear from both large and small 
vendors. There are certainly others. "Oh, don't get me started" is 
commonly heard when discussing FIPS 140 testing.

>   Compared to other security evaluation criteria, e.g. CC or the old 
>Organge Book, most security folks I know view the FIPS 140 
>evaluation program as well managed and not very onerous.

"Security folks" are not good evaluators on how some processes affect 
the market.

>Also, if buyers believe that a device that "could be evaluated" is 
>good enough, they are being rather naive.

Maybe. If a buyer cares about "does this box support the needed 
crypto in an interoperable fashion", then the perception is fine. If 
they care about all the details that FIPS 140 testing covers, then of 
course it is naive.

>Experience with the FIPS 140 program has shown that a significant 
>fraction of of products submitted for evaluation fail the process.

This statement is usually bandied about without any quantification of 
what failures were found, as it is here. As someone who creates test 
suites, I can assure you that I can make a few picky rules for 
systems that would cause some of those systems to fail, but most end 
users who understood those few rules would say that those rules were 
silly. Of course, different end users have different requirements. It 
is generally felt by vendors that some/many of the rules for FIPS 140 
are silly and unneeded; on the other hand, it is likely that at least 
a few USGovt customers would really want some of those rules for 
particular reasons.

>Presumably a vendor submitting a product for evaluation believes 
>that the product is ready, and will pass, otherwise the vendor is 
>wasting money on the evaluation effort.

Wrong, very wrong. You cannot tell what the testing agency might try 
to knock you down on ahead of time. Of course, you cover what you 
can, but you also know that there is a good chance they'll just whack 
you a bit because it makes them look good.

>The fact that many products fail evaluation (for good security 
>reasons), tells me that a vendor's claim that a product "could be 
>evaluated" is not much of an assurance.

If you believe in the testing regime, that's fine. Others don't, and 
I believe for very good reasons in their own use scenarios.

--Paul Hoffman, Director
--VPN Consortium
_______________________________________________
saag mailing list
[e-mail]
http://mailman.mit.edu/mailman/listinfo/saag