Re: [websec] draft-ietf-websec-key-pinning

[Joseph Bonneau <[e-mail]>]

On Wed, Aug 27, 2014 at 9:14 PM, Ryan Sleevi <[e-mail]> wrote:

> Right, so, I do agree with Joe that I do think we've reached conclusions
> on the suitability for security and the suitability for testing, and I do
> want to see what can be done to editorially resolve this.
>
> Without having fully drafted the text, it seems like one possible solution
> is to describe HOW a site operator might use this to test deployment of
> pins, knowing that it may not be a perfect solution for all use cases, it
> would at least help to clarify both the strengths and limitations.
>
> The example I'm thinking of is as follows:
>

With Ryan's proposed testing plan I fear many site operators can't easily
enumerate all subdomains and even if they try this is a particularly
difficult test mode because there's no feedback if you forget a subdomain.
This allows you to make sure it works on all subdomains you know about-but
what about the ones you don't?

Beyond that though, we're probably kidding ourselves if we think most site
operators will actually sit down and read an RFC front-to-back. They won't.
Some evidence from a project a student of mine is working on, finding a
large number of sites (~30%) are not properly setting HSTS headers per the
spec (which is comparably simple to achieve):
http://jbonneau.com/doc/KB14-hsts_pinning_survey_working_draft.pdf. Clearly
a lot of people didn't read the RFC and that's not even counting minor
errors like incorrect capitalization.

I think we should be realistic that this RFC will primarily be read by
developers building support into user agents and not site operators. Site
operators will, at best, read somebody's 500-word how-to blog post. Many
will probably just copy what some other site is doing and modify it until
it appears to work.

In a perfect world, we'd do "developer testing" on an draft like this where
we give a random site-operator-off-the-street the RFC and ask if they can
comprehend it and add support to their site. This is similar to how any new
software tool should have user testing. Short of that, the evidence we have
is that multiple people (including myself) who are security enthusiasts and
have been following this draft for years had a broken mental model of how
PKP-RO works. I think this design is violating the Principal of Least
Astonishment and we're asking for trouble no matter how much explanatory
text gets added to the RFC.