Re: [Trans] Directory instead of .well-known for URL structure

[Rob Stradling <[e-mail]>]

Andrew, thanks for your analysis.  Comments inline...

On 01/07/2019 20:37, Andrew Ayer wrote:
> I also think .well-known URLs are a bad idea, for the same reasons
> as Jacob.
> 
> I think directories are even worse.  Although they've been used
> successfully in ACME, CT has different needs which make directories
> unsuitable.  ACME's usage pattern is to make a series of related
> requests over a short time period to drive a certificate request to
> issuance, with the client maintaining state between each request.
> This makes it natural for a client to fetch the directory once at the
> beginning of an issuance "session" and use it for the duration, since
> the cost of fetching the directory is amortized over all the requests,
> and the directory is unlikely to become invalid during the session.
> 
> However, CT's usage pattern is to make a lot of unrelated one-off
> requests spread over a long period of time - e.g. submitting a
> (pre)certificate, fetching an inclusion proof, fetching the latest STH
> to send to auditors. Clients would have to either fetch a new directory
> for every request (doubling the number of requests made to the log) or
> cache directories in long-term state (which requires dealing with
> cached directories going stale, and requires keeping long-term state
> which might not otherwise be necessary).
> 
> ACME's use of directories is underspecified since it doesn't say how
> long a directory remains valid.  It's not a big deal for ACME because
> ACME servers are presumed to be sane, or people would switch to another
> CA.

It's not been a big deal for ACME...yet.  But why leave it to chance?

I just filed this erratum against RFC8555:
https://www.rfc-editor.org/errata/eid5771

> However, CT is meant to be an adversarial protocol and has to
> anticipate logs doing crazy things like constantly changing their
> directory in an effort to stymie auditing and hide misbehavior.
> 
> Thus, CT's use of directories would need to be quite well specified.
> It is not a change that should be rushed at the last minute without a
> chance for people to carefully examine and poke holes in it.

Agreed.

> CT clients already need to be configured with a number of parameters
> for each log - MMD, hash algorithm, public key, log ID, and so on.
> Adding a directory would bifurcate log metadata between the existing
> parameter set and the new directory object.
> 
> I propose satisfying BCP 190 by simply specifying the URL for each
> endpoint as a separate log parameter.  This is a very minimal change to
> the protocol and avoids the problems above.

I agree that it would make sense to require each endpoint to be 
specified as a log parameter.  Your point about bifurcation is well made.

So then I think the question is: Should 6962-bis specify a "directory" 
endpoint that returns a JSON structure that contains all of the log's 
parameters (including each of the endpoint URLs)?  Or should 6962-bis 
not mandate any particular format by which log operators should 
distribute this information?

Currently, 6962-bis doesn't mandate a format, but it does point to an 
example format:
   "[JSON.Metadata] is an example of a metadata format which includes the
    above elements."

Sadly, this claim is not currently true, because [JSON.Metadata] 
(https://www.gstatic.com/ct/log_list/log_list_schema.json) does not 
currently "include the above elements".  We should fix this somehow. 
But how?

Also, how often is a CT client expected to refresh log metadata it has 
obtained from (for example) [JSON.Metadata] ?
Is it wise to underspecify this?

> Regards,
> Andrew

-- 
Rob Stradling
Senior Research & Development Scientist
Sectigo Limited