Re: [Trans] Directory instead of .well-known for URL structure

[Eran Messeri <[e-mail]>]

I support Andrew Ayer and Rob Stradling's suggestion to " require each
endpoint to be specified as a log parameter."

The reasons outlined make a lot of sense from an implementation
perspective, our experience with RFC6962 suggest that absent caching
mechanisms, logs would face significant load if all clients were to audit
them. Adding a requirement to fetch a directory would increase the load for
(as far as I can tell) no good reason.

I think under-specifying it right now is the only option as we have no
specification of the log metadata. I don't think it's too big of a deal as
when people start implementing 6962-bis I expect the log metadata format
will evolve based on the existing schemas and, if necessary, could be
standardized.


On Wed, Jul 3, 2019 at 2:52 PM Rob Stradling <[e-mail]> wrote:

> Thanks Rob.  Both the to-be-deprecated and new schemas are of course
> focused on CT v1 (RFC6962), so ISTM that neither of them are suitable
> candidates for 6962-bis to point at.
>
> Maybe we should simply remove the "[JSON.Metadata] is an example of a
> metadata format which includes the above elements" sentence from 6962-bis.
>
> On 02/07/2019 18:51, Rob Percival wrote:
> > FYI: That JSON schema is planned for deprecation; a new version was
> > proposed last year and, following a period for feedback, parties have
> > been slowly migrating to the new schema:
> >
> https://groups.google.com/a/chromium.org/d/topic/ct-policy/26dmpWdaSHM/discussion
> >
> > That thread would be a good place to propose schema changes.
> >
> > On Tue, 2 Jul 2019, 16:08 Rob Stradling, <[e-mail]
> > <mailto:[e-mail]>> wrote:
> >
> >     Andrew, thanks for your analysis.  Comments inline...
> >
> >     On 01/07/2019 20:37, Andrew Ayer wrote:
> >      > I also think .well-known URLs are a bad idea, for the same reasons
> >      > as Jacob.
> >      >
> >      > I think directories are even worse.  Although they've been used
> >      > successfully in ACME, CT has different needs which make
> directories
> >      > unsuitable.  ACME's usage pattern is to make a series of related
> >      > requests over a short time period to drive a certificate request
> to
> >      > issuance, with the client maintaining state between each request.
> >      > This makes it natural for a client to fetch the directory once at
> the
> >      > beginning of an issuance "session" and use it for the duration,
> since
> >      > the cost of fetching the directory is amortized over all the
> >     requests,
> >      > and the directory is unlikely to become invalid during the
> session.
> >      >
> >      > However, CT's usage pattern is to make a lot of unrelated one-off
> >      > requests spread over a long period of time - e.g. submitting a
> >      > (pre)certificate, fetching an inclusion proof, fetching the
> >     latest STH
> >      > to send to auditors. Clients would have to either fetch a new
> >     directory
> >      > for every request (doubling the number of requests made to the
> >     log) or
> >      > cache directories in long-term state (which requires dealing with
> >      > cached directories going stale, and requires keeping long-term
> state
> >      > which might not otherwise be necessary).
> >      >
> >      > ACME's use of directories is underspecified since it doesn't say
> how
> >      > long a directory remains valid.  It's not a big deal for ACME
> because
> >      > ACME servers are presumed to be sane, or people would switch to
> >     another
> >      > CA.
> >
> >     It's not been a big deal for ACME...yet.  But why leave it to chance?
> >
> >     I just filed this erratum against RFC8555:
> >     https://www.rfc-editor.org/errata/eid5771
> >
> >      > However, CT is meant to be an adversarial protocol and has to
> >      > anticipate logs doing crazy things like constantly changing their
> >      > directory in an effort to stymie auditing and hide misbehavior.
> >      >
> >      > Thus, CT's use of directories would need to be quite well
> specified.
> >      > It is not a change that should be rushed at the last minute
> without a
> >      > chance for people to carefully examine and poke holes in it.
> >
> >     Agreed.
> >
> >      > CT clients already need to be configured with a number of
> parameters
> >      > for each log - MMD, hash algorithm, public key, log ID, and so on.
> >      > Adding a directory would bifurcate log metadata between the
> existing
> >      > parameter set and the new directory object.
> >      >
> >      > I propose satisfying BCP 190 by simply specifying the URL for each
> >      > endpoint as a separate log parameter.  This is a very minimal
> >     change to
> >      > the protocol and avoids the problems above.
> >
> >     I agree that it would make sense to require each endpoint to be
> >     specified as a log parameter.  Your point about bifurcation is well
> >     made.
> >
> >     So then I think the question is: Should 6962-bis specify a
> "directory"
> >     endpoint that returns a JSON structure that contains all of the log's
> >     parameters (including each of the endpoint URLs)?  Or should 6962-bis
> >     not mandate any particular format by which log operators should
> >     distribute this information?
> >
> >     Currently, 6962-bis doesn't mandate a format, but it does point to an
> >     example format:
> >         "[JSON.Metadata] is an example of a metadata format which
> >     includes the
> >          above elements."
> >
> >     Sadly, this claim is not currently true, because [JSON.Metadata]
> >     (https://www.gstatic.com/ct/log_list/log_list_schema.json) does not
> >     currently "include the above elements".  We should fix this somehow.
> >     But how?
> >
> >     Also, how often is a CT client expected to refresh log metadata it
> has
> >     obtained from (for example) [JSON.Metadata] ?
> >     Is it wise to underspecify this?
> >
> >      > Regards,
> >      > Andrew
> >
> >     --
> >     Rob Stradling
> >     Senior Research & Development Scientist
> >     Sectigo Limited
> >
> >     _______________________________________________
> >     Trans mailing list
> >     [e-mail] <mailto:[e-mail]>
> >     https://www.ietf.org/mailman/listinfo/trans
> >
>
> --
> Rob Stradling
> Senior Research & Development Scientist
> Email: [e-mail]
> Bradford, UK
> Office: [phone]
> Sectigo Limited
>
> This message and any files associated with it may contain legally
> privileged, confidential, or proprietary information. If you are not the
> intended recipient, you are not permitted to use, copy, or forward it,
> in whole or in part without the express consent of the sender. Please
> notify the sender by reply email, disregard the foregoing messages, and
> delete it immediately.
> _______________________________________________
> Trans mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/trans
>