Re: [Trans] What's the load on a CT log?

On Thu, Mar 13, 2014 at 9:48 PM, Ben Laurie <benl@google.com> wrote:

> On 13 March 2014 20:27, Rob Stradling <rob.stradling@comodo.com> wrote:
> > I'm not sure average load tells the whole story.
>
> Agreed.
>
> > Won't there be a surge in audit traffic in the aftermath of a busy site
> > installing a new cert?
>
> Yes.
>
This is where separate calls may be beneficial - there's a higher chance
that a response to each of the individual calls will be cached rather than
a single call that is a response to a client request which contains an STH.

> >
> >
> > On 13/03/14 16:06, Ben Laurie wrote:
> >>
> >> Several people have asked me this recently. Here's a nice way to
> estimate
> >> load.
> >>
> >> Let's assume a single log that takes all the load.
> >>
> >> Firstly, we see about 5,000 new certificates a day, so that's around
> >> 0.06 new certificates per second. Clearly a trivial load.
> >>
> >> Next is load from audit (i.e. from browsers that wish to validate SCTs
> >> accompanying certificates they see). Given some assumptions, we can
> >> calculate the load from audit.
> >>
> >> * Clients cache audit results.
> >>
> >> * There are approximately b = 2.5B browsers in the world
> >> (http://www.internetworldstats.com/stats.htm)
> >>
> >> * The average user visits w = 89 websites a month
> >> (http://www.creditloan.com/blog/how-the-world-spends-its-time-online/
> >> quoting a Nielsen report). Assume these are all TLS sites.
> >>
> >> * Assume a certificate lifetime of l = 12 months.
> >>
> >> So, each user sees w / l new certificates a month. Each new
> >> certificate needs to be audited, which means in practice, three web
> >> operations (fetch STH, fetch STH consistency proof, fetch SCT
> >> inclusion proof) - it might be a good idea to create a new API to do
> >> all three in one go.
> >>
> >> So, total average load is 3 * b * w / l ~ 20,000 web fetches per
> >> second. If we optimise the API we can get that down to 7,000 qps. Each
> >> query (in the optimised case) would be around 3 kB, which gives a
> >> bandwidth of around 150 kb/s.
> >>
> >> Monitors add extra load, but should only be at around the new
> >> certificate rate - i.e. ~ .06 * number of monitors fetches per second.
> >>
> >> IMO, this is achievable on a single machine (modulo reliability), with
> >> some care. Clearly not a vast farm, however its done.
> >>
> >> In practice, no one log would have to take this full load, this is a
> >> worst case analysis.
> >>
> >> _______________________________________________
> >> Trans mailing list
> >> Trans@ietf.org
> >> https://www.ietf.org/mailman/listinfo/trans
> >>
> >
> > --
> > Rob Stradling
> > Senior Research & Development Scientist
> > COMODO - Creating Trust Online
> > Office Tel: +44.(0)1274.730505
> > Office Fax: +44.(0)1274.730909
> > www.comodo.com
> >
> > COMODO CA Limited, Registered in England No. 04058690
> > Registered Office:
> >   3rd Floor, 26 Office Village, Exchange Quay,
> >   Trafford Road, Salford, Manchester M5 3EQ
> >
> > This e-mail and any files transmitted with it are confidential and
> intended
> > solely for the use of the individual or entity to whom they are
> addressed.
> > If you have received this email in error please notify the sender by
> > replying to the e-mail containing this attachment. Replies to this email
> may
> > be monitored by COMODO for operational or business reasons. Whilst every
> > endeavour is taken to ensure that e-mails are free from viruses, no
> > liability can be accepted and the recipient is requested to use their own
> > virus checking software.
>
> --
> You received this message because you are subscribed to the Google Groups
> "certificate-transparency" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to certificate-transparency+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>