Re: [Trans] What's the load on a CT log?

Eran Messeri <eranm@google.com> Fri, 14 March 2014 10:02 UTC

Return-Path: <eranm@google.com>
X-Original-To: trans@ietfa.amsl.com
Delivered-To: trans@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8DA91A0102 for <trans@ietfa.amsl.com>; Fri, 14 Mar 2014 03:02:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.925
X-Spam-Level:
X-Spam-Status: No, score=-1.925 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=unavailable
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bez_0jnfwfe7 for <trans@ietfa.amsl.com>; Fri, 14 Mar 2014 03:02:50 -0700 (PDT)
Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) by ietfa.amsl.com (Postfix) with ESMTP id BEEDC1A00EE for <trans@ietf.org>; Fri, 14 Mar 2014 03:02:49 -0700 (PDT)
Received: by mail-ob0-f178.google.com with SMTP id wp18so2303681obc.37 for <trans@ietf.org>; Fri, 14 Mar 2014 03:02:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZBMK/k3DMgeB355e7/rxW/u1c5Y3o1z9nConl1DwoGI=; b=PrvWHG+enFQaSsWPxYd4OsG7uE3nxun7iNpPPVSy0kslmZ9YULRycZxcoZ6g28nS2m jDuKycLLDxam8oxYv9zron96pCOIEJcCzBuJxjmvdw+krtzMPVym6Lg7uJwZcqaCpLs5 bqsp8Fe7h12ZQ3eN2iewtjfqwCZgYkk9chL21MM5FB7SYeB2bAU605Xrs1aB9osG0yCo efWc41bMDElLhY6H/Sm4OJ9tqxBC5egRVgtnOqW9Higi4yU7l9j6LdaeEudfW1Kc0dwT mCamN69uwYuMBfnlPCg7S2O4MMFDPn/saTsKczR/l7kNOPFWoodUINc83gVrlBTpsMGG 59tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=ZBMK/k3DMgeB355e7/rxW/u1c5Y3o1z9nConl1DwoGI=; b=js4GqolU+/c/EWBkdGUfZDK+ir3PI0YuefYjBux/cuqi7L5Li/kBowksW1UCNogwKi Kzr0MZZLE0WsB6yL1MD59fBU55RN5g6wTagZfsJzULSv8I7522LVX0Uu6TeI5MMXSBDy 6OKkwZfX6I2gWLJ3IyKh5fBRVqYA/HflvCZwR5fAYCQYp5UZQJSDFLtb2HfAZbqjNJ4l 838PZQWKd1gCQKwLsrIBi0KPAQICBKavALd+Z1wquCvvDiiV/3x+plAxy4y42PfRMPah LVnlLcD2OOZciAhZ8tiApygBY1bFN9NvFwQ8Yx2P2QdkyMsBlGRtNlSA1us6V45py9/b aoeQ==
X-Gm-Message-State: ALoCoQlsmmfrHedF82cnRH7gnCRAZ7LywBkynuh6Gz0TigT1pfm9m2Pm7P/GB6vMSEPYXSLvGRYheZKm5RsPzAJiMB3m5WNJhNIdiT1Fw1mFXw1P5u9D+xC3+SyHYeiQxiXwsUPqR46rz+n01BllsWWxQCDZqj1rJBgoPWktODsYAZKn+9Hakkno88WnKvvbFlLLvOTt39wj
MIME-Version: 1.0
X-Received: by 10.60.119.73 with SMTP id ks9mr195341oeb.75.1394791362875; Fri, 14 Mar 2014 03:02:42 -0700 (PDT)
Received: by 10.182.142.198 with HTTP; Fri, 14 Mar 2014 03:02:42 -0700 (PDT)
In-Reply-To: <CABrd9SSdZZRGr2Q6CoHsquNM-TOFSJehjEACPXzEdK=h7=CpAg@mail.gmail.com>
References: <CABrd9SR4G6hEUEW9yHLyS40Km3+jmK8K-tEjLMjLqN1M+Go_=g@mail.gmail.com> <53221499.40301@comodo.com> <CABrd9SSdZZRGr2Q6CoHsquNM-TOFSJehjEACPXzEdK=h7=CpAg@mail.gmail.com>
Date: Fri, 14 Mar 2014 10:02:42 +0000
Message-ID: <CALzYgEfCvgQRQ2ur74kXui=vEXyqqhBo=jnn=YiiLXenNvzyNA@mail.gmail.com>
From: Eran Messeri <eranm@google.com>
To: certificate-transparency@googlegroups.com
Content-Type: multipart/alternative; boundary=047d7b33cddcb86d3e04f48e2bcc
Archived-At: http://mailarchive.ietf.org/arch/msg/trans/hHKGzz-qWdvsoX9F0vTsAokQ-SA
Cc: "therightkey@ietf.org" <therightkey@ietf.org>, Rob Stradling <rob.stradling@comodo.com>, "trans@ietf.org" <trans@ietf.org>, CABFPub <public@cabforum.org>
Subject: Re: [Trans] What's the load on a CT log?
X-BeenThere: trans@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Public Notary Transparency working group discussion list <trans.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/trans>, <mailto:trans-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/trans/>
List-Post: <mailto:trans@ietf.org>
List-Help: <mailto:trans-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/trans>, <mailto:trans-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Mar 2014 10:02:52 -0000

On Thu, Mar 13, 2014 at 9:48 PM, Ben Laurie <benl@google.com> wrote:

> On 13 March 2014 20:27, Rob Stradling <rob.stradling@comodo.com> wrote:
> > I'm not sure average load tells the whole story.
>
> Agreed.
>
> > Won't there be a surge in audit traffic in the aftermath of a busy site
> > installing a new cert?
>
> Yes.
>
This is where separate calls may be beneficial - there's a higher chance
that a response to each of the individual calls will be cached rather than
a single call that is a response to a client request which contains an STH.


> >
> >
> > On 13/03/14 16:06, Ben Laurie wrote:
> >>
> >> Several people have asked me this recently. Here's a nice way to
> estimate
> >> load.
> >>
> >> Let's assume a single log that takes all the load.
> >>
> >> Firstly, we see about 5,000 new certificates a day, so that's around
> >> 0.06 new certificates per second. Clearly a trivial load.
> >>
> >> Next is load from audit (i.e. from browsers that wish to validate SCTs
> >> accompanying certificates they see). Given some assumptions, we can
> >> calculate the load from audit.
> >>
> >> * Clients cache audit results.
> >>
> >> * There are approximately b = 2.5B browsers in the world
> >> (http://www.internetworldstats.com/stats.htm).
> >>
> >> * The average user visits w = 89 websites a month
> >> (http://www.creditloan.com/blog/how-the-world-spends-its-time-online/
> >> quoting a Nielsen report). Assume these are all TLS sites.
> >>
> >> * Assume a certificate lifetime of l = 12 months.
> >>
> >> So, each user sees w / l new certificates a month. Each new
> >> certificate needs to be audited, which means in practice, three web
> >> operations (fetch STH, fetch STH consistency proof, fetch SCT
> >> inclusion proof) - it might be a good idea to create a new API to do
> >> all three in one go.
> >>
> >> So, total average load is 3 * b * w / l ~ 20,000 web fetches per
> >> second. If we optimise the API we can get that down to 7,000 qps. Each
> >> query (in the optimised case) would be around 3 kB, which gives a
> >> bandwidth of around 150 kb/s.
> >>
> >> Monitors add extra load, but should only be at around the new
> >> certificate rate - i.e. ~ .06 * number of monitors fetches per second.
> >>
> >> IMO, this is achievable on a single machine (modulo reliability), with
> >> some care. Clearly not a vast farm, however its done.
> >>
> >> In practice, no one log would have to take this full load, this is a
> >> worst case analysis.
> >>
> >> _______________________________________________
> >> Trans mailing list
> >> Trans@ietf.org
> >> https://www.ietf.org/mailman/listinfo/trans
> >>
> >
> > --
> > Rob Stradling
> > Senior Research & Development Scientist
> > COMODO - Creating Trust Online
> > Office Tel: +44.(0)1274.730505
> > Office Fax: +44.(0)1274.730909
> > www.comodo.com
> >
> > COMODO CA Limited, Registered in England No. 04058690
> > Registered Office:
> >   3rd Floor, 26 Office Village, Exchange Quay,
> >   Trafford Road, Salford, Manchester M5 3EQ
> >
> > This e-mail and any files transmitted with it are confidential and
> intended
> > solely for the use of the individual or entity to whom they are
> addressed.
> > If you have received this email in error please notify the sender by
> > replying to the e-mail containing this attachment. Replies to this email
> may
> > be monitored by COMODO for operational or business reasons. Whilst every
> > endeavour is taken to ensure that e-mails are free from viruses, no
> > liability can be accepted and the recipient is requested to use their own
> > virus checking software.
>
> --
> You received this message because you are subscribed to the Google Groups
> "certificate-transparency" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to certificate-transparency+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>