IETF Logo Mail Archive

Re: [Trans] What's the load on a CT log?

Ben Laurie <benl@google.com> Thu, 13 March 2014 16:51 UTC

Return-Path: <benl@google.com>
X-Original-To: trans@ietfa.amsl.com
Delivered-To: trans@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 97EF21A0A27 for <trans@ietfa.amsl.com>; Thu, 13 Mar 2014 09:51:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.926
X-Spam-Level:
X-Spam-Status: No, score=-1.926 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n02oiC5MHvLU for <trans@ietfa.amsl.com>; Thu, 13 Mar 2014 09:51:27 -0700 (PDT)
Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 173EE1A0484 for <trans@ietf.org>; Thu, 13 Mar 2014 09:51:26 -0700 (PDT)
Received: by mail-vc0-f175.google.com with SMTP id lh14so1420460vcb.34 for <trans@ietf.org>; Thu, 13 Mar 2014 09:51:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wdrhN3U2CPr0BzTR++RCrRQOxYvYlpymZ6NY2IivIMo=; b=FZgPHWRmchsDn0mj7izclBSqMDbAflcjY8KxBQbcTeQA4yQ4X2jDJCXPPV6qRotQVq 8m5VUzK+qNrmoOy5P3AIGwX/qoqLXRas/RH4o7iySofS03+605cKcyYIWtLKlHSSt/VT lcHCTfjdjF7crMVOu797Pz8X+5ImPNHAxkFLJ7wQjQWE5gevi520W7r7QIprBbekR7S7 JK6iUgeHVdGQ/PuT0ELgak8mC4aGCRsbGwwxg1ibW2rZ3NECjYChW0jLgH86Yr1Xf+oq mPgTb0ZKnpIGTXvFlRL26CuNrPluV/HG4pYwzMbob/yXXz9Lv45n9ZnkcSFfpzOnSUdU GJww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=wdrhN3U2CPr0BzTR++RCrRQOxYvYlpymZ6NY2IivIMo=; b=g0u5wcJCyXdE39O0CccFug0a52KgfVKc/am/erZFN81uKBZqRWWxRiLNUc6DDlHQ9i m0lzj/LEIfZ0XwVNtg02T5DMU5Q9c9HGMKJUBVcC7oy2/vyNb/LVYXn6lXT7nh3m8P9V nha7JOBs4Gj5gffzyl+u/uwBbQDI9CP5NY081WTrdKLCyR+NnKSX3FkjeEMfkEKooCoM vWNHfaASnhh6m7H0GkjgsxNMFmYirC5279ia2HkMto4iVTsR5l/JcB7gsld4r8h7f1pe Sch7P5GyREeziAMNrPnR8rGXZh2fczdLm9su1PmPPeXQJO0jfj7rfHhsnW0YaA99vZoW Albw==
X-Gm-Message-State: ALoCoQnVc5HHxfvpXoA3KKJLbEdL1LsaVNmRRSxTcLl0x7WT8KjV3Bw6llrE7Rt35Jms+k7GsrReVNuAsUrdjju4w9ITcxPqJddMhZpZxfzvqyLWlNGzF5p/hunD9Rd0w5vaGm1u5aRzxQis0a8G8mJfDE7gJD8bHzXIX/Nsnvrfj2nnLxpq6hkLtVU/7ohNVTCPvZlk9Jca
MIME-Version: 1.0
X-Received: by 10.220.159.4 with SMTP id h4mr2267836vcx.1.1394729480350; Thu, 13 Mar 2014 09:51:20 -0700 (PDT)
Received: by 10.52.230.105 with HTTP; Thu, 13 Mar 2014 09:51:20 -0700 (PDT)
In-Reply-To: <5321DD69.2040805@fifthhorseman.net>
References: <CABrd9SR4G6hEUEW9yHLyS40Km3+jmK8K-tEjLMjLqN1M+Go_=g@mail.gmail.com> <5321DD69.2040805@fifthhorseman.net>
Date: Thu, 13 Mar 2014 16:51:20 +0000
Message-ID: <CABrd9SR9tczbHx4XrHfg3JJjmq2QtmaAEkFpn0r92gfEpm+2RA@mail.gmail.com>
From: Ben Laurie <benl@google.com>
To: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Content-Type: text/plain; charset="UTF-8"
Archived-At: http://mailarchive.ietf.org/arch/msg/trans/a3lluA3-3EJDBe26RTtu2UwVTG4
Cc: "therightkey@ietf.org" <therightkey@ietf.org>, "trans@ietf.org" <trans@ietf.org>, "certificate-transparency@googlegroups.com" <certificate-transparency@googlegroups.com>, CABFPub <public@cabforum.org>
Subject: Re: [Trans] What's the load on a CT log?
X-BeenThere: trans@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Public Notary Transparency working group discussion list <trans.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/trans>, <mailto:trans-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/trans/>
List-Post: <mailto:trans@ietf.org>
List-Help: <mailto:trans-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/trans>, <mailto:trans-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Mar 2014 16:51:28 -0000

On 13 March 2014 16:31, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> On 03/13/2014 12:06 PM, Ben Laurie wrote:
>> So, total average load is 3 * b * w / l ~ 20,000 web fetches per
>> second.
>
> This part i follow (you're switching temporal units between months and
> years and seconds, but i get roughly the same final figures)
>
>> If we optimise the API we can get that down to 7,000 qps. Each
>> query (in the optimised case) would be around 3 kB,
>
> And i agree this seems like a win.  Why was the API broken into three
> parts instead of the complete proof originally?  what (other than
> conceptual cleanliness) might we lose by creating the optimized API?
>
>> which gives a bandwidth of around 150 kb/s.
>
> This looks off by a few orders of magnitude to me.  7kqps and 3kB/q
> gives me 7000*3000*8 bits per second, which is 168Mbps.  Am i missing
> something?

Sorry, you are correct - I meant 150,000 kb/s!

> Should we be considering swarm-based distribution of this kind of data,
> or hierarchical proxying for load distribution?

Maybe :-)

One thing we're working on is distributing the proofs via DNS, which
is obviously of exactly that nature - which would definitely reduce
bandwidth at the servers.

v2.23.0 | Report a Bug | By Email