Re: [sidr] Slides for "RPKI Over BitTorrent" presentation

Rob Austein <sra@hactrn.net> Fri, 30 March 2012 18:10 UTC

Return-Path: <sra@hactrn.net>
X-Original-To: sidr@ietfa.amsl.com
Delivered-To: sidr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3612821F8625 for <sidr@ietfa.amsl.com>; Fri, 30 Mar 2012 11:10:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.415
X-Spam-Level:
X-Spam-Status: No, score=-102.415 tagged_above=-999 required=5 tests=[AWL=0.184, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MwHft2+Q7Mqn for <sidr@ietfa.amsl.com>; Fri, 30 Mar 2012 11:10:22 -0700 (PDT)
Received: from cyteen.hactrn.net (cyteen.hactrn.net [66.92.66.68]) by ietfa.amsl.com (Postfix) with ESMTP id 2928321F8622 for <sidr@ietf.org>; Fri, 30 Mar 2012 11:10:21 -0700 (PDT)
Received: from minas-ithil.hactrn.net (ter75-1-81-57-68-77.fbx.proxad.net [81.57.68.77]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "nargothrond.hactrn.net", Issuer "Grunchweather Associates" (verified OK)) by cyteen.hactrn.net (Postfix) with ESMTPS id 1874428465 for <sidr@ietf.org>; Fri, 30 Mar 2012 18:10:19 +0000 (UTC)
Received: from minas-ithil.hactrn.net (localhost [127.0.0.1]) by minas-ithil.hactrn.net (Postfix) with ESMTP id A9BE46EFC01 for <sidr@ietf.org>; Fri, 30 Mar 2012 20:10:17 +0200 (CEST)
Date: Fri, 30 Mar 2012 20:10:17 +0200
From: Rob Austein <sra@hactrn.net>
To: sidr@ietf.org
In-Reply-To: <4F75B916.6000807@gmail.com>
References: <20120326173305.D1CB96E71B3@minas-ithil.hactrn.net> <A76204B3-5EF3-4212-9136-B8E3266C7D17@tcb.net> <20120329100434.BBE5A6EE29F@minas-ithil.hactrn.net> <1DC0A868-0B15-448E-93E6-C61CC89AB6A6@lacnic.net> <4F75B916.6000807@gmail.com>
User-Agent: Wanderlust/2.15.5 (Almost Unreal) Emacs/22.3 Mule/5.0 (SAKAKI)
MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka")
Content-Type: text/plain; charset=US-ASCII
Message-Id: <20120330181017.A9BE46EFC01@minas-ithil.hactrn.net>
Subject: Re: [sidr] Slides for "RPKI Over BitTorrent" presentation
X-BeenThere: sidr@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Secure Interdomain Routing <sidr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sidr>, <mailto:sidr-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sidr>
List-Post: <mailto:sidr@ietf.org>
List-Help: <mailto:sidr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sidr>, <mailto:sidr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 30 Mar 2012 18:10:23 -0000

While I will admit to some astonishment that the following explanation
could possibly be news to long-time participants in this WG (given how
much time I've spent whining about this issue over the last five years
or so both in public and in private), let me quote from the slides:

    * How efficient [fetching RPKI repositories using rsync] is
      depends heavily on how the publication repositories are
      organized.

    * In an efficiently organized repository, filesystem hierarchy
      follows X.509 certificate hierarchy, so that one can pick up
      significant subtrees with a single rsync connection.

    * To date, the RIRs have chosen to deploy flat hierarchies where
      there is no relationship at all between filesystem hierarchy
      within the repository and certificate hierarchy.

To make that more concrete, here's an example.  Let's assume we have
the following trivial hierarchy: Bob and Betty are issued by Alice,
Carol and Carl are issued by Bob, Dave, and Dana are issued by Carol,
Dara is issued by Carl, and and all of these are hosted in a single
repository.

In an inefficient, "flat" repository, the publication points for
objects issued by these entities would look something like this:

    rsync://example.org/rpki/Alice/
    rsync://example.org/rpki/Betty/
    rsync://example.org/rpki/Bob/
    rsync://example.org/rpki/Carl/
    rsync://example.org/rpki/Carol/
    rsync://example.org/rpki/Dana/
    rsync://example.org/rpki/Dara/
    rsync://example.org/rpki/Dave/

In a hierarchical repository, the same publication points would look
more like this:

    rsync://example.org/rpki/Alice/
    rsync://example.org/rpki/Alice/Betty/
    rsync://example.org/rpki/Alice/Bob/
    rsync://example.org/rpki/Alice/Bob/Carl/
    rsync://example.org/rpki/Alice/Bob/Carl/Dara/
    rsync://example.org/rpki/Alice/Bob/Carol/
    rsync://example.org/rpki/Alice/Bob/Carol/Dana/
    rsync://example.org/rpki/Alice/Bob/Carol/Dave/

Assuming top-down tree walk (the normal case), retrieving objects
issued by this set of entities takes eight rsync connections with the
flat repository, as opposed to one rsync connection with the
hierarchical repository.

In practice one might want a slightly more complex structure to limit
the size of individual directories, but it doesn't matter so long as
the filesystem hierarchy is organized in such a way that picking up
an issuer's publication point picks up a non-trivial number of its
subjects' publication points automatically.   It doesn't have to be
perfect, just has to do enough better than the flat model to amortize
the cost of setting up and tearing down the rsync connection over a
significantly larger number of files.

This is not about PKI, it's purely an rsync efficiency issue.

Presumably there are scaling limitations to the hierarchical approach,
but anecdotal evidence among the people I've asked ("I tried ... and
it worked") suggests that, if the underlying networks and filesystems
are in good shape, a single rsync connection ought to be able to
handle up to at least 10,000 small files, perhaps a lot more than
that.  Note that this is just talking about rsync itself: mileage
might vary significantly if the underlying networks or filesystems are
seriously broken.  Also note that these anecdotal estimates have not
been tested in any rigorous fashion as far as I know, so that's
another entry on my list of things we ought to be measuring.

Hope this helps to clarify the change I've been suggesting.