Re: [sidr] Slides for "RPKI Over BitTorrent" presentation

Eric Osterweil <> Sun, 01 April 2012 19:44 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 42EB121F8A25 for <>; Sun, 1 Apr 2012 12:44:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.999
X-Spam-Status: No, score=-3.999 tagged_above=-999 required=5 tests=[BAYES_50=0.001, RCVD_IN_DNSWL_MED=-4]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id ljpVA2SoXXyQ for <>; Sun, 1 Apr 2012 12:44:35 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id C957421F8A23 for <>; Sun, 1 Apr 2012 12:44:06 -0700 (PDT)
Received: from ([]) (using TLSv1) by ([]) with SMTP ID DSNKT3iwBop/; Sun, 01 Apr 2012 12:44:07 PDT
Received: from ( []) by (8.13.6/8.13.4) with ESMTP id q31Ji5Wo000843; Sun, 1 Apr 2012 15:44:05 -0400
Received: from ([]) by with Microsoft SMTPSVC(6.0.3790.4675); Sun, 1 Apr 2012 15:44:03 -0400
Mime-Version: 1.0 (Apple Message framework v1084)
Content-Type: text/plain; charset="us-ascii"
From: Eric Osterweil <>
In-Reply-To: <>
Date: Sun, 01 Apr 2012 12:44:01 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <>
To: Rob Austein <>
X-Mailer: Apple Mail (2.1084)
X-OriginalArrivalTime: 01 Apr 2012 19:44:04.0825 (UTC) FILETIME=[CB96BC90:01CD103F]
Subject: Re: [sidr] Slides for "RPKI Over BitTorrent" presentation
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Secure Interdomain Routing <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 01 Apr 2012 19:44:36 -0000

Hey Rob,

One more meta-comment on this thread: we (as designers) should not expect to be prescribing exactly how operators will run their systems.  Here in the very early stages we are seeing the early adopters deploy in ways that make _their_ operations easy.  imho, this is always going to be the bottom line/starting point.  If a solution requires a specific operational model (when others are technically correct), and the overall system scalability can suffer this way, I think we have a fundamental problem.  

More succinctly, operators shouldn't need to know why you want their certs in a hierarchy.  Rather, we should recognize that they are telling us that flat structures make their lives easier, and we should use that to reengineer what _they_ need.

Just my 0.02,


On Mar 30, 2012, at 11:10 AM, Rob Austein wrote:

> While I will admit to some astonishment that the following explanation
> could possibly be news to long-time participants in this WG (given how
> much time I've spent whining about this issue over the last five years
> or so both in public and in private), let me quote from the slides:
>    * How efficient [fetching RPKI repositories using rsync] is
>      depends heavily on how the publication repositories are
>      organized.
>    * In an efficiently organized repository, filesystem hierarchy
>      follows X.509 certificate hierarchy, so that one can pick up
>      significant subtrees with a single rsync connection.
>    * To date, the RIRs have chosen to deploy flat hierarchies where
>      there is no relationship at all between filesystem hierarchy
>      within the repository and certificate hierarchy.
> To make that more concrete, here's an example.  Let's assume we have
> the following trivial hierarchy: Bob and Betty are issued by Alice,
> Carol and Carl are issued by Bob, Dave, and Dana are issued by Carol,
> Dara is issued by Carl, and and all of these are hosted in a single
> repository.
> In an inefficient, "flat" repository, the publication points for
> objects issued by these entities would look something like this:
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
> In a hierarchical repository, the same publication points would look
> more like this:
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
>    rsync://
> Assuming top-down tree walk (the normal case), retrieving objects
> issued by this set of entities takes eight rsync connections with the
> flat repository, as opposed to one rsync connection with the
> hierarchical repository.
> In practice one might want a slightly more complex structure to limit
> the size of individual directories, but it doesn't matter so long as
> the filesystem hierarchy is organized in such a way that picking up
> an issuer's publication point picks up a non-trivial number of its
> subjects' publication points automatically.   It doesn't have to be
> perfect, just has to do enough better than the flat model to amortize
> the cost of setting up and tearing down the rsync connection over a
> significantly larger number of files.
> This is not about PKI, it's purely an rsync efficiency issue.
> Presumably there are scaling limitations to the hierarchical approach,
> but anecdotal evidence among the people I've asked ("I tried ... and
> it worked") suggests that, if the underlying networks and filesystems
> are in good shape, a single rsync connection ought to be able to
> handle up to at least 10,000 small files, perhaps a lot more than
> that.  Note that this is just talking about rsync itself: mileage
> might vary significantly if the underlying networks or filesystems are
> seriously broken.  Also note that these anecdotal estimates have not
> been tested in any rigorous fashion as far as I know, so that's
> another entry on my list of things we ought to be measuring.
> Hope this helps to clarify the change I've been suggesting.
> _______________________________________________
> sidr mailing list