Re: [sidr] sidrSlides for "RPKI Over BitTorrent" presentation

Rob Austein <> Thu, 24 May 2012 22:14 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 1906311E80A3 for <>; Thu, 24 May 2012 15:14:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -102.599
X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id kXiYapnbWqiB for <>; Thu, 24 May 2012 15:14:13 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 50AF311E8081 for <>; Thu, 24 May 2012 15:14:13 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "", Issuer "Grunchweather Associates" (verified OK)) by (Postfix) with ESMTPS id 31B302847F for <>; Thu, 24 May 2012 22:14:10 +0000 (UTC)
Received: from (localhost [IPv6:::1]) by (Postfix) with ESMTP id 0020017244 for <>; Thu, 24 May 2012 18:14:09 -0400 (EDT)
Date: Thu, 24 May 2012 18:14:09 -0400
From: Rob Austein <>
In-Reply-To: <>
References: <> <> <> <>
User-Agent: Wanderlust/2.14.0 (Africa) Emacs/23.4 Mule/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka")
Content-Type: text/plain; charset="US-ASCII"
Message-Id: <>
Subject: Re: [sidr] sidrSlides for "RPKI Over BitTorrent" presentation
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Secure Interdomain Routing <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 24 May 2012 22:14:14 -0000

At Thu, 24 May 2012 17:11:55 -0400, Christopher Morrow wrote:
> On Thu, May 24, 2012 at 4:13 PM, John G. Scudder <> wrote:
> > Wes,
> >
> > On May 23, 2012, at 9:22 PM, Wes Hardaker wrote:
> >
> >> Bittorrent works well for sharing the load, but either requires a lot of
> >> bittorrent start files (whatever they're called), which then becomes
> >> hard to syncronize; or a small number of bittorrent files (one per
> >> aggregation point) that indicate you need to refetch and entire .zip
> >> file even if you already have most of it anyway.

The .zip file is just a wrapper around a .torrent file and an
associated text manifest (which is mostly present to provide a
stronger checksum than BitTorrent provides).

.zip files over HTTPS are the control channel.  RPKI data files over
BitTorrent are the data channel.  Confusing the two will lead to brain

> > I'm far from an expert on Bittorrent but I believe this is not right. A
> > .torrent file (what you called a "bittorrent start file") can contain
> > contain a list of files to be transferred, there's no mandate to zip them
> > into a single file first. Presumably the BT protocol is smart about not
> > transferring files you already have locally -- that's kind of the whole
> > point. Rob's "operational overview" slide makes it clear he's imagining
> > operating in the collection-of-files mode.
> per publication-point I think?

No.  It's one torrent per gatherer.  More precisely, one torrent at
time per gatherer, as each gatherer generates a new torrent when it
thinks it has a new batch of data that's worth sharing.

> So you have to collect N-thousand of these torrent files, and keep
> N-thousand torrent jobs running and keep state on N-thousand remote
> torrent files changing over time... some of this could be alleviated
> in many ways, worst case though is that.

One torrent per gatherer at a time.  The mechanism outlined in the
slides uses HTTPS polling to distribute the .zip files containing the
.torrent files.  The HTTPS service used for this can be replicated via
the usual techniques.

So the clients poll the HTTPS servers to get the .torrent files, then
use the .torrent files to retrieve the ten zillion little object files
from the BitTorrent cloud.  Note that BitTorrent is clever enough not
to retrieve files it already has, so in effect this produces a similar
effect to rsync (client only pulls what it needs), without imposing
the same kind of server load.