Re: [rrg] procedural aggregation Wed, 05 March 2014 09:05 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 113FE1A0255 for <>; Wed, 5 Mar 2014 01:05:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: 0.254
X-Spam-Status: No, score=0.254 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.547, SPF_PASS=-0.001] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id BYXzVM3trrS2 for <>; Wed, 5 Mar 2014 01:05:03 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id 03D3E1A0177 for <>; Wed, 5 Mar 2014 01:05:03 -0800 (PST)
Received: from ( []) by (Outbound Mail Relay) with ESMTP id 5D57D701E9B94; Wed, 5 Mar 2014 04:04:59 -0500 (EST)
Received: from ( []) by (OMAG/Core Interface) with ESMTP id 1D6A238000083; Wed, 5 Mar 2014 04:04:59 -0500 (EST)
References: <>
In-Reply-To: <>
X-MB-Message-Source: WebUI
MIME-Version: 1.0
X-MB-Message-Type: User
Content-Type: multipart/alternative; boundary=""
X-Mailer: AOL Webmail 38430-STANDARD
Received: from by ( with HTTP (WebMailUI); Wed, 05 Mar 2014 04:04:58 -0500
Message-Id: <>
X-Originating-IP: []
Date: Wed, 5 Mar 2014 04:04:58 -0500 (EST)
x-aol-global-disposition: G
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20121107; t=1394010299; bh=+xn8KYdkBs6RdrDAybvUvZQAasVb9+RVVfrJJexbHA8=; h=From:To:Subject:Message-Id:Date:MIME-Version:Content-Type; b=bz3eul+eRdx+vMTJusuyRFnw7JwThCoHeKuihxUqeI68Bds1x1xh1fh0gKfejsjjE FyvnU86BxXvWNT84JwB7AdCQjqx/kyTFxeSOplrwKAv7KIjrWro/+3Y4QNg0qB+W9f 6g+WhT+VWupTw9le+bk+9cJCWHKIWNbKyDjakx3Q=
x-aol-sid: 3039ac1afeb05316e8bb2264
Subject: Re: [rrg] procedural aggregation
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IRTF Routing Research Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 05 Mar 2014 09:05:08 -0000

But, let me quote you wrt to a much more important issue.
You once wrote: 
Problem Statement:
IPv6 comes with too much baggage. All we really needed was IPv4 with a larger address space.

IMHO: IPv6 won't get any boost by the IPv4 address expiration.
But shouldn't that be the most urgent RRG problem ?


-----Urspr√ľngliche Mitteilung----- 
Von: William Herrin <>
An: RRG <>
Verschickt: Mi, 5 Mrz 2014 1:12 am
Betreff: [rrg] procedural aggregation

Hi Folks,

Given recent comments, I thought it'd be an opportune time to introduce a concept I've been fiddling with for a couple years which I tentatively call "procedural aggregation."

I have discovered no new math which would overcome the technical challenges to aggregation identified here in the past. I come at the problem from a different angle instead.

My basic premise is that with current techniques (not necessarily the exact protocols) there is no practical upper limit to the number of routes which can be introduced to the Internet provided that each recipient of a route advertisement is sufficiently financially compensated. He must be paid enough for carrying each route to allow him to purchase and operate hardware that carries all of the routes for which he's being paid. With that in place, total routes will self-limit based on what technology is capable of at each price point.

>From there I develop administrative procedures for determining who announces what aggregates and how to pay who for carriage of the more-specifics in order to yield a technically useful network configuration.

I'll post a couple examples in the next few days. Before that, I want to take the remainder of this message to review how the Internet currently routes packets. I'll largely skip the technical part. You're all experts or you wouldn't be here. Instead I will clarify the financial process. Somebody pays each and every organization on the Internet to accept your packets and forward them to the next hop. Every single Autonomous System. Every single packet. I'll explain who and I'll explain how that money flows with the technical parts of packet routing.

Understanding the money flow will be important when I ask you to consider procedural aggregation. Exterior gateway protocols need not produce *all* technically possible networks. They need only (must only) produce those for which the participants are properly compensated.

__ Transit and Peering __

>From a financial point of view there are two types of Internet connections between two organizations: transit and peering. 

transit - Internet provider A provides access to "the entire Internet" to organization B. B may be an end user or another service provider.

hosting - Internet provider A connects servers under the control of organization B to "the entire Internet." Hosting is transit plus floorspace.

peering - Internet provider A provides Internet provider Z with access to only its downstream transit customers and vice versa, not to the entire Internet. 

settlement free peering - peering for which no no money changes hands.

For the purposes of this discussion:

Transit and hosting are the same thing. In each case, someone pays someone else to connect them to "the Internet." I will only use the term transit. 

Usually B pays A for Internet access. Sometimes it's indirect: buy coffee get "free" Internet. Sometimes A's compensation is non-monetary, e.g. good will from a charitable donation. Other times A is paid by a third party to service B. For the purposes of the discussion, transit service is always paid for: B always pays A for access to the Internet. 

For simplicity's sake I will treat all peering as settlement free peering, even though many service providers opt to behave as robber barons. 

__ Routes flow with the money __

We have two types of connections between Autonomous Systems: transit and peering.

Let's say Endpoint 1 (E1) and endpoint 2 (E2) are both connected to the Internet. They pay their ISPs who pay ISPs who peer with each other. What do those connections look like? Typically:

E1 -> B -> A - Z <- X <- E2

* E1 is a transit customer of ISP B. He pays ISP B to connect him to the entire Internet. So, B is paid to send packets from and receive packets for E1.

* B is a small ISP. He collects the payments from all of his customers like E1, keeps some of it and spends some of it on transit service from a larger ISP A to connect all of his customers to the Internet. A is expected to send packets from and receive packets for all of B's customers, including E1.

* E2's relationship with X is the same as E1's relationship with B. X's relationship with Z is the same as B's relationship with A. The former buys transit service from the latter.

* A buys transit service from several other large ISPs (not shown) but he also has a peering relationship with Z. E2 has paid Z (indirectly via X) to connect him to the Internet, so Z advertises to A that he's willing to accept packets addressed to E2. That's what X is paying Z for.

So, a packet travels from endpoint 1 (E1) to endpoint 2 (E2):

E1 -> B -> A -> Z -> X -> E2

Follow the money:

* E1 sends a packet destined for E2 over to B because it's E1's packet and he wants it sent. B accepts it because E1 paid B to accept it.

* B sends it to A because E1 paid B to send that packet. A accepts it from B because B paid A to accept it.

* A sends it to Z because B paid A to send that packet. Z accepts it from A because X paid Z to _receive_ packets for E2.

See the change there at the peering link? Now the money is flowing from the other endpoint.

* Z sends the packet to X because X paid Z to receive packets for E2. X accepts it because E2 paid X to accept it.

*  X sends the packet to E2 because E2 paid X to receive packets for E2. E2 accepts the packet because he wants to receive packets addressed to him.

And that's how everybody gets paid for E1 to send packets to E2. Comparable payment relationships exist for every combination of two endpoints which attempt to communicate with each other on the Internet.

__ Key Takeaways __

Sometimes E2 is a transit customer of A or of B instead of being on the other side of a peering session. In that case A or B got paid twice and banked a little extra money. 
As often as not there's a peering session somewhere in the path. Either way, some key takeaways are: 

1. Everybody involved was directly or indirectly paid by E1, E2 or both to forward that packet to the next hop. 

2. Some or all of them were paid by only E1 or E2 but not both.

3. A packet may cross many paid transit links but no packet will cross two free peering links. 

Now you know who pays for the Internet. And how.

Finally, I offer you this estimate of the worldwide systemic cost of carrying a BGP route. The estimate is out of date but the methodology is sound enough to use for discussion purposes:

Before I dive in to how this structure might be beneficially changed with procedural aggregation, are there any questions about any of this? Anything I can clarify about how the money moves?

Bill Herrin

William D. Herrin ................
3005 Crane Dr. ...................... Web: <>
Falls Church, VA 22042-3004

rrg mailing list