Re: [arch-d] A Public Option for the Core

Jared Mauch <> Wed, 12 August 2020 12:08 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id CBD9D3A122E for <>; Wed, 12 Aug 2020 05:08:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id oHzRRiI0ER80 for <>; Wed, 12 Aug 2020 05:08:27 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 77EEB3A0F86 for <>; Wed, 12 Aug 2020 05:08:27 -0700 (PDT)
Received: from [] ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPSA id E3C505400FC; Wed, 12 Aug 2020 08:08:24 -0400 (EDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.\))
From: Jared Mauch <>
In-Reply-To: <>
Date: Wed, 12 Aug 2020 08:08:23 -0400
Cc: Lars Eggert <>,
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <>
To: Brian E Carpenter <>
X-Mailer: Apple Mail (2.3608.
Archived-At: <>
Subject: Re: [arch-d] A Public Option for the Core
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: open discussion forum for long/wide-range architectural issues <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 12 Aug 2020 12:08:29 -0000

> On Aug 11, 2020, at 6:53 PM, Brian E Carpenter <> wrote:
> Maybe it's time to update
> Reactions to that draft from the then major transit ISPs were interestingly
> negative.

Looking at this 20+ years in the future, there’s some interesting dynamics going on, which many of us are interested in for the centralization discussions that are ongoing.

There’s the usual last-mile vs major city dynamics at play when it comes to cost.  Construction is expensive, but the fiber isn’t the expensive bit, it’s the dirt movement that costs quite a bit.

With more traffic traversing private backbones vs public ones, there is also a lack of measurements that are feasible.  You can’t measure from within the Akamai backbone as it’s our private network.  Public networks, you can utilize things like RIPE Atlas or similar, but there are significant limitations to it.  Cellular network measurement either require some applications at scale that run in the background, but the various charging cultures that exist globally around voice/sms/data pose challenges.

Public sources of routing data are generally available from the major backbones (route-views, ripe ris, etc) but not for the private clouds (Microsoft, google, amazon, ovh, ibm/oracle).  

The costs of running these backbones are often much lower than public and allow us to optimize for our business use cases in a way a public network can’t solve for everyone.

This is in part a natural economic progression.  Why cloud computing?  Because power is expensive at my house, so I put data on a VM somewhere else for $5/mo.  If I put it in a colocation facility I may pay up to 2500/mo for colocation plus the server and bandwidth costs, plus then I have to deal with swapping failed disk vs $5/mo as a service.

We don’t often talk about these economic factors that drive the shift, but look at the architecture.  It’s not devoid of economic costs or impacts.  Someone is paying for all that compute at Google/Facebook and power, it’s often not a direct expense we see, like we don’t see the credit card fees that merchants pay.

Why are things like neutral IX not successful?  It requires people to sacrifice and value their time very low.  It’s not a surprise that some of the larger IX points operate their own networks to extend their reach (their own private backbones).  Not everyone can interconnect at the speeds necessary at the neighborhood level, there must be some aggregation and it may not be close.

My personal telecom expenses are relatively fixed over time, but the bits/second tend to go up over time.

I am hoping speeds continue to go up, costs go down and the risk of centralization of the network follows the same risk evaluations that people have done in 2020 with their supply chain risks and how they were often centralized.  Diversity is helpful, but if we are always driving for the lowest cost as a consumer, it will take a few shocks to restore that diversity and supply chain depth and balance the ecosystem.

- jared