Re: [tsvwg] [Ecn-sane] ECN CE that was ECT(0) incorrectly classified as L4S

Jonathan Morton <> Fri, 02 August 2019 09:47 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 9E7A812007C; Fri, 2 Aug 2019 02:47:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.749
X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id KTBaRaJQNsyY; Fri, 2 Aug 2019 02:47:11 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::32a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id F241412004E; Fri, 2 Aug 2019 02:47:10 -0700 (PDT)
Received: by with SMTP id u25so55377174wmc.4; Fri, 02 Aug 2019 02:47:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=2bNlBJG7BPKPCc0Bkg+mHeGfatut4ewrSZR9mqXgnzo=; b=p6kD7vvAyLaEztIRPRGWhrHu6uH5r1PN4dsPPATlO3oFTvUIdTBj14mWHMR0yPRbQs 91/sNw1p1we9QmXjJjQcE4zi2AwtUvh9FcYGG4gp39RKYR/jf9dDfaFHOP2eJVwB77cd B1gX+1uTIQ2WW8533+yWQ6PIO5UU+eOG0sfOuWR3+ZD5TpAvp1Ld+Pn5PTayhTvV9zdC Ju4vb9j5kGHGrhcqOPl+feomUzauGlPg9vbMpLaQ9Yym57f2nzggZf/OOz69c66Pc2I7 jLXDfszdUGrRdDc4xIdHudCWFge8wH5azv7t2K2oVTKdxXeUcg57ESHluIrh6Kqn2yH6 pUvg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=2bNlBJG7BPKPCc0Bkg+mHeGfatut4ewrSZR9mqXgnzo=; b=pjS8V7StPx/t8ANYcyg74wDCIUhKu2WEtnesTnhxkJ8yLAWSSlol1ki57hfciBe3M/ WTJzt0TaKStfiW059KPQ+3NHS1TIkbw+MJ+TSaAyF0oLXScdnphPMY0wyUb9hj5LzZRG lUfd1XA+ahxaFp8MPg9HMMEfztyS+IB3I620iwEcd2NpJ5pt0qc9UVRB2lHNh7TFWv9a ah5+E/jhf1jSJUIZVN8JfN3kkx07F6c/2sY/TVduE7pVv6fTLvhjdZpqZwqurVgOPrhq VXtgA+HifHkQrKlowRxsX0Qkm9k+eJ6KTmJbs7BkHiE53SeeuxYdOUnPu/AAye1m1+vA Hkqg==
X-Gm-Message-State: APjAAAWsDIgiwGOzFYvgKNVWFk1hQ/Yxe73ExNhb78+ztevKWWrr4TGa 1LUCku1x3+1rMfQMGih9V3o=
X-Google-Smtp-Source: APXvYqwu4RXOzQ7LFFDoP/0Hcz/SjulHkJmfSX4btD5ZDtIA13+YWjLjHSm5GysXGrCAobunqza+yQ==
X-Received: by 2002:a7b:cae9:: with SMTP id t9mr3740479wml.126.1564739229409; Fri, 02 Aug 2019 02:47:09 -0700 (PDT)
Received: from [] ( []) by with ESMTPSA id w67sm104132142wma.24.2019. (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Aug 2019 02:47:08 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
From: Jonathan Morton <>
Date: Fri, 2 Aug 2019 10:47:06 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <FRXPR01MB0710085AE4AEFA896819AFA69CD90@FRXPR01MB0710.DEUPRD01.PROD.OUTLOOK.DE>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <>
Subject: Re: [tsvwg] [Ecn-sane] ECN CE that was ECT(0) incorrectly classified as L4S
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 02 Aug 2019 09:47:13 -0000

> On 2 Aug, 2019, at 9:29 am, <> <> wrote:
> Hi Jonathan,
> could you provide a real world example of links which are consecutively narrower than sender access links?
> I could figure out a small campus network which has a bottleneck at the Internet access and a second one connecting the terminal equipment. But in a small campus network, the individual terminal could very well have a higher LAN access bandwidth, than the campus - Internet connection (and then there's only one bottleneck again).
> There may be a tradeoff between simplicity and general applicability. Awareness of that tradeoff is important. To me, simplicity is the design aim. 

A progressive narrowing of effective link capacity is very common in consumer Internet access.  Theoretically you can set up a chain of almost unlimited length of consecutively narrowing bottlenecks, such that a line-rate burst injected at the wide end will experience queuing at every intermediate node.  In practice you can expect typically three or more potentially narrowing points:

1: Core to Access network peering.  Most responsible ISPs regularly upgrade these links to minimise congestion, subject to cost effectiveness.  Some consumer ISPs however are less responsible, and permit regular congestion here, often on a daily and/or weekly cycle according to demand.  Even the responsible ones may experience short-term congestion here due to exceptional events.  Even if the originating server's NIC is slower than the peering link, queuing may occur here if the link is congested overall due to statistical multiplexing.

2: Access to Backhaul provisioning shaper.  Many ISPs have a provisioning shaper to handle "poverty tariffs" with deliberately limited capacity.  It may also be used to impose a sanity check on inbound traffic bursts, to limit backhaul network traffic to that actually deliverable to the customer (especially when the backhaul network is itself subcontracted on a gigabytes-carried basis, as is common in the UK).  In the ADSL context it's often called a BRAS.

3: Backhaul to head-end device.  Generally the backhaul network is sized to support many head-end devices, each of which serves some number of consumer last-mile links.  I'm being deliberately generic here; it could be a CMTS on a cable network, a DSLAM in a phone exchange, a cell tower, or a long-range wifi hub.  In many cases the head-end device shares several subscribers' lines over a common last-mile medium, so there is still some statistical multiplexing.  In the particular case of a cell tower, the subscriber usually gets less link capacity (due to propagation conditions) than his tariff limit.

4: CPE bufferbloat mitigation shaper.  This is *post-last-mile* ingress shaping, with AQM and FQ, to reduce the effects of the still-ubiquitous dumb FIFOs on the above bottlenecks, especially the head-end device.  The IQrouter is a ready-made CPE device which does this.

5: LAN, wifi, or powerline link.  Most people now have GigE, but 100base-TX is still in use in cheaper CPE and some low-end computers, and this will be a further bottleneck in cases where the last mile is faster.  For example, the Raspberry Pi has only just upgraded to GigE in its latest versions, and used 100base-TX before.  Wifi is also a likely bottleneck, especially if the mobile station is on the opposite side of the house from the base CPE, and is additionally a "bursty MAC" with heavy aggregation.  Some CPE now runs airtime-fairness and FQ-AQM on wifi to help manage that.

I think the above collection is not at all exotic.  Not all of these will apply in every case, or at all times in any given case, but it is certainly possible for all five to apply consecutively.

 - Jonathan Morton