Re: [tcpm] [Ecn-sane] [tsvwg] ECN CE that was ECT(0) incorrectly classified as L4S

Sebastian Moeller <> Fri, 02 August 2019 13:15 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 4BE2B1200DB; Fri, 2 Aug 2019 06:15:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.349
X-Spam-Status: No, score=-2.349 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id abF0A_vM1GTM; Fri, 2 Aug 2019 06:15:36 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 7DA251200CD; Fri, 2 Aug 2019 06:15:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1564751726; bh=ugUBSujZuKIGXPHRqQ4v/2lEkJOKLnlRX2iF5ZGTGbQ=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=O894eUXy95vo7VUQ2yHP/ML5KIiVO7p6WrlZ0ypxLTHfMcqBZT0SJXHobJG6ZTMeC z4t12PXp6TxBTshD7mMDP4bnUNI1nTw2as3oeDtAmM1mmx9t4frek/7qMK+HmtWEQ+ N3xWww4WsOIBnX3Re+ZSaO5Avf97rqTmrENcErg4=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [] ([]) by (mrgmx103 []) with ESMTPSA (Nemesis) id 0LjZn2-1iV58w0lqq-00bXpz; Fri, 02 Aug 2019 15:15:26 +0200
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <>
Date: Fri, 02 Aug 2019 15:15:23 +0200
Cc: Jonathan Morton <>,, ECN-Sane <>,
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <FRXPR01MB0710085AE4AEFA896819AFA69CD90@FRXPR01MB0710.DEUPRD01.PROD.OUTLOOK.DE>
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:6XHtP07waXiolpH/2jXVlCbpP1vT9X1zFgvFyqTpj7OtlxGi626 37sAbLEu3/s/iftInsG0fKwoKx3og88lAUSI1LdF3Jixg/k+f5T2eeyR1lXDPOtSUpQGBUX tIqn33x4DjcMw8Od3kcD2//WM4oeX5Vs3Wt4aG11xxMU+Han7EKSDZF6bx75atervwIz8E/ cGxpiTQw98xJM4dWxTnZQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:YqH0/9ThHVQ=:3pUbOKbFr67gK0m/1tdivo +TVOwRbgLzp3Zmv+bUX8TbZ0QzsY4XvkLf64AtUVTNOeUAL8U1gOO87ofozwx2CAg0eVYlgIL JmDY5+4ekArsLOe9c3CAGjCUjrXiA7WXRuxenOQlkDIDxvlDcRq1FVbdarprw1g/wIuCERuDK V1UfAYKXVbWKpRxc3upF0HoCRF9l91tqZD7gmi7v4kRME8dvrWMb04GmKajN9J6UG13Nwj/bK o7v4yAogCCfqgl8kKKxew4FwA3xw4zAIrboZMsAhe3PDEJnTxaL8I4CUT5qftQzwiz9g0lkmz 2fBESRa4VFGu/JxRHS+8GlobDpPfKUiYY2oYvomC0PB2yz/0fZ8YnZnHJt8DKUrdTG/cWomhT JWgdaEY+VXsV1YiSDjorqjFD+4c9rNeK7zFtWvvVmjiz2Bp/aeVozKBSrXfwSsmvcDpbppjU7 XMepC/PSkg1fjqOK/5xvrGCm/Vrmj9vhYXPPbMl0L69gTTPBkf/QFhRgCtKfBhblNsdL3xY58 jkxpjg6hL/6rCchz8VArzMV/tIk2KhU9fzvtjptoWab957HGnZ/ojvXrWFhgXMLhKCUUa5geH 8ogzrOvQafNJO1WKcjJ8380JhBEaG7LFPXEOSH6C+CV1sWooeBIiAulOvJbPew76egc4CGrK9 mJ/v6IZYz5fKeebkOhWRtaZsJkngcdGr6dv1YeYi2Jm6oy1F3F4eLx9FRcBFWErz+8/2RHmbt laD928QXC+wPUrcFOLjg7c4JIc5KIwKAR3M4IIw1P4IrpOt+plxH0WdEfnVeVMeQbFEufVSBK 1cMnszF2JU1TFj14VfHcknhFMZLZP7e++5Q7HaGLaZTh+dvm3KoViymbyOKpgqtLtlmi1SOFU BRHPygV7hznJ7y4p43cH0/FNccpsVu3mbZyUswQFSJWgurP50aqxP12JXJjZivbhwUb5ct4qQ mW7xxq0glwkHfyKh6CTUBrbMijZaPANcP0449c2BcL8JdBmVOS+yhB8AmtFSbwT4glbCOVZ1R K6V54Yp9Iwv55Zp3SdR7ViLzQlEeutkYnQAtWimLiG9YPh6ORT12e3iqaJTNN6vrBInejRtqX piz/wy22VdYpbJg33L5VSo7g2PsatQdk9rpLV0R0Sja83SIgbgW6jsY0GfjEMB8iIk+hYWE5/ P3DjU=
Archived-At: <>
Subject: Re: [tcpm] [Ecn-sane] [tsvwg] ECN CE that was ECT(0) incorrectly classified as L4S
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 02 Aug 2019 13:15:38 -0000

Hi Ruediger,

> On Aug 2, 2019, at 10:29, <> <> wrote:
> Hi Jonathan,
> could you provide a real world example of links which are consecutively narrower than sender access links?

	Just an example from a network you might be comfortable with, in DTAGs internet access network there typically are traffic limiting elements at the BNGs (or at the BRAS for the legacy network), I am not 100% sure whether these are implemented as policers or shapers, but they tended to come with >= 300ms buffering. Since recently, the BNG/BRAS traffic shaper's use the message field of the PPPoE Auth ACK to transfer information about the TCP/IPv4 Goodput endusers can expect on their link as a consequence of the BNG/BRAS"s traffic limiter. In DOCSIS and GPON networks the traffic shaper seems mandated by the standards, in DSL networks it seems optional (but there even without a shaper the limited bandwidth of the access link would be a natural traffic choke point).
 Fritzbox home router's now use this information to automatically set egress (and I believe also) ingress traffic shaping on the CPE to reduce the bufferbloat users experience. I have no insight in what Telekom's own Speedport routers do, but I would not be surprised if they would do the same (at least for egress). 
	As Jonathan and Dave mentioned, quite a number of end-users, especially the latency sensitive ones, employ their own ingress and egress traffic shapers at their home routers as the 300ms buffers of the BNG's are just not acceptable for any real-timish uses (VoIP, on-line twitch gaming, even for interactive sessions like ssh 300ms delay are undesirable). E.g. personally, I use an OpenWrt router with an FQ AQM for both ingress and egress (based on Jonathan's excellent cake qdisc) that allows a family of 5 to happily share a 50/10 connection between video streaming and interactive use with very little interference between the users, the same link with out the FQ-AQM active makes interactive applications feel like submerged in molasses once the link gets saturated...
	As far as I can tell there is a number of different solutions that offer home-router based bi-directional traffic shaping to solve bufferbloat" from home (well, not fully solve it, but remedy its consequences), including commercial options like evenroute's iqrouter, and open-source options like OpenWrt (with sqm-scripts as shaper packet). 
	It is exactly this use case and the fact that latency-sensitive users often opt for this solution, that causes me to ask the L4S crowd to actually measure the effect of L4S on RFC3168-FQ-AQMs in the exact configuration it is actually used today, to remedy the same issue L4S wants to tackle.

Best Regards

> I could figure out a small campus network which has a bottleneck at the Internet access and a second one connecting the terminal equipment. But in a small campus network, the individual terminal could very well have a higher LAN access bandwidth, than the campus - Internet connection (and then there's only one bottleneck again).
> There may be a tradeoff between simplicity and general applicability. Awareness of that tradeoff is important. To me, simplicity is the design aim. 
> Regards,
> Ruediger 
> -----Ursprüngliche Nachricht-----
> Von: tsvwg <> Im Auftrag von Jonathan Morton
> Gesendet: Dienstag, 9. Juli 2019 17:41
> An: Bob Briscoe <>
> Cc: tcpm IETF list <>;; tsvwg IETF list <>
> Betreff: Re: [tsvwg] [Ecn-sane] ECN CE that was ECT(0) incorrectly classified as L4S
>> On 13 Jun, 2019, at 7:48 pm, Bob Briscoe <> wrote:
>>      1.  It is quite unusual to experience queuing at more than one
>>          bottleneck on the same path (the available capacities have to
>>          be identical).
> Following up on David Black's comments, I'd just like to note that the above is not the true criterion for multiple sequential queuing.
> Many existing TCP senders are unpaced (aside from ack-clocking), including FreeBSD, resulting in potentially large line-rate bursts at the origin - especially during slow-start.  Even in congestion avoidance, each ack will trigger a closely spaced packet pair (or sometimes a triplet).  It is then easy to imagine, or to build a testbed containing, an arbitrarily long sequence of consecutively narrower links; upon entering each, the burst of packets will briefly collect in a queue and then be paced out at the new rate.
> TCP pacing does largely eliminate these bursts when implemented correctly.  However, Linux' pacing and IW is specifically (and apparently deliberately) set up to issue a 10-packet line-rate burst on startup.  This effect has shown up in SCE tests to the point where we had to patch this behaviour out of the sending kernel to prevent an instant exit from slow-start.
> - Jonathan Morton
> _______________________________________________
> Ecn-sane mailing list