Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text

Sebastian Moeller <> Sun, 01 November 2020 13:03 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id DC53D3A07E6; Sun, 1 Nov 2020 05:03:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.649
X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id RrnNIcFM_Swu; Sun, 1 Nov 2020 05:03:18 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 652D33A0820; Sun, 1 Nov 2020 05:03:18 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1604235748; bh=EEh7n1KOIdMLzZbkAgnJ8DvJnJJIxHC5ZGJo2t2te6E=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=C3sVLRtX5ww6v5W4nFXJkhj8+RdOaFd+hypnLSoeGTUCBdor+EPNqnRgLTsUH1dt/ At+Sn4VHOcp3oZNX8oztCLD4mIB+qnYY/rNRwnkTby/HRYk7JSWEgP6KwhJ2MMswN2 34rVUL22mjv9l2XaUcLOjTFGNgA2q1g3mihbE0us=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [] ([]) by (3c-app-gmx-bs25.server.lan []) (via HTTP); Sun, 1 Nov 2020 14:02:28 +0100
MIME-Version: 1.0
Message-ID: <trinity-422295bd-0f86-4a94-b649-e1f4044ba0fe-1604235747990@3c-app-gmx-bs25>
From: Sebastian Moeller <>
To: Christian Huitema <>
Cc: Bob Briscoe <>, tsvwg IETF list <>, iccrg IRTF list <>, TCP Prague List <>, "De Schepper, Koen (Koen)" <>
Content-Type: text/plain; charset=UTF-8
Date: Sun, 1 Nov 2020 14:02:28 +0100
Importance: normal
Sensitivity: Normal
In-Reply-To: <>
References: <> <>
Content-Transfer-Encoding: quoted-printable
X-UI-Message-Type: mail
X-Priority: 3
X-Provags-ID: V03:K1:hzvJxyoDylKZVHhiXLqno+1Prnh9ezBFoUmnT3xHwwSWsA1WdKrQL3x3sOodJXv1Y+pqx dmhdckHjFCIQdhOBDQnrx2YNFTpHyVVFgSVj4bf9/9MJAZatoWzlLHUKsQ7yRAZcxBK6OXM0/wDn 1Wocn2ZAw5eBfv/Gfk6waGvGWx+AQNrofxMZt3DvCDa/D+KpP/LYMRkD0B0Gvlen+YLCuYKMxQkv 90vl6TjH2hGH5eP0gjwX5TkHD1QROsopL9x9XQQySrY6q2zcV5hUV9zIbMv0z0kJqnPcYng0uoJL W4=
X-UI-Out-Filterresults: notjunk:1;V03:K0:Ak71coR95FM=:K1XGHQG6RTVs1Ztn5jONgj i8zYzngoKMn8V5Kd7G6Rk0e2Uxzjv64MfQx4dYn2xaBvgXHSkq55MEaF+3DzdD9F++8DQIWm6 +cEtBl78YCUkdI0HsgRimiqfSqsCjQUCLOBT6C12bD6/l9WWASXhwITBLA6Xe2EDz7DRD7AID 1cmylyya3GoUX5B7O1K0j62k1gv0GUCWs0d2pvUk1lfR33KKxZfB7jx+gyDI8kxd04C/33zwC D/CUfDLxI61ykbYU49OX4ifEoyuNkxtadk+hvn51ERvj+99h6ibUpmVcHPGsmeVYj4ZST8XIr PakVl2IB+yqud2Dt3MLJ6PsgJ9wrAFMqdcBpj6Zla3oUFWQI7GEKZsDtZ7plv+9J3vIbRADkY yX5ldWjYinOnPODkjBRP62iUhnxnxZ1mLNTCLCbojzl3kbuMzw/+dj4RGzXjOKzHaErWqMqd7 3ZfkETsptduTS9qms2W82UIKwxyPJVtS86SB/ypEe3jKsh3XYrx2/ryStSXN7tPC0x87kZJqU QC7hD46BXuTohHDisn1jyJ3VFr9ziwCZrOzZ3nywVccl7tX1BRcJMuLrhtWPmjn5WPRs5BieQ qnIic3mTr5wUs=
Archived-At: <>
Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 01 Nov 2020 13:03:21 -0000

Dear Christian,

I do not really want to drag you into the L4S protocol requirements issue, but since you seem to be contemplating making your protocol L4S-aware and -compliant, what is your interpretation of the "eliminate RTT bias" requirement in its original and in Bob's proposed modified version. Did you understand that this required you to account for 15ms higher sustained RTT under load in the classic queue of an L4S AQM by slowing down the CC response by ECT(1) flows by the same 15ms? If no, could you opinie whether my proposed change for this requirement is any clearer? 
I top post this as it does not address your specific issues (but I am quite intersted in learning team L4S' response to your questions), but seems sufficiently related to merit as response (past attempts at soliciting input from L4S implementers fell on deaf ears, so I will not cold-post any such questions anymore).

Thank you very much in advance!
Best Regards

Gesendet: Sonntag, 01. November 2020 um 02:07 Uhr
Von: "Christian Huitema" <>
An: "Bob Briscoe" <>et>, "tsvwg IETF list" <>
Cc: "iccrg IRTF list" <>rg>, "TCP Prague List" <>rg>, "De Schepper, Koen (Koen)" <>
Betreff: Re: [tsvwg] [tcpPrague] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text
I am reading the L4S ECN-AQM proposal with an eye on responding to it in an implementation of QUIC, and I have a couple of questions regarding use of ECN marking with QUIC.
The document does not mention QUIC, yet QUIC is already used in a large fraction of Internet traffic. QUIC does specify support for ECN, and QUIC acknowledgements may carry counts of each category of ECN marks received from the peer -- three counters for ECT(0), ECT(1) and CE. In theory, QUIC implementations could take advantage of L4S -- in fact, at least one implementation supports DC-TCP like CC already. Is there interest in specifying L4S for QUIC?
My next question regards the interaction of the proposed L4S ECN-AQM with CC algorithms like BBR that attempt to discover the bottleneck packet rate for the connection, and use pacing to send packets at that rate. I observed that BBR is never mentioned in the draft, yet BBR is used in a sizeable part of Internet traffic. Do we have data on how a non-L4S aware implementation of BBR interacts with the proposed L4S AQM?
My last question regards potential use of ECT(1) marking. Most current implementations set ECT(0), but setting ECT(1) instead is trivial. This should elicit an L4S compatible response in L4S-AQM, and the BBR implementation might be modified to use the signals as part of the bottleneck bandwidth tracking. But there is a small issue there. With BBR, QUIC packets are supposedly paced at just under the bottleneck rate, except during "probe" periods in which they probe for 1 RTT at a slightly higher rate. The L4S AGM might degenerate in a form of ON-OFF control -- no feedback at all most of the time, then a bunch of CE marks if the probe rate exceeds the bottleneck bandwidth. As anyone experimented with that?
-- Christian Huitema

On 10/31/2020 2:54 AM, Bob Briscoe wrote:Folks,

The co-authors of ECN L4S ID have been reviewing the correctness of the normative 'Prague' requirements.
This is the second of 2 emails, about 2 of the requirements that we think ought to be reworded a little.

If you agree with the rationale, but think the new wording still doesn't fully capture the requirement, pls suggest sthg better.
If you disagree with the rationale, pls discuss.
4.3.  Prerequisite Congestion Response

   o  A scalable congestion control MUST react to ECN marking from a
      non-L4S but ECN-capable bottleneck in a way that will coexist with
      a TCP Reno congestion control [RFC5681[]] (see Appendix A.1.4[] for

      Note that a scalable congestion control is not expected to change
      to setting ECT(0) while it falls back to coexist with Reno.
   o  A scalable congestion control MUST implement monitoring in order
      to detect a likely non-L4S but ECN-capable AQM at the bottleneck.
      On detection of a likely ECN-capable bottleneck it SHOULD be
      capable (dependent on configuration) of automatically adapting its
      congestion response to coexist with TCP Reno congestion controls
      [RFC5681] (see Appendix A.1.4 for rationale and a referenced

      Note that a scalable congestion control is not expected to change
      to setting ECT(0) while it falls back to coexist with Reno.

1/ The requirement as currently written says what an omniscient sender MUST do. So there's an implied requirement that a sender MUST be omniscient, which is of course impossible.
2/ The requirement needs to be recast to require a sender to aim to be as knowledgeable as possible. Then, what it does as a result needs to take into account the a priori likelihood of there being a non-L4S bottleneck present.
3/ This includes the possibility that the operator of the host knows that the network it serves has not deployed any single queue classic ECN AQM (e.g. in a CDN case they're doing out of band testing, or they've asked the ISP). So we've included the possibility of fall-back being disabled by configuration.
4/ Nonetheless, as has been pointed out on the list, there is still a possibility that there is a Classic ECN AQM somewhere else on the path (to continue the CDN example, perhaps beyond the ISP in a home network). The 'MUST monitor' requirement still stands to ensure the operator doesn't miss these cases.
5/ Then, if the server operators have disabled fall-back for their deployment, they can reconsider their policy or at least do more focused testing if they are frequently detecting a single-queue Classic ECN AQM.

Items 3-5 are the "react via management" model that I've talked about on the list, given the unfairness doesn't amount to starvation, and it is possible that the prevalence of the problem is very low.

Finally, after the bullet list of requirements in section 4.3, (which are prerequisites for setting the ECT1 codepoint), we propose to add the following requirement, as suggested on the tsvwg list:

      To participate in the L4S experiment, a scalable congestion control MUST
      be capable of being replaced by a Classic congestion control (by
      application and by administrative control). A Classic congestion control
      will not tag its packets with the ECT(1) codepoint.



Bob Briscoe                     []
tcpPrague mailing[]