Re: [tsvwg] [Ecn-sane] Compatibility with singlw queue RFC3168 AQMs

Sebastian Moeller <> Fri, 26 July 2019 16:07 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id EE2BE1201C8 for <>; Fri, 26 Jul 2019 09:07:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.348
X-Spam-Status: No, score=-2.348 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 63wIZH2vH4zV for <>; Fri, 26 Jul 2019 09:07:56 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 51FF6120048 for <>; Fri, 26 Jul 2019 09:07:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1564157215; bh=qi0zwc14JrhuCqL3Y0PUllL4Y8jvSEgVOz/WgLNXvAk=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=kVu/zk14AanpHOVxzI/KcGkOm4Brano7rc1xsWwQ/OoW1fds8MYyN2YiVMVjlwve4 f4nFV4wf3jky2OJEIbjqQGPAAj4Jhp97+EP7ohRWK+FUb2GZxhMW7ZC/4Vajo+S/Rf uR+e3MDcWkJdRFwmr2iR66ifPVj+zivrnI/fOXX4=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from hms-beagle2.lan ([]) by (mrgmx104 []) with ESMTPSA (Nemesis) id 1MF3DM-1hbdTT0Wa0-00FPIM; Fri, 26 Jul 2019 18:06:55 +0200
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <>
In-Reply-To: <>
Date: Fri, 26 Jul 2019 18:06:52 +0200
Cc: Bob Briscoe <>, "" <>, "" <>, Dave Taht <>, "De Schepper, Koen (Nokia - BE/Antwerp)" <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
To: "Black, David" <>
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:Yfu1HOADIxKe4a1njkPhWyoANoQ83Cnf9GjCdHYWB62nApTQJy9 dvZyufAKRiT+LWf0rYeAvOctSNe6HWTcY8FMmRC/J5E5jawlA7oS5tSKTPVRQqKorXcSBHN 3nDmziWMACh1t4PANPdUwSVIKOq2EHYl7RL1NZSBM58xLoW9M15fz8T5tf0iNiYsnHFFnxe CpX28wUlDoEB6AHc0mxNQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:QeaqRFys1Is=:bgF6DNIS61JboM5vZ2Fu55 Njltio3i342UHilzpVGW/zcgNB+ceYf/liYmIHXqVpAZUxohpAYQFXHRfMkCAZIxQRbgqTGMB 3CEuhW9bpIMX9qxshPzlqDoqqMnNVoMLsSGRWooJBeLftIuAXR9Uz+ssBeFFFjd7dauV9QWVv WG30N73Z4vm6g6izv9iyPFwU7J4W57/hT3hkeG0j5p3EobRxdVHQ8fY4Yafzv6X849pcffDSL NAwjqzjsC5kcYxQImgMWQZ8GxHqtPI4RsuB9ccvy2VhHkoV8fb9yk3YfXmrNVDFZUx2JG1BnI 4DaH8581lTH6C1N8CvUio5kmRmTRo2zObJysgG5iV1m+dQVPLcYuyN9kNtntwwORBKU7BgfKM e1x2vFft+ODiI50dK72LwHNqOYvT9F0zsbOXq4acpbywWKi8hY6sLM5suque0t9jh/FsGMnWe +VaRGWT9O/1GyFNZhc0TdAbtTvPbLSLG7SxA0OYIRLIs14Xd0YtCuB4ZUgxloAe4I6jbUeZxq Sxe4a6okFDrlt2EHYd/cqoEWao+hez5LJaVdMLsLzZJiM0AuYtgnwGE6Z7L4dux5huXQ6uF0H MN2HjQEP0qzBEfnfMSGYv0EuqJYwMfoyLJLhPduvpwUOQUlpArBRptfc3OuJsIzELLXFealvc 4nTRpc5bF5VOxL1AXNlYDZmff6Fg6l5tmi4jZQTuOfFnJLeVTTCufzXDO8SYsdiE0bJqczmnS MHmu5lnTYhKyKqVcvP/c5/5CYR1zo5MJi7RnoR3+JnAUiU7kocVUbuBogml18aj4WM1d32DUe H4KfKiNDJOpYz+MLy9nTnx4+17xZJoO4uUJIz3iWmR700OCOHqCD6GaoxWF2ZPBge6pu2sLIN 0Kt9tRKRSPst6GWOK63Z5tSoJzg48JmD/qCHRYBT+8wUunkolADlWnYyYvn2SV5gatOtb5MDW uD4xJn3FnPUpxTTqw+OrjiV0WAWkvk3obGzGBPfSz1R/lr95BFosELoXOj4Oe8M7nYdJbesPC VNatPF/7O2WOWY4kSwOkEC/T/SVimWg4SjgtZba+XDJLDH/Kpq16mWRtctABAhGqCPWIYBtzU kj7l6Pv7REUKGg=
Archived-At: <>
Subject: Re: [tsvwg] [Ecn-sane] Compatibility with singlw queue RFC3168 AQMs
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 26 Jul 2019 16:08:02 -0000

Dear David,

thanks for your clearing things up. I see, I should have read deeper into the relevant "web" of RFCs before asking.

Am I correct in interpreting the following  sentence from RFC 8311:
"ECN experiments are expected to coexist with deployed ECN
   functionality, with the responsibility for that coexistence falling
   primarily upon designers of experimental changes to ECN."
as meaning, that L4S will need to implement the long discussed fall-back to RFC3168 compliant responses to CE marks, if a RFC3168 AQM is detected as being active on a path, and that L4S endpoint need to closely monitor for signs of RFC3168 behavior? I ask because section 4.1 fails to put in those safe-guard clauses explicitly (in my reading this effectively says anything goes, as long as it is defined in its own RFC)

Now looking at the L4S RFC I see ( (assuming that this is one of the RFCs required to allow the exemption according to RFC8311)):

"Classic ECN support is starting to materialize on the Internet as an
   increased level of CE marking.  Given some of this Classic ECN might
   be due to single-queue ECN deployment, an L4S sender will have to
   fall back to a classic ('TCP-Friendly') behaviour if it detects that
   ECN marking is accompanied by greater queuing delay or greater delay
   variation than would be expected with L4S (see Appendix A.1.4 of [I-D.ietf-tsvwg-ecn-l4s-id]).  
   It is hard to detect whether this is
   all due to the addition of support for ECN in the Linux
   implementation of FQ-CoDel, which would not require fall-back to
   Classic behaviour, because FQ inherently forces the throughput of
   each flow to be equal irrespective of its aggressiveness."

Which I believe to be problematic, as it conflates issues. The problem with L4S-CE response on non L4S-AQMs is that it will give L4S flows an unfair and unexpected advantage, so L4S endpoints should aim at detecting non-L4S AQMs on the path and not (just) "that ECN marking is accompanied by greater queuing delay or greater delay variation than would be expected with L4S". Sure delay variations can be a eans of trying to detect such an AQM, but this text basically gives L4S the license to just look at RTT variations and declare victory if these stay below an arbitrary threshold.
	Also I voiced concerns about the rationale for excluding RFC3168 FQ-AQMs from this fall-back treatment, and gave an explicit example of a system in use (post-true bottleneck ingress shaping) that I would like to see to be tested first. This should be easy to test (and as far as I know these tests are planned if not already done) so that the RFC can either be amended with a link to the data showing that this is harmless, or changed ot indicate that the fall-back might also be required for FQ-AQMs under certain conditions.

Now if I look at, I see the following:

"A.1.4.  Fall back to Reno-friendly congestion control on classic ECN bottlenecks

   Description: A scalable congestion control needs to react to ECN
   marking from a non-L4S but ECN-capable bottleneck in a way that will
   coexist with a TCP Reno congestion control [RFC5681].

   Motivation: Similarly to the requirement in Appendix A.1.3, this
   requirement is a safety condition to ensure a scalable congestion
   control behaves properly when it builds a queue at a network
   bottleneck that has not been upgraded to support L4S.  On detecting
   classic ECN marking (see below), a scalable congestion control will
   need to fall back to classic congestion control behaviour.  If it
   does not comply with this requirement it could starve classic

   It would take time for endpoints to distinguish classic and L4S ECN
   marking.  An increase in queuing delay or in delay variation would be
   a tell-tale sign, but it is not yet clear where a line would be drawn
   between the two behaviours.  It might be possible to cache what was
   learned about the path to help subsequent attempts to detect the type
   of marking."

Here, the special casing of FQ-AQMs does not seem to be present, which L4S RFC will have precedence here?

Anyway, am I correct in interpreting all of the above as a clear an unambiguous requirement for L4S components like TCP-Prague to implement RFC3168-AQM detection and fall-back to appropriate behavior before being given the permission for usage on the wider internet?

Best Regards

> On Jul 26, 2019, at 16:10, Black, David <> wrote:
> Inline comment on "IETF's official stance":
>> The first option seems highly undesirable to me, as a) (TCP-friendly) single queue
>> RFC3168 AQM are standards compliant and will be for the foreseeable future, so
>> ms making them ineffective seems like a no-go to me (could someone clarify
>> what the IETF's official stance is on this matter, please?),
> The IETF expects that all relevant technical concerns such as this one will be raised by participants and will be carefully considered by the WG in determining what to do.
> That was the technical answer, now for the official [officious? :-) ] answer ... the current L4S drafts do not modify RFC 3168 beyond the modifications already made by RFC 8311.  If anyone believes that to be incorrect, i.e., believes at least one of the L4S drafts has to further modify RFC 3168, please bring that up with a specific reference to the text in "RFC 3168 as modified by RFC 8311" that needs further modification.
> Thanks, --David
>> -----Original Message-----
>> From: Sebastian Moeller <>
>> Sent: Friday, July 26, 2019 6:20 AM
>> To: Bob Briscoe
>> Cc: Black, David;;; Dave Taht; De
>> Schepper, Koen (Nokia - BE/Antwerp)
>> Subject: [Ecn-sane] [tsvwg] Compatibility with singlw queue RFC3168 AQMs
>> Dear Bob,
>> we have been going through the consequences and side effects of re-defining
>> the meaning of a CE-mark for L4S-flows and using ECT(1) as a flllow-classifying
>> heuristic.
>> One of the side-effects is that  a single queue ecn-enabled AQM will CE-marl L4S
>> packets, expecting a strong reduction in sending rate, while the L4S endpoints
>> will only respond to that signal with a mild rate-reduction. One of the
>> consequences of this behaviour is that L4S flows will crowd out RFC3168 and
>> non-ECN flows, because these flows half their rates on drop or CE-mark
>> (approximately) making congestion go away with the end result that the L4S
>> flows gain an undesired advantage, at least that is my interpretation of the
>> discussion so far.
>> Now there are two options to deal with this issue, one is to declare it
>> insignificant and just ignore it, or to make L4S endpoints detect that condition
>> and revert back to RFC3168 behaviour.
>> The first option seems highly undesirable to me, as a) (TCP-friendly) single queue
>> RFC3168 AQM are standards compliant and will be for the foreseeable future, so
>> ms making them ineffective seems like a no-go to me (could someone clarify
>> what the IETF's official stance is on this matter, please?), b) I would expect most
>> of such AQMs to be instantiated close to/at the consu,er's edge of the internet,
>> making it really hard to ameasure their prevalence.
>> In short, I believe the only sane way forward is to teach L4S endpoints to to the
>> right thing under such conditions, I believe this would not be too onerous an ask,
>> given that the configuration is easy to set up for testing and development and a
>> number of ideas have already been theoretically discussed here. As far as I can
>> see these ideas mostly riff on the idea that such anAQM will, under congesation
>> conditions, increase each ftraversing flow's RTT and that should be quickly and
>> robustly detectable. I would love to learn more about these ideas and the state
>> of development and testing.
>> Best Regards & many thanks in advance
>> 	Sebastian Moeller