Re: [tsvwg] [Ecn-sane] Comments on L4S drafts

Sebastian Moeller <> Tue, 23 July 2019 10:34 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 8D2BB12002E for <>; Tue, 23 Jul 2019 03:34:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.648
X-Spam-Status: No, score=-1.648 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id EOxQtX0rRxnI for <>; Tue, 23 Jul 2019 03:34:00 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 731701201C9 for <>; Tue, 23 Jul 2019 03:33:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1563877988; bh=lIHwbYIb76wtenit+K/CZtcQFdZx1D4FvUhVvuktILM=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=Fg7MHnZl9afbuW40ZtNtBtOI+/pJN6bNC0ss465ITPfXPIir+slVGMs+3IJgUkbAo yul9hS16x5qqEyQC3KJIkXMDvtoJ7KI7yflFVfbEwU4SWUkyi7fghhH9V+nQvzjv45 rqJExX90aNH73YZUFXWsbsaUtGPZZeZ8fxAEOUaY=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from hms-beagle2.lan ([]) by (mrgmx002 []) with ESMTPSA (Nemesis) id 0MF4eJ-1heaUx2Sr5-00GIqQ; Tue, 23 Jul 2019 12:33:08 +0200
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <>
In-Reply-To: <>
Date: Tue, 23 Jul 2019 12:33:06 +0200
Cc: Jonathan Morton <>, Bob Briscoe <>, "Black, David" <>, "" <>, "" <>, Dave Taht <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
To: "De Schepper, Koen (Nokia - BE/Antwerp)" <>
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:Nfn1zlINeCaaV4c536HBCMHbcHqs67IBRDZcQA9pL+m4ooG33M0 3QQlk3+KDVN3aVV0Rxcfu7zBgqGpj5quiPIg40FdG358dlSHA5hCylc97BKp1shdytn3LCf FXWrFA12qMwt132hXzooZ2uiGyB0L3QiMZ8f0nBg936vGBu75gEi7lxBW94mfTE0fGiQM8G GOX/GiEntiTkavdTybyow==
X-UI-Out-Filterresults: notjunk:1;V03:K0:pgeBJLhI/UY=:F85iyAaJbDeR+YfYy9Jzx1 QrRzrTjAUWm/U83+7MsRtKixJzWz0//2pyroHbXO+BqgvMYD/cmvA1d+VH7GeDkv9BzO0Hh7y +bYGDdBFsGTYYPi/DXtlfyfb4J0TSEMrKHcdd+7Zjj3iX23mPMPVTtFsYHRanGQtybyGyzfhf 8amDlnTTLAM03MwKpPhCUqeiAhcRqpTXJkTPTYaM/XCC91ExNgJMZhD9904sTWxsDIYEVrgvO BlTBXQop998KE1K7u9odLRXBMQjvCkVp1jzKxY46E1wQES/33Ur5dydvD0Y4oHlkJkntGC2eM K22eB6X+QGzhauBvMSN04hYncD/C1yXeLYMKYe+oDqoyhwDbgybX6PlB8n9fiQEIbOD+mDCpm 7Nh+zgEjZ2M3FgL99tL+9ulIVjLBakW1VwQL6aYm/wtQ0UkmK6EZpxMsI+Cp+4PfYKX5VLm9u baWN0eeLf5J1wY7KkaYHz1AE9cbm878droU7gxl1050cDsMtfznNgrfFOJBh8t432r5QWq+GE k3EeinXB2L7XkxvET3yVQLtwbpNaA2VdKIgjY4lHfibRhin93DaAkH2QllVNsydNf2RtnDBYX i31teyM+L42nrzYugY36C8QIgEBV/hQKRuFcP+HXxe12FtKNz5BQdSmugU51dpNzmv6pS9lXC tsPg/fOf10JTLjzUscQZo7WrKuSFNTON1nOSW08f6pdccvh1VbptQ/PC/oNmtlTxU5VZ0UQEP Er6RG9OIELmMm331UQuH3VV2FXFwFBZjCuaXOxum4cW5jhOENmL413BDOrLI+bX8nZguYL8o3 vYYt50Wzw43Fd3cQmLY8lew9JdDV2O+WCGos8eH2SjxC0qHdVqhES59Y1QzIdObOjM7qUQt+4 ylip1LBWqFTNonPyEPpY1sP0ABiUGxzRUZ2zfKyidbdvDwOJMd3XukN+CiJnp1v7L1MUqnxng O6cpwC9ri3WpBYOLabyByMF3lKWiIpKnBC2UK/ISVgEhzd5sOV5fj8+jJdafthxPV+E31YmR9 E8HhwYuvsMox4Y7ODZ6E1MizIAYq0A25WIx6q4VVzf3wgRL3dMzzIC17DntyDXzmADkM5KkU3 BFWt6FzhC+zjug=
Archived-At: <>
Subject: Re: [tsvwg] [Ecn-sane] Comments on L4S drafts
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 23 Jul 2019 10:34:02 -0000

Hi Koen,

> On Jul 22, 2019, at 20:15, De Schepper, Koen (Nokia - BE/Antwerp) <> wrote:
> Jonathan,
> I'm a bit surprised to read what I read here... I had the impression that we were on a much better level of understanding during the hackathon and that :
> - we both agreed that the latest updates in the Linux kernels had quite some impact on DCTCP's performance (burstyness) that both you and we are working on. As also our testbed showed it had the same impact on DualPI2 and FQ-Codel (yes we do understand FQ_Codel and did extensively compare DualQ with it since the beginning of L4S).
> - the current TCP-Prague we have in the public GitHub, which is DCTCP using accurate ECN and ect(1) and is drop compliant with Reno, is what SCE can use as well, and whatever you called SCE-TCP can be used for L4S, as (what I showed you mathematically) it is actually perfectly working according to DCTCP's law of 1/p, because it is DCTCP with some simple pacing tweaks you did. I thought we agreed that there is no difference in the congestion control part, and we want the same thing, and the only difference is how to use the code-point.
> - related to the testbed setups, we have several running, the first since 2013. We support all kernel versions since 3.19 up to the latest 5.2-rc5. We have demonstrated L4S since 2015 in IETF93 and the L4S BoF with real equipment and software that is still the same as we use today.
> - the testbed I brought (5 laptops and a switch that got broken during travel and I had to replace in the nearest shop), I had to install during the hackathon from scratch from our public GitHub (I arrived only at 14:00 on Saturday) which we made immediately available for you guys to put the flent testing tools on.
> - related to the flent testing, you might have expected to find big differences, but both measurements showed exactly the same results. I understood you need to extent your tools to get more measurement parameters included which were missing compared to ours.
> - we planned to complete your test list during this week and maybe best that we jointly report on the outcome of those to avoid different interpretations again.
> - anybody who had interest in L4S could have evaluated it since we made our DUALPI2 code available in 2015 (actually many did).

	`Well, at IETF 104 there was a promise on the lists of VMs with both endpoints for a L4S system, which as far as I can tell never materialized, which made me refrain from testing... And I believe I did ask/propose the VM thing on this very list and got no response.

> (To Dave That: if you wanted to evaluate DualPI2 you had plenty of opportunity, 4 years by now. I find it weird that suddenly you were not able to install a qdisc in Linux. Even if you wanted us to setup a testbed for you, you could have asked us.)
> Maybe some good news too, we also had a (first time right) successful accurate ECN interop test between our Linux TCP-Prague and FreeBSD Reno (acc-ecn implementation provided by Richard Scheffenegger).
> I hope these accusations of incompetence can stop now, and that we get to the point of finally getting a future looking low latency Internet deployed.

	??? sorry to be so negative, but the "getting [...] deployed" part is out of our control. 

> Anybody else who doubts on the performance/robustness of L4S, let me know and we arrange a test session this week.

	Not that it counts for much, but I am neither convinced of L4S reaching its stated performance or robustness goals under adversarial conditions and long RTTs. I am looking forward to the outcome of this weeks testing (and hope my concerns will have been unfounded).

Best Regards

> Koen.
> -----Original Message-----
> From: Jonathan Morton <> 
> Sent: Sunday, July 21, 2019 6:01 PM
> To: Bob Briscoe <>
> Cc: Sebastian Moeller <>de>; De Schepper, Koen (Nokia - BE/Antwerp) <>om>; Black, David <>om>;;; Dave Taht <>
> Subject: Re: [Ecn-sane] [tsvwg] Comments on L4S drafts
>> On 21 Jul, 2019, at 7:53 am, Bob Briscoe <> wrote:
>> Both teams brought their testbeds, and as of yesterday evening, Koen and Pete Heist had put the two together and started the tests Jonathan proposed. Usual problems: latest Linux kernel being used has introduced a bug, so need to wind back. But progressing.
>> Nonetheless, altho it's included in the tests, I don't see the particular concern with this 'Cake' scenario. How can "L4S flows crowd out more reactive RFC3168 flows" in "an RFC3168-compliant FQ-AQM". Whenever it would be happening, FQ would prevent it.
>> To ensure we're not continually being blown into the weeds, I thought the /only/ concern was about RFC3168-compliant /single-queue/ AQMs.
> I drew up a list of five network topologies to test, each with the SCE set of tests and tools, but using mostly L4S network components and focused on L4S performance and robustness.
> 1: L4S sender -> L4S middlebox (bottleneck) -> L4S receiver.
> This is simply a sanity check to make sure the tools worked.  Actually we fell over even at this stage yesterday, because we discovered problems in the system Bob and Koen had brought along to demo.  These may or may not be improved today; we'll see.
> 2: L4S sender -> FQ-AQM middlebox (bottleneck) -> L4S middlebox -> L4S receiver.
> This is the most favourable-to-L4S topology that incorporates a non-L4S component that we could easily come up with, and therefore .  Apparently the L4S folks are also relatively unfamiliar with Codel, which is now the most widely deployed AQM in the world, and this would help to validate that L4S transports respond reasonably to it.
> 3: L4S sender -> single-AQM middlebox (bottleneck) -> L4S middlebox -> L4S receiver.
> This is the topology of most concern, and is obtained from topology 2 by simply changing a parameter on our middlebox.
> 4: L4S sender -> ECT(1) mangler -> L4S middlebox (bottleneck) -> L4S receiver.
> Exploring what happens if an adversary tries to game the system.  We could also try an ECT(0) mangler or a Not-ECT mangler, in the same spirit.
> 5: L4S sender -> L4S middlebox (bottleneck 1) -> Dumb FIFO (bottleneck 2) -> FQ-AQM middlebox (bottleneck 3) -> L4S receiver.
> This is Sebastian's scenario.  We did have some discussion yesterday about the propensity of existing senders to produce line-rate bursts occasionally, and the way these bursts could collect in *all* of the queues at successively decreasing bottlenecks.  This is a test which explores that scenario and measures its effects, and is highly relevant to best consumer practice on today's Internet.
> Naturally, we have tried the equivalent of most of the above scenarios on our SCE testbed already.  The only one we haven't explicitly tried out is #5; I think we'd need to use all of Pete's APUs plus at least one of my machines to set it up, and we were too tired for that last night.
> - Jonathan Morton