Re: [tsvwg] L4S status: #17 Interaction w/ FQ AQMs

Sebastian Moeller <moeller0@gmx.de> Mon, 11 November 2019 00:59 UTC

Return-Path: <moeller0@gmx.de>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C9331200E5 for <tsvwg@ietfa.amsl.com>; Sun, 10 Nov 2019 16:59:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.649
X-Spam-Level:
X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Oy09XHuqnlp2 for <tsvwg@ietfa.amsl.com>; Sun, 10 Nov 2019 16:59:17 -0800 (PST)
Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 244DF12001A for <tsvwg@ietf.org>; Sun, 10 Nov 2019 16:59:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1573433955; bh=3FbJdO7uclUvoEUAKsEPlUpzage23rqpg5yYwXi/Qp0=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=Q+r8XDUWBD7d6Vlp9jjfpfcRMydtfccj4wWV72uLjJr8HbRBgnxvvIr2Gd/GXMrRR W+r8vfHPiI+ZvK7pRrNvW5tOtgSE7RW+K+++ECrLNI98S4CekdSg1H1vm2fXqdxe8e IVTIXzRq7OTD+y1vmkQmj8qf3LLty/KGvjr6C5c8=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from hms-beagle2.lan ([77.6.95.82]) by mail.gmx.com (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MPGW7-1iG0Q7135Z-00Pad6; Mon, 11 Nov 2019 01:54:06 +0100
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <moeller0@gmx.de>
In-Reply-To: <VI1PR07MB598167BC77A7FEFB8D220F4BE0740@VI1PR07MB5981.eurprd07.prod.outlook.com>
Date: Mon, 11 Nov 2019 01:54:04 +0100
Cc: Pete Heist <pete@heistp.net>, "tsvwg@ietf.org" <tsvwg@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <E0D269DA-D750-40BA-B7ED-1A8B510C437F@gmx.de>
References: <8321f975-dfe7-694c-b5cc-09fa371b9b61@mti-systems.com> <B58A5572-510E-42C7-8181-42A0BE298393@gmail.com> <D2E12331-F504-4D5F-B8E7-A1A5E98DDF7E@cablelabs.com> <2275E6A5-C8F8-477F-A24A-3E6168917DDF@gmail.com> <55F724CD-6E74-40D9-8416-D1918C2008DD@cablelabs.com> <BBE7C7A9-0222-4D84-BF27-8D5CAE2F995E@gmail.com> <6f189711-ffa0-90f4-fd16-3464ba4df3ce@mti-systems.com> <4A706B11-3239-4DAC-BE85-0B4BFF2D8FF8@heistp.net> <8B28ECE4-FF4B-4BB2-ACBE-80B30708F97E@cablelabs.com> <AAEA9AC2-B8A1-4837-A7C9-8EEA21A7C523@gmx.de> <D5D560CB-BC47-45BE-811E-E73E2D4909E3@cablelabs.com> <3332A911-3AA0-4986-9AA9-B97266A3337F@heistp.net> <VI1PR07MB598169BE81401DF2B4FD2CF2E07B0@VI1PR07MB5981.eurprd07.prod.outlook.com> <949AF60C-EA09-44ED-8521-7C333DF326D4@gmx.de> <VI1PR07MB598167BC77A7FEFB8D220F4BE0740@VI1PR07MB5981.eurprd07.prod.outlook.com>
To: "Tilmans, Olivier (Nokia - BE/Antwerp)" <olivier.tilmans@nokia-bell-labs.com>
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:tB0Z2u1LQshUtw3YpXJ6CdnoFdMwMhPMSjIUbnBIh9Ar7mzGD0W 6Pe/f9U+e0/ts6pLUUfT7QH2HATDVbZgJSm6+6PvwHBcEXhSgWjvKBRoNI6pOop4pwdfUQt MCY2qvsxJ9homJCsrL8IEU9AuppV11hFVhXAQIHpaoEQXNtvXuC0oRn7sRa+hMftHeRC47K xXSKziKsYpCtJqfrqyEvQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:1DlbYfdmueg=:N0xY7NtZS5AwidfxvRfOZl NpywDmvFhG4OBqu+meAZETeG4tIYOpzfnmPfBXvCd6qldv27wxJ9cHnMQErma/wMsBcZsU0Ps fm7n7+9frZfvDHdgHL7Yjgjh/ZMTZq0MRTMHmp3EXeg6HqVy8BxaXuStxGIMshA3Raqfo0SYV OITdgSRdzXcANlGCuleY5j3zSiG1BDp0ahF2YfiqWsxaL+O4KyX4MX0n8pivXXs3OMtKK6VUI js7De3F0jZMUbKgH70Dyjsh1WqmeO00N2p6wg/gbY/mffFlIr0q8vKoHMjOEBlUyGNANXQTo4 6hPEq2tjC8yl8esVa4blRFWq8UREIJni59w8vq80wpcRf5Dp4+ue1acRQKS9ow/SBShSwmdzE nyIvR02UqCKCvaauqQLFizAdBekSq83FBGOSL70f952HqdhtAYqOZT2yFnFPGHmeXtiroQ0yG G0XcmFfNRLFBj+/3R7NgKWA45f9YWfCy3QZxN1ONdc+CpLeG6hXOtiXrYx9sdzGm2juXkHu+z m/x0GMV/rcbker0VvxcIW74munCV18y5aZzcYpc7b2PjVfU26H7ob1y9PWdn6q+0uGCY7f9ox butPyVd9J1HL+qnk93TVh6BtSCMVJopEe3djg4UM25QMd2GLwZfTgOSnqmAV0ROrbkEXYZzy4 qsH7S2ioI/nBWW1La1HB58QnP/HNRZhVo7M4lso3CvW3i0Vdx6TuIzLdKosabXqwLHcg1BzQ/ oY1PffbFLyzr3xQlwhYGq0i5JYR56uCjLJNsjjUg4ytgMP6t/z1libCH5XxqgFt0L/tCrHDvH SoX0SfLJ+bAeqqdfN7euhCE4gQi0v7RkYknT9bMgbjIY2tRnda3nZkJwRLS0mkM5Ty8YYILLO 5yBQR/4LB033cl9z1xkpLWMempKpncFgYPOWvCNt1SOmHQYWjQmZSQClPiJLGJkSBAXJBzSmJ uW2W+AEqrA45trB9AFLHTkpvzkjV88+pKUqAMA3gb13HP3IbH+GzFkOPdpTj7O14nVJ/m32p+ siRIU4tfYKDo/QQZVEkaNCHwl4wnMKFtlq1iyix+THZUlhazD2wOfdEEQZ0yU4MxfU9UAUTmT DiKkqTvctPswzRQ/SGH3iIxC4QoVl4yWjwBTrIS0Re6LdWMQBARn46A54F6Kw6vl5h9VIaeqU D2hmJkk6jrTy+d/B6E62egufpeRBHYB3PjMA2DHgmOfnblZviGbdwbZm3md6ucSgaPE8kEqor crCvGk5+wyPWgL60oguSRhqPHN9hgs6GERZDdGUlknXWhHL1dIrmNa6VXiOk=
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/mesQoFvo-dMDTee7fy5kVOfBfTo>
Subject: Re: [tsvwg] L4S status: #17 Interaction w/ FQ AQMs
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Nov 2019 00:59:19 -0000

Hi Oliver,

> On Nov 11, 2019, at 01:38, Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-labs.com> wrote:
> 
> Hi Sebastian,
> 
>> I wonder how you intend to deal with scenario 1, cubic versus prague, where
>> prague for short RTTs pummels cubic throughput, quite unlike what one
>> naively could expect from reading the dualQ draft:
>> 
>> "Analytical study and implementation testing of the Coupled AQM have
>>   shown that Scalable and Classic flows competing under similar
>>   conditions run at roughly the same rate."
>> 
>> at 0ms RTT prague is ~10 times faster than cubic, and at 10ms this still is
>> around a factor of 3, only at 80ms that gets into what I would consider
>> "roughly the same rate". 
> 
> I had the same question a couple of months ago when introduced to the dualQ.

	[SM] You are telling me you were sitting on the knowledge that DualQ does not work as advertised, and just accepted that? No attempt have been made to remedy this (like setting the WRR to 50%).

> The short answer here is that both flows do not compete under similar conditions:
> they operate at different RTTs, which varies depending on their propagation delays.
> E.g., at 0ms base, L4S flows see 1ms on average while classic flows see 15ms; at
> 10ms base, L4S sees 11ms and classic 25ms.

	[SM] Sorry that is not how humans interpret the claim in the introduction of the dualQ rfc. Similar conditions clearly imply the externally visible path length (aka unloaded RTT). 

> 
> Given that flows have different overall RTTs, but are subject to the same marking
> probabilities (albeit tuned to their reponse, i.e., linear or multiplicative), you
> instead need to look at their cwnd--which must be equal[1]. Unfortunately, eye-balling
> this from flent graph is not super convenient--tracing it, e.g., using eBPF, is
> however straightforward.

	[SM] This confirms my hypothesis that dualQ is not the proper tool here, just compare https://l4s.cablelabs.com/l4s-testing/key_plots/batch-l4s-s1-2-cubic-vs-prague-50Mbit-0ms_var.png with https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-13T045427-r1/l4s-s2-2/batch-l4s-s2-2-cubic-vs-prague-50Mbit-10ms_fixed.png. This is the expected 50:50 haring behaviour the dualQ draft implies, this is what you need to get close to if you want dualQ to be allowed on the internet, as on short RTTs to the nearest DC, L4S/TCP Prague will absolutely eat normal TCP's lunch and dinner. 

> 
> [1] The 0ms case will likely yield a smaller than expected cwnd for L4S, cause by
> 10% WRR protection kicking it and artificially increasing classic's cwnd (as 1ms
> vs 15ms would yield a 15x throughput difference if it was reno).
> 
>> I now wonder what happens if you pair a short RTT
>> (say 10ms) prague flow (simulating traffic from a nearby data center) with a
>> cubic flow with a typical internet RTT (say 80ms). My prediction is that
>> prague will dominate cubic similarly to the 0ms RTT. I guess this is coming
>> from the default 1/16 scheduling weight for the normal queue, do you agree?
> 
> You can disable the WRR entirely (i.e., set it to 0%) and the results will stay
> the same (except for the 0ms case...).

	[SM] Again that implies that on intranet timescales dualQ does not offer the quality of isolation required to have L4S and non-L4S-style traffic peacefully coexist. But that is dualQ's whole "raison d'être". And this in the best case scenario, with an L4S AQM, this is quite sad. 


> 
> 
> Best,
> Olivier
> 
>> 
>> 
>> 
>> 
>>> On Nov 8, 2019, at 11:41, Tilmans, Olivier (Nokia - BE/Antwerp)
>> <olivier.tilmans@nokia-bell-labs.com> wrote:
>>> 
>>>> Hi Greg, would it be possible to merge any changes you’d like to include
>> in
>>>> further testing into the testing (default) branch at
>>>> https://github.com/L4STeam/linux? We’ll evaluate what we can with what
>> time
>>>> there is, but a prerequisite to that is making sure we have the right
>>>> changes you want tested. :)
>>> 
>>> Hi Pete,
>>> 
>>> As mentioned at https://l4s.cablelabs.com/l4s-testing/README.html (link
>>> at the bottom of the page Greg linked), we ran your full set of flent
>>> tests using this tag:
>>> https://github.com/L4STeam/linux/releases/tag/testing%2F5-11-2019
>>> 
>>> I doubt we'll push anything else on that repo for the moment being
>>> unless someone comes out with a bug/fix/improvement.
>> 
>> 	Well, I would like to see scenario 1 with unequal RTTs between the
>> two flows, like prague@10ms vs. cubic@80ms, and prage@80ms vs. cubic@10ms
>> (and the same for cubic vs. cubic)
>> and I would like to see tests pitting 2 flows of one type versus one flow of
>> the other type, like 2 cubic versus 1 prague and 2 prague vs. 1 cubic.
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> 
>>> Best,
>>> Olivier
>