Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <moeller0@gmx.de>
In-Reply-To: <HE1PR07MB4425DD6FE15DB130B24BCEA1C27A0@HE1PR07MB4425.eurprd07.prod.outlook.com>
Date: Sat, 9 Nov 2019 21:38:00 +0100
Cc: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>,
 "tsvwg@ietf.org" <tsvwg@ietf.org>, "tcpm@ietf.org" <tcpm@ietf.org>,
 "iccrg@irtf.org" <iccrg@irtf.org>,
 "koen.de_schepper@nokia.com" <koen.de_schepper@nokia.com>,
 "Bob Briscoe (ietf@bobbriscoe.net)" <ietf@bobbriscoe.net>
Content-Transfer-Encoding: quoted-printable
Message-Id: <DBE43199-0BFB-464B-8045-919C975397AC@gmx.de>
References: <HE1PR07MB4425A148B5BB1E6FD8E5A3FBC27A0@HE1PR07MB4425.eurprd07.prod.outlook.com>
 <2083E62F-0E6D-40B1-B726-A198BFA36220@gmx.de>
 <HE1PR07MB4425DD6FE15DB130B24BCEA1C27A0@HE1PR07MB4425.eurprd07.prod.outlook.com>
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/yA29W1PE8a82wemowZEcKj5sPgk>
Subject: Re: [tsvwg] L4S related activity in 3GPP
Precedence: list

Dear Ingemar,

more in-line below.

> On Nov 9, 2019, at 20:24, Ingemar Johansson S =
<ingemar.s.johansson@ericsson.com> wrote:
>=20
> Hi Sebastian=20
>=20
> Please see inline below
> /Ingemar
>=20
>> -----Original Message-----
>> From: Sebastian Moeller <moeller0@gmx.de>
>> Sent: den 9 november 2019 18:13
>> To: Ingemar Johansson S
>> <ingemar.s.johansson=3D40ericsson.com@dmarc.ietf.org>
>> Cc: tsvwg@ietf.org; tcpm@ietf.org; iccrg@irtf.org; Ingemar Johansson =
S
>> <ingemar.s.johansson@ericsson.com>; koen.de_schepper@nokia.com
>> Subject: Re: [tsvwg] L4S related activity in 3GPP
>>=20
>> Dear Ingemar,
>>=20
>> thanks for posting this. I read "This document describes an Active =
Queue
>> Management mechanism, =E2=80=98L4S=E2=80=99, that can support =
low-latency services in a way
>> that does not degrade the performance of other traffic." and I wonder =
whether
>> 3GPP has run any experiments to confirm/support this claim, that you =
are willing
>> to share?
>=20
> [IJ] We have done simulation studies with our proprietary 4G/5G system =
simulation tools but I can not disclose the results from these.

	[SM] Ah, I see, not posting internal data is fine, but by all =
means, go and perform real-world measurements as well with all the parts =
you actually want to roll-out (in other words testing with dctcp for =
example is scientifically interesting, unless you expect your users to =
actually use dctcp on your links).


>>=20
>> Sidenote: in the powerpoint file it says about L4s " Standardized in =
IETF [1]",
>> which seems premature, in the process of being standardized seems a =
better
>> description of the status quo...
>=20
> [IJ] Yes, you are right about this, it should have been "Subject to =
standardization..."

	[SM] Fair enough, as much as I regret to say it, standardization =
is quite likely.

>=20
>>=20
>>=20
>> I ask about test data from your side as the recent test data is quite =
sobering
>> from a number of angles, including quality of isolation:
>=20
> [IJ] First of all the outline of the proposal for L4S in 5G is to use =
separate radio bearers for L4S and classic (or normal or non-L4S) =
traffic. This means that it is the 5G QoS framework and the scheduling =
in the RAN (Radio Access Network) that dictates the fairness.

	[SM] Sounds like a workable approach, that side steps the dualQ =
implementation/theory issues.


> But of course there are wireline transport link in the core network =
that may be using DualQ when that becomes commercially available. The =
bitrates over these links are however likely to be considerably higher =
than 50Mbps.

	That does not matter, the issue is not the limit of 50 Mbps, but =
the fact that dualQ, at last the current implementation in conjunction =
with the current TCP prague implementation, simply do not reliably and =
robustly do what the rfc draft claims. My fear is that dualQ, with its =
weak coupling vie marking probabilities, simply is a design, that only =
guarantees approximate fairness. In other words I doubt that dualQ works =
as a naive reading of the RFC implies. But if I  understood Bob correct, =
the only gurantee he is willing to give is L4S does not starve non-L4S =
traffic. Which IMHO is not sufficient to merit inflicting the L4S =
experiment on the wider internet. Now, that is not my call to make, but =
the IETF process allows me t voice my concerns, and I just use this =
opportunity in an attempt to minimize undesired side-effects on the =
"normal" internet.

>=20
>>=20
>> 	Looking at the current L4S tests from the various testbeds I see =
severe
>> breakdown of the promised isolation between bulk and the =
"low-latency" queue,
>> apparently making the current implementation, if not the concept =
itself IMHO
>> unfit for release into the wider internet, before these issues have =
been
>> sufficiently addressed. Especially the observation that the weighted =
scheduler
>> built in as a safety mechanism in the dualQ aqm seems to trigger =
considerably
>> more often than expected/predicted from reading the dualQ draft =
(giving 1/16
>> of the bandwidth to on of two classes _might_ be justified as a =
temporary stop-
>> gap measure in extremely unlikely cases, but this already triggers in =
a simple
>> one-flow-per queue scenario at low RTTs). In short it currently looks =
like the
>> "does not degrade performance of other traffic" part is not yet =
finished.
>>=20
>> 	To illustrate my criticism, here is a link to a test comparing =
the simplest
>> functionality test imaginable, running one L4S-style and one =
non-L4S-style TCP
>> connection through the L4S dual queue AQM at a intranet like RTT =
(nomnally
>> 0ms, but in reality):
>> https://protect2.fireeye.com/v1/url?k=3Dfcd5c73d-a05c68f4-fcd587a6-
>> 0cc47ad93e1c-6acfac650abe7f09&q=3D1&e=3D6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=3Dhttps%3A%2F%2Fl4s.cablelabs.com%2Fl4s-
>> =
testing%2Fkey_plots%2Fbatch-l4s-s1-2-cubic-vs-prague-50Mbit-0ms_var.png
>> As can be seen the L4S flow (labeled prague) ends up very quickly =
taking ~90% of
>> the available bandwidth, leaving only 10% to the non-L4S flow =
(labeled cubic),
>> wit increasing RTT this bias gets less extreme, but even at 80ms the =
prague flow
>> still gets around 1.5 times the bandwidth of the cubic flow
>> (https://protect2.fireeye.com/v1/url?k=3D77272802-2bae87cb-77276899-
>> 0cc47ad93e1c-56d0590db27309ef&q=3D1&e=3D6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=3Dhttps%3A%2F%2Fl4s.cablelabs.com%2Fl4s-
>> =
testing%2Fkey_plots%2Fbatch-l4s-s1-2-cubic-vs-prague-50Mbit-80ms_var.png),=

>> indicating an RTT dependent performance degradation for "other =
traffic". Test
>> from another testbed confirm the break-down of fairness between L4S =
and non-
>> L4S traffic at short RTTs =
(https://protect2.fireeye.com/v1/url?k=3D5710f0ef-
>> 0b995f26-5710b074-0cc47ad93e1c-418920d155a62bf0&q=3D1&e=3D6cd6c372-
>> b712-46be-8b10-
>> 22955dec7b38&u=3Dhttps%3A%2F%2Fwww.heistp.net%2Fdownloads%2Fsce-l4s-
>> bakeoff%2Fbakeoff-2019-09-13T045427-r1%2Fl4s-s1-2%2Fbatch-l4s-s1-2-
>> cubic-vs-prague-50Mbit-0ms_fixed.png).
>=20
> [IJ] I understand that the 0 RTT results can be because the congestion =
window is only a few segments large (my guess is ~3 to 4 segments ?). I =
know that this is a problem with e.g. DCTCP.  The question is whether =
this is a permanent problem or something that will be solved over time?.

	[SM] With DualQ, I assume it is a structural problem, instead of =
designing this as a robust system with two fair sharing queues it does =
its clever "indirect coupling" via marking/dropping probabilities. Have =
a look at =
https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-13T045427=
-r1/l4s-s2-2/batch-l4s-s2-2-cubic-vs-prague-50Mbit-10ms_fixed.png for =
how sharing in a FQ system looks. Even with the leaky ECT(1) as =
classifier abuse it shoukd be simple to fair queue between EXT(1) and =
not ECH(1) flows and have robust separation. This is not rocket science, =
and mostly, as far as I can tell a solved problem.

> What is more interesting to me and what makes L4S attractive for the =
use cases outlined in the 3GPP contributions, is the very low queue =
delay with L4S.

	[SM] Well, I believe low queueing delay can be achieved without =
dualQ and without abusing the ECT(1) codepoint as a classifier (the =
desired classifier requires one complete independent bit, ECT(1) is just =
1/2 of a bit, and hence has failure modes in separating L4S-style flows =
and non-L4S flows). The idea of using the finer-grained congestion =
signaling of DCTCP over the wider internet seems sane see the SCE draft =
proposal as an alternative that would/will allow dctcp-style ECN =
signaling over the internet in a truly backward compatible fashion...


> There are other promising features with L4S. One being that it can =
enable congestion control algorithms to quickly grab free capacity.

	[SM] When you say L4S, do you mean the congestion signaling? =
Then https://tools.ietf.org/html/draft-morton-taht-sce-00 should have =
you covered, without the nasty side-effects of the current L4S proposal. =
Or do you mean the actual congestion controller as implemented in TCP =
prague?=20


> 5G access can have large variations in link throughput for example =
related to dual connectivity and L4S can be helpful here as well.

	[SM] Dealing better with variable rate links is certainly =
desirable, but I am not sure that DualQ/TCP Prague really are the best =
or only tools for that purpose. At least not until all of the kinks are =
ironed out. Again, none of this is my call to make, so all I am voicing =
is my educated subjective assessments.


Best Regards
	Sebastian

>=20
>>=20
>> 	Personally, I can not help thinking that it might be better to =
first make
>> sure there is a complete, real-world tested implementation that =
reliably
>> demonstrates that dualQ delivers the required isolation between the =
two traffic
>> classes and that the L4S approach actually delivers the reported =
reliable low
>> latency performance under adverse real-world conditions while not =
degrading
>> the performance of other traffic, before trying to codify L4S into =
RFCs.
>>=20
>>=20
>> Best Regards
>> 	Sebastian
>>=20
>>=20
>>> On Nov 9, 2019, at 17:00, Ingemar Johansson S
>> <ingemar.s.johansson=3D40ericsson.com@dmarc.ietf.org> wrote:
>>>=20
>>> Gentlepeople!
>>>=20
>>> This 3GPP SA2 contribution is recently submitted
>>> =
https://protect2.fireeye.com/v1/url?k=3D803d870e-dcb428c7-803dc795-0cc47
>>> ad93e1c-77a60175f8460bc6&q=3D1&e=3D6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=3D
>>>=20
>> https%3A%2F%2Fwww.3gpp.org%2Fftp%2Ftsg_sa%2FWG2_Arch%2FTSGS2_136
>> _Reno%
>>> 2FDocs%2FS2-1911250.zip Title =E2=80=9C5QI values for efficient =
handling of
>>> congestion control marked IP packets=E2=80=9D
>>> The contribution introduces L4S to the SA2 audience and explains =
where the
>> technology is useful. The contribution also proposes a method to bake =
L4S into
>> the 5G QoS framework.
>>> If agenda time permits, it will be discussed at the SA WG2 meeting =
in Reno on
>> Nov 18-22, hopefully with good results.
>>>=20
>>> Also a RAN2 contribution was submitted and discussed at an earlier =
RAN2
>> meeting.
>>> =
https://protect2.fireeye.com/v1/url?k=3D06eb4ef7-5a62e13e-06eb0e6c-0cc47
>>> ad93e1c-f387514c9f633599&q=3D1&e=3D6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=3D
>>>=20
>> https%3A%2F%2Fwww.3gpp.org%2Fftp%2FTSG_RAN%2FWG2_RL2%2FTSGR2_1
>> 07bis%2F
>>> Docs%2FR2-1913888.zip
>>>=20
>>> Enjoy
>>> /Ingemar
>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D
>>> Ingemar Johansson  M.Sc.
>>> Master Researcher
>>>=20
>>> Ericsson Research
>>> Network Protocols & E2E Performance
>>> Labratoriegr=C3=A4nd 11
>>> 971 28, Lule=C3=A5, Sweden
>>> Phone +46-1071 43042
>>> SMS/MMS +46-73 078 3289
>>> ingemar.s.johansson@ericsson.com
>>> www.ericsson.com
>>>=20
>>>      The best way to find out if you
>>>  can trust somebody is to trust them
>>>              Ernest Hemingway
>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20