Re: [tsvwg] L4S related activity in 3GPP

[Sebastian Moeller <[e-mail]>]

Dear Ingemar,

more in-line below.

> On Nov 9, 2019, at 20:24, Ingemar Johansson S <[e-mail]> wrote:
> 
> Hi Sebastian 
> 
> Please see inline below
> /Ingemar
> 
>> -----Original Message-----
>> From: Sebastian Moeller <[e-mail]>
>> Sent: den 9 november 2019 18:13
>> To: Ingemar Johansson S
>> <ingemar.s.johansson=[e-mail]>
>> Cc: [e-mail]; [e-mail]; [e-mail]; Ingemar Johansson S
>> <[e-mail]>; [e-mail]
>> Subject: Re: [tsvwg] L4S related activity in 3GPP
>> 
>> Dear Ingemar,
>> 
>> thanks for posting this. I read "This document describes an Active Queue
>> Management mechanism, ‘L4S’, that can support low-latency services in a way
>> that does not degrade the performance of other traffic." and I wonder whether
>> 3GPP has run any experiments to confirm/support this claim, that you are willing
>> to share?
> 
> [IJ] We have done simulation studies with our proprietary 4G/5G system simulation tools but I can not disclose the results from these.

	[SM] Ah, I see, not posting internal data is fine, but by all means, go and perform real-world measurements as well with all the parts you actually want to roll-out (in other words testing with dctcp for example is scientifically interesting, unless you expect your users to actually use dctcp on your links).


>> 
>> Sidenote: in the powerpoint file it says about L4s " Standardized in IETF [1]",
>> which seems premature, in the process of being standardized seems a better
>> description of the status quo...
> 
> [IJ] Yes, you are right about this, it should have been "Subject to standardization..."

	[SM] Fair enough, as much as I regret to say it, standardization is quite likely.

> 
>> 
>> 
>> I ask about test data from your side as the recent test data is quite sobering
>> from a number of angles, including quality of isolation:
> 
> [IJ] First of all the outline of the proposal for L4S in 5G is to use separate radio bearers for L4S and classic (or normal or non-L4S) traffic. This means that it is the 5G QoS framework and the scheduling in the RAN (Radio Access Network) that dictates the fairness.

	[SM] Sounds like a workable approach, that side steps the dualQ implementation/theory issues.


> But of course there are wireline transport link in the core network that may be using DualQ when that becomes commercially available. The bitrates over these links are however likely to be considerably higher than 50Mbps.

	That does not matter, the issue is not the limit of 50 Mbps, but the fact that dualQ, at last the current implementation in conjunction with the current TCP prague implementation, simply do not reliably and robustly do what the rfc draft claims. My fear is that dualQ, with its weak coupling vie marking probabilities, simply is a design, that only guarantees approximate fairness. In other words I doubt that dualQ works as a naive reading of the RFC implies. But if I  understood Bob correct, the only gurantee he is willing to give is L4S does not starve non-L4S traffic. Which IMHO is not sufficient to merit inflicting the L4S experiment on the wider internet. Now, that is not my call to make, but the IETF process allows me t voice my concerns, and I just use this opportunity in an attempt to minimize undesired side-effects on the "normal" internet.

> 
>> 
>> 	Looking at the current L4S tests from the various testbeds I see severe
>> breakdown of the promised isolation between bulk and the "low-latency" queue,
>> apparently making the current implementation, if not the concept itself IMHO
>> unfit for release into the wider internet, before these issues have been
>> sufficiently addressed. Especially the observation that the weighted scheduler
>> built in as a safety mechanism in the dualQ aqm seems to trigger considerably
>> more often than expected/predicted from reading the dualQ draft (giving 1/16
>> of the bandwidth to on of two classes _might_ be justified as a temporary stop-
>> gap measure in extremely unlikely cases, but this already triggers in a simple
>> one-flow-per queue scenario at low RTTs). In short it currently looks like the
>> "does not degrade performance of other traffic" part is not yet finished.
>> 
>> 	To illustrate my criticism, here is a link to a test comparing the simplest
>> functionality test imaginable, running one L4S-style and one non-L4S-style TCP
>> connection through the L4S dual queue AQM at a intranet like RTT (nomnally
>> 0ms, but in reality):
>> https://protect2.fireeye.com/v1/url?k=fcd5c73d-a05c68f4-fcd587a6-
>> 0cc47ad93e1c-6acfac650abe7f09&q=1&e=6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=https%3A%2F%2Fl4s.cablelabs.com%2Fl4s-
>> testing%2Fkey_plots%2Fbatch-l4s-s1-2-cubic-vs-prague-50Mbit-0ms_var.png
>> As can be seen the L4S flow (labeled prague) ends up very quickly taking ~90% of
>> the available bandwidth, leaving only 10% to the non-L4S flow (labeled cubic),
>> wit increasing RTT this bias gets less extreme, but even at 80ms the prague flow
>> still gets around 1.5 times the bandwidth of the cubic flow
>> (https://protect2.fireeye.com/v1/url?k=77272802-2bae87cb-77276899-
>> 0cc47ad93e1c-56d0590db27309ef&q=1&e=6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=https%3A%2F%2Fl4s.cablelabs.com%2Fl4s-
>> testing%2Fkey_plots%2Fbatch-l4s-s1-2-cubic-vs-prague-50Mbit-80ms_var.png),
>> indicating an RTT dependent performance degradation for "other traffic". Test
>> from another testbed confirm the break-down of fairness between L4S and non-
>> L4S traffic at short RTTs (https://protect2.fireeye.com/v1/url?k=5710f0ef-
>> 0b995f26-5710b074-0cc47ad93e1c-418920d155a62bf0&q=1&e=6cd6c372-
>> b712-46be-8b10-
>> 22955dec7b38&u=https%3A%2F%2Fwww.heistp.net%2Fdownloads%2Fsce-l4s-
>> bakeoff%2Fbakeoff-2019-09-13T045427-r1%2Fl4s-s1-2%2Fbatch-l4s-s1-2-
>> cubic-vs-prague-50Mbit-0ms_fixed.png).
> 
> [IJ] I understand that the 0 RTT results can be because the congestion window is only a few segments large (my guess is ~3 to 4 segments ?). I know that this is a problem with e.g. DCTCP.  The question is whether this is a permanent problem or something that will be solved over time?.

	[SM] With DualQ, I assume it is a structural problem, instead of designing this as a robust system with two fair sharing queues it does its clever "indirect coupling" via marking/dropping probabilities. Have a look at https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-13T045427-r1/l4s-s2-2/batch-l4s-s2-2-cubic-vs-prague-50Mbit-10ms_fixed.png for how sharing in a FQ system looks. Even with the leaky ECT(1) as classifier abuse it shoukd be simple to fair queue between EXT(1) and not ECH(1) flows and have robust separation. This is not rocket science, and mostly, as far as I can tell a solved problem.

> What is more interesting to me and what makes L4S attractive for the use cases outlined in the 3GPP contributions, is the very low queue delay with L4S.

	[SM] Well, I believe low queueing delay can be achieved without dualQ and without abusing the ECT(1) codepoint as a classifier (the desired classifier requires one complete independent bit, ECT(1) is just 1/2 of a bit, and hence has failure modes in separating L4S-style flows and non-L4S flows). The idea of using the finer-grained congestion signaling of DCTCP over the wider internet seems sane see the SCE draft proposal as an alternative that would/will allow dctcp-style ECN signaling over the internet in a truly backward compatible fashion...


> There are other promising features with L4S. One being that it can enable congestion control algorithms to quickly grab free capacity.

	[SM] When you say L4S, do you mean the congestion signaling? Then https://tools.ietf.org/html/draft-morton-taht-sce-00 should have you covered, without the nasty side-effects of the current L4S proposal. Or do you mean the actual congestion controller as implemented in TCP prague? 


> 5G access can have large variations in link throughput for example related to dual connectivity and L4S can be helpful here as well.

	[SM] Dealing better with variable rate links is certainly desirable, but I am not sure that DualQ/TCP Prague really are the best or only tools for that purpose. At least not until all of the kinks are ironed out. Again, none of this is my call to make, so all I am voicing is my educated subjective assessments.


Best Regards
	Sebastian

> 
>> 
>> 	Personally, I can not help thinking that it might be better to first make
>> sure there is a complete, real-world tested implementation that reliably
>> demonstrates that dualQ delivers the required isolation between the two traffic
>> classes and that the L4S approach actually delivers the reported reliable low
>> latency performance under adverse real-world conditions while not degrading
>> the performance of other traffic, before trying to codify L4S into RFCs.
>> 
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>>> On Nov 9, 2019, at 17:00, Ingemar Johansson S
>> <ingemar.s.johansson=[e-mail]> wrote:
>>> 
>>> Gentlepeople!
>>> 
>>> This 3GPP SA2 contribution is recently submitted
>>> https://protect2.fireeye.com/v1/url?k=803d870e-dcb428c7-803dc795-0cc47
>>> ad93e1c-77a60175f8460bc6&q=1&e=6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=
>>> 
>> https%3A%2F%2Fwww.3gpp.org%2Fftp%2Ftsg_sa%2FWG2_Arch%2FTSGS2_136
>> _Reno%
>>> 2FDocs%2FS2-1911250.zip Title “5QI values for efficient handling of
>>> congestion control marked IP packets”
>>> The contribution introduces L4S to the SA2 audience and explains where the
>> technology is useful. The contribution also proposes a method to bake L4S into
>> the 5G QoS framework.
>>> If agenda time permits, it will be discussed at the SA WG2 meeting in Reno on
>> Nov 18-22, hopefully with good results.
>>> 
>>> Also a RAN2 contribution was submitted and discussed at an earlier RAN2
>> meeting.
>>> https://protect2.fireeye.com/v1/url?k=06eb4ef7-5a62e13e-06eb0e6c-0cc47
>>> ad93e1c-f387514c9f633599&q=1&e=6cd6c372-b712-46be-8b10-
>> 22955dec7b38&u=
>>> 
>> https%3A%2F%2Fwww.3gpp.org%2Fftp%2FTSG_RAN%2FWG2_RL2%2FTSGR2_1
>> 07bis%2F
>>> Docs%2FR2-1913888.zip
>>> 
>>> Enjoy
>>> /Ingemar
>>> ================================
>>> Ingemar Johansson  M.Sc.
>>> Master Researcher
>>> 
>>> Ericsson Research
>>> Network Protocols & E2E Performance
>>> Labratoriegränd 11
>>> 971 28, Luleå, Sweden
>>> Phone [phone]
>>> SMS/MMS [phone]
>>> [e-mail]
>>> www.ericsson.com
>>> 
>>>      The best way to find out if you
>>>  can trust somebody is to trust them
>>>              Ernest Hemingway
>>> =================================
>