Re: [tcpm] [tsvwg] L4S related activity in 3GPP

[Sebastian Moeller <[e-mail]>]

Hi Ingemar,


> On Nov 10, 2019, at 11:20, Ingemar Johansson S <[e-mail]> wrote:
> 
> Hi Sebastian
> Please see inline
> 
> /Ingemar
> 
>> -----Original Message-----
>> From: Sebastian Moeller <[e-mail]>
>> Sent: den 9 november 2019 21:38
>> To: Ingemar Johansson S <[e-mail]>
>> Cc: Ingemar Johansson S
>> <ingemar.s.johansson=[e-mail]>; [e-mail];
>> [e-mail]; [e-mail]; [e-mail]; Bob Briscoe
>> ([e-mail]) <[e-mail]>
>> Subject: Re: [tsvwg] L4S related activity in 3GPP
>> 
>> Dear Ingemar,
>> 
>> more in-line below.
>> 
>>> On Nov 9, 2019, at 20:24, Ingemar Johansson S
>> <[e-mail]> wrote:
>>> 
>>> Hi Sebastian
>>> 
>>> Please see inline below
>>> /Ingemar
>>> 
>>>> -----Original Message-----
>>>> From: Sebastian Moeller <[e-mail]>
>>>> Sent: den 9 november 2019 18:13
>>>> To: Ingemar Johansson S
>>>> <ingemar.s.johansson=[e-mail]>
>>>> Cc: [e-mail]; [e-mail]; [e-mail]; Ingemar Johansson
>>>> S <[e-mail]>; [e-mail]
>>>> Subject: Re: [tsvwg] L4S related activity in 3GPP
>>>> 
>>>> Dear Ingemar,
>>>> 
>>>> thanks for posting this. I read "This document describes an Active
>>>> Queue Management mechanism, ‘L4S’, that can support low-latency
>>>> services in a way that does not degrade the performance of other
>>>> traffic." and I wonder whether 3GPP has run any experiments to
>>>> confirm/support this claim, that you are willing to share?
>>> 
>>> [IJ] We have done simulation studies with our proprietary 4G/5G system
>> simulation tools but I can not disclose the results from these.
>> 
>> 	[SM] Ah, I see, not posting internal data is fine, but by all means, go and
>> perform real-world measurements as well with all the parts you actually want to
>> roll-out (in other words testing with dctcp for example is scientifically
>> interesting, unless you expect your users to actually use dctcp on your links).
> 
> [IJ] I hope to have data to show later on

	[SM] Excellent, looking forward to that.

> 
>> 
>> 
>>>> 
>>>> Sidenote: in the powerpoint file it says about L4s " Standardized in
>>>> IETF [1]", which seems premature, in the process of being
>>>> standardized seems a better description of the status quo...
>>> 
>>> [IJ] Yes, you are right about this, it should have been "Subject to
>> standardization..."
>> 
>> 	[SM] Fair enough, as much as I regret to say it, standardization is quite
>> likely.
>> 
>>> 
>>>> 
>>>> 
>>>> I ask about test data from your side as the recent test data is quite
>>>> sobering from a number of angles, including quality of isolation:
>>> 
>>> [IJ] First of all the outline of the proposal for L4S in 5G is to use separate radio
>> bearers for L4S and classic (or normal or non-L4S) traffic. This means that it is
>> the 5G QoS framework and the scheduling in the RAN (Radio Access Network)
>> that dictates the fairness.
>> 
>> 	[SM] Sounds like a workable approach, that side steps the dualQ
>> implementation/theory issues.
> 
> [IJ] The proposed approach for 5G is for practical reasons, we believe that it is more implementation and standardization friendly as it will utilize the existing QoS framework as much as possible. 

	[SM] Well, what ever the rationale, by avoiding the insufficiently isolating DualQ AQM you will save yourself from a number of headaches; I simply assume that you implement stricter airtime sharing between the two bearers than dualQ, and if you err you err on the side of conserving normal traffic (unlike DualQ's "L4S first" approach). But you still are saddled with L4S poor choice of ECT(1) as heuristic to separate L4S from non-L4S traffic. Calling that aETC(1) classifier seems plain wrong, as classifier implies strict separation.


Best Regards
	Sebastian


> 
>> 
>> 
>>> But of course there are wireline transport link in the core network that may be
>> using DualQ when that becomes commercially available. The bitrates over these
>> links are however likely to be considerably higher than 50Mbps.
>> 
>> 	That does not matter, the issue is not the limit of 50 Mbps, but the fact
>> that dualQ, at last the current implementation in conjunction with the current
>> TCP prague implementation, simply do not reliably and robustly do what the rfc
>> draft claims. My fear is that dualQ, with its weak coupling vie marking
>> probabilities, simply is a design, that only guarantees approximate fairness. In
>> other words I doubt that dualQ works as a naive reading of the RFC implies. But
>> if I  understood Bob correct, the only gurantee he is willing to give is L4S does
>> not starve non-L4S traffic. Which IMHO is not sufficient to merit inflicting the
>> L4S experiment on the wider internet. Now, that is not my call to make, but the
>> IETF process allows me t voice my concerns, and I just use this opportunity in an
>> attempt to minimize undesired side-effects on the "normal" internet.
>> 
>>> 
>>>> 
>>>> 	Looking at the current L4S tests from the various testbeds I see
>>>> severe breakdown of the promised isolation between bulk and the
>>>> "low-latency" queue, apparently making the current implementation, if
>>>> not the concept itself IMHO unfit for release into the wider
>>>> internet, before these issues have been sufficiently addressed.
>>>> Especially the observation that the weighted scheduler built in as a
>>>> safety mechanism in the dualQ aqm seems to trigger considerably more
>>>> often than expected/predicted from reading the dualQ draft (giving
>>>> 1/16 of the bandwidth to on of two classes _might_ be justified as a
>>>> temporary stop- gap measure in extremely unlikely cases, but this
>>>> already triggers in a simple one-flow-per queue scenario at low RTTs). In
>> short it currently looks like the "does not degrade performance of other traffic"
>> part is not yet finished.
>>>> 
>>>> 	To illustrate my criticism, here is a link to a test comparing the
>>>> simplest functionality test imaginable, running one L4S-style and one
>>>> non-L4S-style TCP connection through the L4S dual queue AQM at a
>>>> intranet like RTT (nomnally 0ms, but in reality):
>>>> https://protect2.fireeye.com/v1/url?k=fcd5c73d-a05c68f4-fcd587a6-
>>>> 0cc47ad93e1c-6acfac650abe7f09&q=1&e=6cd6c372-b712-46be-8b10-
>>>> 22955dec7b38&u=https%3A%2F%2Fl4s.cablelabs.com%2Fl4s-
>>>> testing%2Fkey_plots%2Fbatch-l4s-s1-2-cubic-vs-prague-50Mbit-0ms_var.p
>>>> ng As can be seen the L4S flow (labeled prague) ends up very quickly
>>>> taking ~90% of the available bandwidth, leaving only 10% to the
>>>> non-L4S flow (labeled cubic), wit increasing RTT this bias gets less
>>>> extreme, but even at 80ms the prague flow still gets around 1.5 times
>>>> the bandwidth of the cubic flow
>>>> (https://protect2.fireeye.com/v1/url?k=77272802-2bae87cb-77276899-
>>>> 0cc47ad93e1c-56d0590db27309ef&q=1&e=6cd6c372-b712-46be-8b10-
>>>> 22955dec7b38&u=https%3A%2F%2Fl4s.cablelabs.com%2Fl4s-
>>>> testing%2Fkey_plots%2Fbatch-l4s-s1-2-cubic-vs-prague-50Mbit-80ms_var.
>>>> png), indicating an RTT dependent performance degradation for "other
>>>> traffic". Test from another testbed confirm the break-down of
>>>> fairness between L4S and non- L4S traffic at short RTTs
>>>> (https://protect2.fireeye.com/v1/url?k=5710f0ef-
>>>> 0b995f26-5710b074-0cc47ad93e1c-418920d155a62bf0&q=1&e=6cd6c372-
>>>> b712-46be-8b10-
>>>> 22955dec7b38&u=https%3A%2F%2Fwww.heistp.net%2Fdownloads%2Fsce-
>> l4s-
>>>> bakeoff%2Fbakeoff-2019-09-13T045427-r1%2Fl4s-s1-2%2Fbatch-l4s-s1-2-
>>>> cubic-vs-prague-50Mbit-0ms_fixed.png).
>>> 
>>> [IJ] I understand that the 0 RTT results can be because the congestion window
>> is only a few segments large (my guess is ~3 to 4 segments ?). I know that this is
>> a problem with e.g. DCTCP.  The question is whether this is a permanent
>> problem or something that will be solved over time?.
>> 
>> 	[SM] With DualQ, I assume it is a structural problem, instead of
>> designing this as a robust system with two fair sharing queues it does its clever
>> "indirect coupling" via marking/dropping probabilities. Have a look at
>> https://protect2.fireeye.com/v1/url?k=53bd001b-0f37caa5-53bd4080-
>> 869a17b5b21b-e4edb2934547a534&q=1&e=314d9ef9-5cc0-4c98-a599-
>> 59f030e6cb2f&u=https%3A%2F%2Fwww.heistp.net%2Fdownloads%2Fsce-l4s-
>> bakeoff%2Fbakeoff-2019-09-13T045427-r1%2Fl4s-s2-2%2Fbatch-l4s-s2-2-
>> cubic-vs-prague-50Mbit-10ms_fixed.png for how sharing in a FQ system looks.
>> Even with the leaky ECT(1) as classifier abuse it shoukd be simple to fair queue
>> between EXT(1) and not ECH(1) flows and have robust separation. This is not
>> rocket science, and mostly, as far as I can tell a solved problem.
>> 
>>> What is more interesting to me and what makes L4S attractive for the use
>> cases outlined in the 3GPP contributions, is the very low queue delay with L4S.
>> 
>> 	[SM] Well, I believe low queueing delay can be achieved without dualQ
>> and without abusing the ECT(1) codepoint as a classifier (the desired classifier
>> requires one complete independent bit, ECT(1) is just 1/2 of a bit, and hence has
>> failure modes in separating L4S-style flows and non-L4S flows). The idea of using
>> the finer-grained congestion signaling of DCTCP over the wider internet seems
>> sane see the SCE draft proposal as an alternative that would/will allow dctcp-
>> style ECN signaling over the internet in a truly backward compatible fashion...
>> 
>> 
>>> There are other promising features with L4S. One being that it can enable
>> congestion control algorithms to quickly grab free capacity.
>> 
>> 	[SM] When you say L4S, do you mean the congestion signaling? Then
>> https://tools.ietf.org/html/draft-morton-taht-sce-00 should have you covered,
>> without the nasty side-effects of the current L4S proposal. Or do you mean the
>> actual congestion controller as implemented in TCP prague?
>> 
>> 
>>> 5G access can have large variations in link throughput for example related to
>> dual connectivity and L4S can be helpful here as well.
>> 
>> 	[SM] Dealing better with variable rate links is certainly desirable, but I
>> am not sure that DualQ/TCP Prague really are the best or only tools for that
>> purpose. At least not until all of the kinks are ironed out. Again, none of this is
>> my call to make, so all I am voicing is my educated subjective assessments.
>> 
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>>> 
>>>> 	Personally, I can not help thinking that it might be better to first
>>>> make sure there is a complete, real-world tested implementation that
>>>> reliably demonstrates that dualQ delivers the required isolation
>>>> between the two traffic classes and that the L4S approach actually
>>>> delivers the reported reliable low latency performance under adverse
>>>> real-world conditions while not degrading the performance of other traffic,
>> before trying to codify L4S into RFCs.
>>>> 
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>> 
>>>>> On Nov 9, 2019, at 17:00, Ingemar Johansson S
>>>> <ingemar.s.johansson=[e-mail]> wrote:
>>>>> 
>>>>> Gentlepeople!
>>>>> 
>>>>> This 3GPP SA2 contribution is recently submitted
>>>>> https://protect2.fireeye.com/v1/url?k=803d870e-dcb428c7-803dc795-0cc
>>>>> 47
>>>>> ad93e1c-77a60175f8460bc6&q=1&e=6cd6c372-b712-46be-8b10-
>>>> 22955dec7b38&u=
>>>>> 
>>>> 
>> https%3A%2F%2Fwww.3gpp.org%2Fftp%2Ftsg_sa%2FWG2_Arch%2FTSGS2_136
>>>> _Reno%
>>>>> 2FDocs%2FS2-1911250.zip Title “5QI values for efficient handling of
>>>>> congestion control marked IP packets”
>>>>> The contribution introduces L4S to the SA2 audience and explains
>>>>> where the
>>>> technology is useful. The contribution also proposes a method to bake
>>>> L4S into the 5G QoS framework.
>>>>> If agenda time permits, it will be discussed at the SA WG2 meeting
>>>>> in Reno on
>>>> Nov 18-22, hopefully with good results.
>>>>> 
>>>>> Also a RAN2 contribution was submitted and discussed at an earlier
>>>>> RAN2
>>>> meeting.
>>>>> https://protect2.fireeye.com/v1/url?k=06eb4ef7-5a62e13e-06eb0e6c-0cc
>>>>> 47
>>>>> ad93e1c-f387514c9f633599&q=1&e=6cd6c372-b712-46be-8b10-
>>>> 22955dec7b38&u=
>>>>> 
>>>> 
>> https%3A%2F%2Fwww.3gpp.org%2Fftp%2FTSG_RAN%2FWG2_RL2%2FTSGR2_1
>>>> 07bis%2F
>>>>> Docs%2FR2-1913888.zip
>>>>> 
>>>>> Enjoy
>>>>> /Ingemar
>>>>> ================================
>>>>> Ingemar Johansson  M.Sc.
>>>>> Master Researcher
>>>>> 
>>>>> Ericsson Research
>>>>> Network Protocols & E2E Performance
>>>>> Labratoriegränd 11
>>>>> 971 28, Luleå, Sweden
>>>>> Phone [phone]
>>>>> SMS/MMS [phone]
>>>>> [e-mail]
>>>>> www.ericsson.com
>>>>> 
>>>>>     The best way to find out if you  can trust somebody is to trust
>>>>> them
>>>>>             Ernest Hemingway
>>>>> =================================