Re: [tsvwg] start of WGLC on L4S drafts

[Sebastian Moeller <[e-mail]>]

Bob,

one point below, [SM4]

> On Oct 14, 2021, at 02:48, Bob Briscoe <[e-mail]> wrote:
> 
> Sebastian, see [BB2] (2 response)
> 
> On 13/10/2021 08:25, Sebastian Moeller wrote:
>> Hi Bob,
>> 
>> more bellow, prefixed [SM2]
>> 
>> 
>>> On Oct 13, 2021, at 00:02, Bob Briscoe <[e-mail]> wrote:
>>> 
>>> Sebastian,
>>> 
>>> Sorry - just noticed I missed this email at the time. See [BB]...
>>> 
>>> On 21/09/2021 21:54, Sebastian Moeller wrote:
>>>> Hi Bob,
>>>> 
>>>> 
>>>> 
>>>>> On Sep 20, 2021, at 18:40, Bob Briscoe <[e-mail]> wrote:
>>>>> 
>>>>> Alex, Gorry, [Sry, just discovered I didn't click end on this]
>>>>> 
>>>>> In "all of a user's applications can shift" the word "can" is important.
>>>>> Similarly, in "The L4S architecture is intended to enable all Internet applications to transition" the word "enable" is important.
>>>>> 
>>>>> Neither statement was meant to imply that all applications would move, or should move, just that they would all be _able_ to.
>>>>> These statements are primarily intended as counterpoints to the commonly held view that low latency comes from priority scheduling (e.g. via differentiated service) so low latency can only be provided for a subset of traffic, because if all packets requested higher priority, there would be no difference to any packet's latency.
>>>> 	[SM] Given that at the core of the L4S reference AQM there sits a (conditional) priority scheduler*, I fail to see how L4S' low latency does not come from priority scheduling? Also in L4S low latency can only be provided to a subset of traffic, namely flows that are well behaved and ECT(1)-marked (given the shared queue design, one bad apple in the L-queue can affect latency/throughput for all flows). I am not sure whether L4S really is contrary to the "commonly held view" here, sure the details differ a bit, but at its core it seems rather similar to traditional techniques for low latency.
>>> [BB] You say 'L4S low latency can *only* be provided to a subset' (my emphasis), but the word 'only' is wrong.
>>> That is the point. L4S low latency /can/ be provided to all traffic.
>> 	[SM2] Well low latency "can" be provided by a dumb FIFO then, as long as the aggregate ingress traffic never exceeds the egress capacity... this is similarly unhelpful as your "can provide".
>> 
>>> Yes, L4S low latency /is/ only provided to flows that are well-behaved and ECT(1) marked, but all traffic /can/ behave smoothly without losing out, and all traffic /can/ use ECT(1) without losing out.
>>> 
>>> The priority scheduler really is not at the core.
>> 	[SM2] My words were "at the core of the L4S reference AQM" and I maintain that that is strictly true, DualQ being:
>> a) the reference AQM for L4S
>> b) the only L4S AQM that has an open reference implementation for Linux
>> c) has seen some modicum of testing by others than the developers
>> 
>> I maintain, that DualQ is at its core a (weighed) priority scheduler.
>> 
>> 
>>> * Consider an L4S FQ scheduler like FQ-CoDel with the ce_threshold parameter applied to ECT(1) traffic. There is no priority scheduling but all traffic can still have very low latency.
>> 	[SM2] Well, show me a) an operational reference implementation of such an AQM and b) sufficient testing data to demonstrate that it works under a broad set of realistic traffic conditions. Sure, I would assume an FQ based AQM will have fewer downsides that DualQ, but I would still want to see data.
> 
> [BB2]  We published all the test data for the ce-threshold variant of FQ_CoDel alongside the DualQ results in July 2019 when I posted this link to this list:
>     https://bobbriscoe.net/pubs.html#DCttH_TR
> The FQ-CoDel tests with ce-threshold enabled for ECT(1) traffic are in the right-hand column of all the pages of plots at the end of the paper.
> The Evaluation section explains the setup of all the tests plotted at the end, and provides commentary on the results.
> 
> This reveals to me that you have never read this material. I think you need to justify on this list why you have not read the primary performance evaluation material about L4S, given everything you have said about L4S performance.
> 
> You will note from Fig 7 that the tail latency of FQ-L4S under load from a mix of short and long flows is pretty good, but not quite as good as the DualQ. That's inevitable because when an approach gives priority to 'new' flows, there will always be runs of 'new' packets that cause packets in the 'old' queues to have to wait longer than they would in a FIFO. Put another way, if you allow some other packet than the first in to be the first out, then you have to delay the first in, which is often counterproductive when the scalable congestion controls are collectively ensuring that nearly all packets are 'new' and any 'old' queues are extremely short anyway.
> 
>> Plus, as long as DualQ is L4S's reference implementation that will be deployed in scale by cable-ISPs it does really matter that there might be better L4S-AQM designs possible, does it?
>>> * Similarly, an L4S DualQ Coupled AQM can work with any scheduler that provides isolation, for instance the Time-Shifted FIFO. Scheduling priority is not 'at the core', 'cos it's not even necessary. We only ended up choosing a priority scheduler because it gave slightly better tail latency.
>> 	[SM2] Well, I am talking about the selected reference AQM for L4S, if you drop the DualQ draft and replace is by  Time-Shifted FIFO I will have a look at that AQM and discuss that. But at the current time DualQ is the relevant AQM, since it is the only one you are currently trying to standardize.
> 
> [BB2] I won't respond any further to everything above this point, because it has just descended into two opinions about what is the core idea of a technology.
> 
> But I will correct two possible misunderstandings about what is being standardized here.
> 
> #1/ DualQ and FQ are equally applicable for L4S. That change was made to the l4s-arch and ecn-l4s-id drafts in Feb 2020. The fact that there is a draft about a DualQ solution and not an FQ draft is partly because the FQ one is so similar to FQ_CoDel that it only needs a short sentence describing the difference.
> 
> Nonetheless, this is not a spectator sport. If you or anyone else on this list wants to prove that an FQ-L4S is better, or as good, or whatever, go ahead and try to repeat or improve on the results given above. A draft about FQ-L4S would also be extremely quick to write - probably less than a page, not including all the top and tail guff.
> 
> #2/ And there is a difference between standardization and implementation:
> The DualQ Coupled AQM draft does not (experimentally) standardize a WRR scheduler. Everything in the appendix about DualPI2 (and the other appendix about Curvy RED) is informative. The aqm-dualq-coupled draft experimentally standardizes a framework for coupling AQMs across two queues (the first line of the introduction says this). The words chosen to be capitalized (i.e. normative) have been carefully selected to be as concrete as possible without limiting choice.
> 
> The only normative statements about the scheduler are:
> "The scheduler draining the two queues MUST give L4S packets priority over Classic, although priority MUST be bounded in order not to starve Classic traffic. The scheduler SHOULD be work-conserving."
> This is deliberately written to apply to the TS-FIFO, and all sorts of other possible schedulers.

	[SM4] From https://bobbriscoe.net/projects/latency/dctth_journal_draft20190726.pdf:
"We have also tried what we call a Time-Shifted FIFO scheduler [37]. It selects the packet with the earliest arrival timestamp, after subtracting a constant time-shift to favour L4S packets. It performs nearly as well as WRR despite its simplicity. However, it tends to allow long bursts of delay to leak from the C to the L queue, so it is not used further in this paper."

	In other words, TS-FIFO is even worse at meeting the promises in the L4S RFCs than DualQ, and was hence "abandoned*" (and not fully characterized in the paper), and yet you bring it up here as example for what exactly? IMHO this demonstrates that the K4S drafts are way too broad, instead on focussing on actual solutions to the stated challenge they offer a multitude of not-good-enough attempts. How about doing one thing/proposing one solution that works well instead?


Sebastian


*) There is a whole ensemble of solutions to L4S existing problems that where explored for some time and then abandoned without having reached a satisfying solution:

a) rfc3168 AQM detection
b) paced chirping
c) RTT de-biasing
d) L4S AQMs that actually meet the promises in the drafts

impressive how long this effort is still maintained in the light of constant failure of its design and implementation to meet its promises.


> 
> Regards
> 
> 
> 
> Bob
> 
> 
>> 
>>> The real core of L4S is scalable congestion control. That is the difference that allows even capacity-seeking traffic to also maintain consistently very low latency. It's easier to see in the L4S FQ example, that it's what keeps a flow's latency down. Not the scheduler.
>> 	[SM2] I would have assumed, if that was the core that the L4S effort would have started with a draft for such a scalable congestion controller, instead TCP Prague came as an after thought relatively late in the game...
>> 
>> 
>>> If TCP had been designed with a scalable congestion control from the start, we would have had L4S with no need for a scheduler at all. The scheduler is primarily a transition mechanism, to isolate from Classic traffic. But no traffic /needs/ to be Classic, it's just how "stuff happened".
>> 	[SM2] Realistically, a considerable fraction of internet traffic is going t stay on classic congestion controller for a long time, so I fail to see how this is relevant for the near and intermediate future. Especially given the demonstrated failure modes of TCP Prague, where realistic traffic patterns result in abysmal throughput...
>> 
>> 
>> Regards
>> 	Sebastian
>> 
>>> 
>>> 
>>> Bob
>>> 
>>>> Regards
>>>> 	Sebastian
>>>> 
>>>> 
>>>> *) https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-19:
>>>> "In this case it would not be appropriate to call the queue an L4S
>>>>    queue, because it is shared by L4S and non-L4S traffic.  Instead it
>>>>    will be called the low latency or L queue.  The L queue then offers
>>>>    two different treatments:
>>>> 
>>>>    o  The L4S treatment, which is a combination of the L4S AQM treatment
>>>>       and a priority scheduling treatment;
>>>> 
>>>>    o  The low latency treatment, which is solely the priority scheduling
>>>>       treatment, without ECN-marking by the AQM."
>>>> 
>>>> 
>>>> https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-l4s-arch-10:
>>>> "The tension between prioritizing L4S and coupling the
>>>>        marking from the Classic AQM results in approximate per-flow
>>>>        fairness.  To protect against unresponsive traffic in the L4S
>>>>        queue taking advantage of the prioritization and starving the
>>>>        Classic queue, it is advisable not to use strict priority, but
>>>>        instead to use a weighted scheduler (see Appendix A of [I-D.ietf-tsvwg-aqm-dualq-coupled])."
>>>> 
>>>> https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-aqm-dualq-coupled
>>>> "Classic traffic needs to build a large queue to prevent under-
>>>>    utilization.  Therefore a separate queue is provided for L4S traffic,
>>>>    and it is scheduled with priority over the Classic queue.  Priority
>>>>    is conditional to prevent starvation of Classic traffic."
>>>> 
>>>> I ignore the yarn spun in those drafts, why this prioritization would only result in latency but not longer-time-scale throughput prioritization, because Pete's data show that for quite a number of realistic traffic patterns DualQ fails to arbitrate capacity equitably between the two queues.
>>>> 
>>>> 
>>>> 
>>>>> I've made a note to myself to make it clear that this is not an intention that all applications will transition, rather it is an intention not to prevent any application from transitioning.
>>>>> 
>>>>> 
>>>>> Bob
>>>>> 
>>>>> On 02/09/2021 19:26, [e-mail] wrote:
>>>>>> Hi Gorry, tsvwg;
>>>>>> 
>>>>>> 
>>>>>> See inline [AB]
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Thursday, August 12, 2021, 11:23:49 AM GMT+1, Gorry Fairhurst<[e-mail]>  wrote:
>>>>>> 
>>>>>> ---
>>>>>> 1b) Issue: I’d also quibble with: “so that all of a user's applications
>>>>>> can shift to it when their stack is updated.“ - again, is the word “all”
>>>>>> really necessary or correct here?
>>>>>> ---
>>>>>> 
>>>>>> [AB]
>>>>>> Not  answering your point, but related:
>>>>>> 
>>>>>> L4s-arch has at the very start the following: “The L4S architecture is intended to enable   _all_ Internet applications to transition away from congestion control algorithms that cause queuing delay, to a new class of  congestion controls that induce very little queuing, aided by  explicit congestion signaling from the network. “
>>>>>> 
>>>>>> 
>>>>>>  There are two quite different end states here:
>>>>>> 
>>>>>> * An internet in which there are, end-to-end, two classes of traffic on an ongoing basis.
>>>>>> 
>>>>>> * An internet in which nearly all traffic has transitioned to L4S, remaining non-L4S traffic being supported on a backwards-compatibility basis
>>>>>> 
>>>>>> As I understand it, if L4S is deployed, either of these could be possible end states. At present it is not clear that all use cases could migrate (eg, long RTT traffic)
>>>>>> 
>>>>>> 
>>>>>> It seems like it would be a good idea for the draft to make clear:
>>>>>> * That either of these end states are possible
>>>>>> * What the formal position of the WG would be if the drafts are accepted, IE what decision has been made (or left for later) regarding the desired end state.
>>>>>> 
>>>>>> 
>>>>>> regards,
>>>>>> 
>>>>>> Alex
>>>>>> 
>>>>> -- 
>>>>> ________________________________________________________________
>>>>> Bob Briscoehttp://bobbriscoe.net/
>>>>> 
>>> -- 
>>> ________________________________________________________________
>>> Bob Briscoe                               http://bobbriscoe.net/
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
>