Re: [tsvwg] FW: New Version Notification for draft-white-tsvwg-l4sops-00.txt

[Greg White <[e-mail]>]

Apologies in advance for the mangled URLs in the quoted text below.  My IT department has implemented a system that mangles URLs on inbound email.

From: Sebastian Moeller <[e-mail]>
Date: Thursday, July 30, 2020 at 3:43 PM
To: Greg White <[e-mail]>
Cc: "[e-mail]" <[e-mail]>
Subject: Re: [tsvwg] FW: New Version Notification for draft-white-tsvwg-l4sops-00.txt


Hi Greg,



my comments for sections 1 & 2 below (more will follow):



"1.  Introduction



   In the majority of network paths, including paths where the

   bottleneck link utilizes packet drops (either due to buffer overrun

   or active queue management) in response to congestion, as well as

   paths that implement a 'flow-queuing' scheduler such as fq_codel or

   Cobalt, and those that implement dual-Q-coupled AQM, L4S traffic

   coexists well with classic congestion controlled traffic."



[SM] This description ignores the birthday paradox issue that Pete Heist demonstrated. A misbehaving L4S will cause less havok there but it is not without side-effects on stochastic flow queueing systems, we might as well acknowledge that, no?

[GW] IIRC that issue can occur in fq_codel, but is much less likely in Cobalt (due to set-associative hashing).  Is that correct?



"   On network paths where the bottleneck link implements a shared-queue

   (FIFO) with an Active Queue Management algorithm that provides

   Explicit Congestion Notification signaling according to RFC3168, it

   has been demonstrated that when a set of long-running flows

   comprising both "Classic" congestion controlled flows and L4S-

   compliant congestion controlled flows compete for bandwidth, the

   classic congestion controlled flows may achieve lower throughput when

   compared to the L4S congestion controlled flows.  This 'unfairness'

   between the two classes appears to be more pronounced on longer RTT

   paths (e.g. 50ms and above) and/or at higher link rates (e.g. 50 Mbps

   and above)."



This is rather cautious I would use more drastic terms, the observed unfairness approaches starvation of the non-L4S flows and this text makes it sound like it is a minor almost theoretical concern.



[GW] We can definitely wordsmith the text, but we need to be sure that we are characterizing the situation accurately and not trying to portray things in drastic terms (or glossy ones) unless it is warranted.  My belief is that it isn’t warranted to use drastic terms here.  I think it is a fairly widely held view that single queue RFC3168 bottlenecks are uncommon at best and the majority of the ones that do exist are likely to be RED AQMs.  The testing with RED AQMs showed approximate fairness in most cases (though more testing would be welcome).  We’ll need to get a sense from the WG as to how to characterize the situation, given what we know (and what we don’t know).



"  The root cause of this unfairness is that RFC3168 does not

   differentiate between packets marked ECT0 (used by classic senders)

   and those marked ECT1 (used by L4S senders), and provides an

   identical congestion signal (CE marks) to both classes, whereas the

   two classes respond differently to that congestion signal."



How about keeping causality intact and frame this as a consequence of L4S redefining what CE means. There are reasons for doing that, but this text leaves it unclear who caused this problem (or rather the text implicates rfc3168, which IMHO violates temporal causality).



[GW] Point taken.  I’ll try to rearrange that.



"The result is that the

   classic senders respond to the CE marks provided by the bottleneck by

   yielding capacity to the L4S flows.  While this has not been

   demonstrated to cause starvation of the classic flows, the resulting

   rate imbalance can be a cause of concern."



Mmmh, https://protection.greathorn.com/services/v2/lookupUrl/43da5eeb-7a88-4655-a9ba-bec785235113/327/acff24c9401e4e78e9981cf8beb4d98870d4f8bf?domain=sce.dnsmgr.net&path=/results/ect1-2020-04-23-final/l4s-s2-twoflow/l4s-s2-twoflow-ns-cubic-vs-prague-codel1q_20ms_-160ms_tcp_delivery_with_rtt.svg

pretty much demonstrated starvation and that is with TCP-Prague with rfc3168 detection.





[GW] Ok, I will update the text to be more clear.   It seems that it can be demonstrated in contrived setups that it is possible to cause starvation.  That condition was a single queue CoDel (do these exist in the wild?) with heavily modified settings. CoDel with default settings did not show unfairness.





"2.  Per-Flow Fairness



   There are a number of factors that influence the relative rates

   achieved by a set of congestion controlled flows sharing a queue in a

   bottleneck link.



   TODO: discuss startup & convergence times, short flows, RTT-

   unfairness, differences in deployed CC algorithms, etc.



   TODO: also mention that flow sharding is commonplace, so per-flow

   fairness does not imply per-application fairness"





This obviously needs flashing out before one can meaningfully comment, but regarding the last section, you realize that application not the relevant classification for intermediate nodes? I would assume that for an ISP a (potentially weighted) per-end-host fairness would be the exact tool to avoid sharding as work-around... This is obviously not a new idea or comment, so if you opt for elaborating on fairness, please also include known remedies. Or better avoid that discussion here altogether.



[GW] Thanks. Yes per-end-host fairness could be useful in some situations.  AFAIK most residential ISPs implement some form of per-customer fairness, which could be argued is more appropriate in that context. I guess the point is that per-flow fairness isn’t always (or even usually) the most important metric for most users/systems.  It seems important to discuss this, seeing as per-flow unfairness is the main concern that has been raised, but I realize this section could be challenging to write in way that all will agree with.



Best Regards

  Sebastian





> On Jul 30, 2020, at 13:05, Greg White  wrote:

>

> TSVWG members-

>

> I've posted a rough draft of Operational Guidance for L4S deployment.  This is not much more than an outline at this point, and is almost certainly incomplete even at that, so please read it with that in mind.

>

> -Greg

>

>

> From: "[e-mail]"

> Date: Thursday, July 30, 2020 at 4:53 AM

> To: Greg White

> Subject: New Version Notification for draft-white-tsvwg-l4sops-00.txt

>

> A new version of I-D, draft-white-tsvwg-l4sops-00.txt

> has been successfully submitted by Greg White and posted to the

> IETF repository.

>

> Name:          draft-white-tsvwg-l4sops

> Revision:      00

> Title:         Operational Guidance for Deployment of L4S in the Internet

> Document date: 2020-07-30

> Group:         Individual Submission

> Pages:         7

> URL:            https://protection.greathorn.com/services/v2/lookupUrl/f7c346c8-8443-44e5-b71e-435217709fda/327/acff24c9401e4e78e9981cf8beb4d98870d4f8bf?domain=www.ietf.org&path=/internet-drafts/draft-white-tsvwg-l4sops-00.txt

> Status:         https://protection.greathorn.com/services/v2/lookupUrl/e5e7af28-9419-4d0e-a559-650861096bf9/327/acff24c9401e4e78e9981cf8beb4d98870d4f8bf?domain=datatracker.ietf.org&path=/doc/draft-white-tsvwg-l4sops/

> Htmlized:       https://protection.greathorn.com/services/v2/lookupUrl/3b91a498-7692-49a5-aeb2-601ef0ba245a/327/acff24c9401e4e78e9981cf8beb4d98870d4f8bf?domain=tools.ietf.org&path=/html/draft-white-tsvwg-l4sops-00

> Htmlized:       https://protection.greathorn.com/services/v2/lookupUrl/813cf103-4e26-44a1-8c2f-a2648d5dacc0/327/acff24c9401e4e78e9981cf8beb4d98870d4f8bf?domain=datatracker.ietf.org&path=/doc/html/draft-white-tsvwg-l4sops

>

>

> Abstract:

>   This is an early, work-in-progress draft - a start at getting some of

>   the ideas from the mailing list and email exchanges on paper.

>

>   This draft is intended to provide guidance to operators of end-

>   systems, operators of networks, and researchers in order to ensure

>   reasonable fairness between L4S and Classic flows sharing a single-

>   queue RFC3168 bottleneck link.  This draft identifies opportunites to

>   prevent and/or detect and resolve fairness problems in such networks.

>

>

>

>

> Please note that it may take a couple of minutes from the time of submission

> until the htmlized version and diff are available at tools.ietf.org.

>

> The IETF Secretariat

>

>

>