Re: [tsvwg] new tests of L4S RTT fairness and intra-flow latency

[Sebastian Moeller <[e-mail]>]

Hi David,

thanks for your thoughtful response. Some comments/questions below, prefixed with [SM]


> On Nov 15, 2020, at 02:26, Black, David <[e-mail]> wrote:
> 
> Hi Sebastian,
> 
> Some comments as an individual, not a WG chair.
> 
> First, I think Pete has pretty clearly established that TCP Prague is research, and hence (IMHO) TCP Prague ought to be headed for ICCRG as its primary forum rather than TSVWG.

	[SM] I fully agree, TCP Prague would be helped considerably by being discussed at the proper WG. To me it is clear that the safety of the L4S elements to be "ratified" by this WG can hardly made dependent on a protocol under active research and development, since at the current time it is still an open question if the TCP Prague claims and aims can realistically be reached; the current TCP Prague falls clearly short of its high aspirations.

> 
> To date, the 'Prague L4S Requirements' (Appendix A of draft-ietf-tsvwg-ecn-l4s-id) have been strongly associated with TCP Prague.  

	[SM] Yes, because so far no other protocol has been aiming for fulfilling the "Prague requirements". I asked both the SCReAM developer and the QUIC maintainer explicitly about that, and Christian ignored the question completely (indicating to me that he takes neither me and/or the Prague requirements seriously), Ingemar (rightly) argued to look at that after L4S is ratified and SCReAM gains official L4S support. 
	That is somewhat a sad state of affairs, because the Prague requirements read a bit like protocol fan-fiction where quite a number of theoretically desirable properties have been listed with little consideration, whether the magnitude and combination of wished for properties is actually realistically implementable. And a bit more discussion of those requirements with actual real-life protocol designers and implementers might have helped to keep the scope realistic, no?


> That association ought to be teased apart so that the resulting L4S scalable congestion control requirements provide a reasonable design space that can include a number of other congestion control designs - in addition to what's been discussed, e.g., SCReAM, it would be useful to better understand what it would take for implementations of protocols such as DCTCP and BBR (e.g., for QUIC) to meet those requirements.

	[SM] +1; yes. Now, how realistic is such a teasing apart and potential redefinition of the protocol parts of L4S after the network parts as currently designed an implemented have been deployed, based on an experimental track RFC? I remember Koen admitting that team L4S's interests in protocols is only marginal, which does not fill me with confidence that their scheme is going to ever work well. I base this on the theoretical observation that communication typically has senders, receivers and signals, and trying to design a meaningful new communication channel by focussing on the signal part alone while mostly ignoring the sender and receiver part seems overly optimistic.


> 
> My overall take on the requirements is that in 20/20 hindsight, some of them were overly optimistic, and hence need to be backed off/toned down/broadened to encompass what is reasonable in "running code" well beyond TCP Prague.

	+1; as I have argued (too) forcefully in the past, especially the RTT de-bias requirement as originally written seems to be impossible to tackle, as long as time flows unidirectionally. But the last proposal for changing that aims considerably too low and does violate the spirit and intention of the original requirement.


> That sort of collision between interesting ideas and network realities is not an unheard-of scenario in IETF, so I hesitate to view the need for changes to these requirements as evidence that the original ideas were inherently defective, as I've seen far more dramatic changes, e.g., some number of years ago, the first design of iSCSI login was elegant ... and resulted in implementations that did not interoperate, resulting in a complete redesign.

	+1; this seems quite realistic. I see two direct consequences from that line of though:

A) If L4S ratification is to happen now, the safety requirements one expects from new network features with to be expected side-effects on other traffic need to be managed and controlled within the new network features and should not be out-sourced onto to-be-done-sometime-in-the-future protocol work (especially since that might look quite different than envisioned right now). IMHO especially dualQ is not finished in that regard and should not be considered "running code", sure it runs, but it does not do what the drafts promise nor what seems necessary under your observation that the Prague requirements are still quite fluid.

B) This WG should come up with an actionable plan on how to assess L4S safety and success and a pre-determined procedure how to undo the L4S experiment entirely (including re-gaining the ECT(1) codepoint for other uses) in case that the current design and implementation of the network features proves to not yield the desired outcome once actually used protocols actually attempt to meet the Prague requirements.


	Now, what I believe is that this WG seems to actually go for option C) of not requiring the network features to be safe by design and not requiring a clear exit-strategy in case of failure. I guess the best I can hope for is that in the course of the rfc4858 process my objections will be reported as extreme discontent under clause 3.1. (1.f).

Best Regards
        Sebastian

P.S.: To head off that argument before it is being repeated by team L4S, if the current state is the best under the self-imposed design constraints for the L4S AQM, then it clearly is time to change those constraints. If I can not participate safely in normal traffic with my hand tied behind my back, maybe I need to untie my hands instead of arguing for laxer requirements to account for my self-imposed handicap?


> 
> Thanks, --David
> 
>> -----Original Message-----
>> From: tsvwg <[e-mail]> On Behalf Of Sebastian Moeller
>> Sent: Thursday, November 12, 2020 6:11 PM
>> To: Pete Heist
>> Cc: tsvwg IETF list
>> Subject: Re: [tsvwg] new tests of L4S RTT fairness and intra-flow latency
>> 
>> 
>> [EXTERNAL EMAIL]
>> 
>> Hi Pete,
>> 
>> great data. IMHO Especially the fact that short-RTT Prague will severely
>> outcompete long-RTT Prague way more than the traditional dumb FIFO will,
>> strongly supports  the following hypotheses I have been posting before:
>> 
>> a) Dualpi2/TCP Prague are no way ready for deployment, but are basically still in
>> the toy stage
>> 
>> b) all that L4S will effectively do, be it by intent or simply by mis-design, is built a
>> "fast lane" for short RTT low hop count traffic. Given all the hype (still!) in the L4S
>> internet drafts about how this is the future of the internet... Also a fast lane that
>> requires active cooperation from the leaf ISP (if they keep their bottlenecks at
>> FIFO, I see no compelling reason for TCP Prague at all), which will require SLAs
>> between CDNs and ISPs.
>> 
>> c) too little, too late. Really only modest gains in RTT (compared to best of class
>> AQMs like fq_codel or cake) and noticeable regressions in sharing behavior both
>> between different CCs and flows of different RTTs (and that compared to dumb
>> queue "management" with a simple FIFO).
>> 
>> 
>> 
>> Also quite sobering, that it AGAIN was not team L4S bringing real data to the table
>> to support their claims and promises. Then again looking at the RTT fairness for
>> Prague versus Prague under DualQ, I can understand why team L4S stayed
>> mumm, and rather argued for allowing unfettered self-mutilation of L4S
>> compliant transport protocols in regards to RTT fairness:
>> 
>> "So there is no need  to mandate that an L4S implementer does no harm to
>> themselves, which window-based congestion controls tend to do at higher RTT.
>> Of course, this doesn't preclude implementers reducing or eliminating RTT bias
>> for larger than typical RTTs, but it removes any requirement to do so."
>> 
>> After years of advertising on increased RTT independence (in spite of the data
>> showing that the proposed combination of DualQ and TCP Prague actually
>> increases RTT bias), team L4S in the last minute no less, decides to do a 180 and
>> just change the requirements to allow for the rather unhealthy behavior
>> demonstrated for L4S...
>> With the current enhanced RTT bias, who in their right mind is going to use TCP
>> Prague; which currently is the only transport, that at least attempted to tackle all
>> the "requirements" L4S poses for transports that want to use the ECT(1) express
>> way? Then again, since none of the network elements are designed and required
>> to actually check and enforce these requirements, calling these "requirements" is
>> a bit of stretch anyway, maybe they should be changed from MUST requirements
>> to COULD musings....
>> 
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>>> On Nov 12, 2020, at 21:31, Pete Heist <[e-mail]> wrote:
>>> 
>>> Hi,
>>> 
>>> We have posted some new tests of L4S here:
>>> 
>>> https://github.com/heistp/l4s-tests
>>> 
>>> These tests cover two-flow fairness in several qdiscs with different
>>> path RTTs per-flow, and transient behavior in fq_codel queues.
>>> 
>>> The results raise some concerns about a bias in favor of L4S flows in
>>> DualPI2 queues, throughput imbalances between flows with different
>>> path RTTs, and intra-flow latency spikes upon rate reductions in fq_codel.
>>> The repo above contains a walk-through of the key findings, and links
>>> to more results in the Appendix.
>>> 
>>> Regards,
>>> Pete
>>> 
>>> 
>