Re: [tsvwg] L4S drafts: Next Steps

[Sebastian Moeller <[e-mail]>]

Hi Jason,

The answer to your question, 'why is it not time for declaring L4S experimental' at the current time is quite simple.
Team L4S has not even scratched the surface on safety and functionality tests that can be performed over the internet or at least participating ASs. Really all requests for data demonstrating functionality over long RTT paths have just been ignored. And basically all tests by other parties demonstrated L4S failing to reach its promises, all the while making imperfect situations considerably worse. 


Just look at Pete's data at https://github.com/heistp/l4s-tests, these are "simple" lab tests that demonstrate that:

L4S's network bits: the dual queue coupled AQM fails to equitable share between the LL and non-LL queue at the same RTT (see second pair of bars under:
687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f32306d735f32306d732e737667 <https://camo.githubusercontent.com/a83b3e8c3aa1824e4ff38b5d0d9136079f431c2e917bb347c15b8b17d64b6a32/687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f32306d735f32306d732e737667>
and
687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f38306d735f38306d732e737667 <https://camo.githubusercontent.com/572896b1f713045d3e146dd32d3431a441d1b6a442e0f3a101d39660094bf93d/687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f38306d735f38306d732e737667>
and
687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f31306d735f3136306d732e737667 <https://camo.githubusercontent.com/0ca81a2fabe48e8fce0f98f8b8347c79d27340684fe0791a3ee6685cf4cdb02e/687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f31306d735f3136306d732e737667>

ostensibly its main reason to exists!


L4S protocol bits: Watch TCP Praque not survive the contact with a FIFO bottlenech as is the norm over arbitrary internet path at the fifth (and in the last plot also the sixth pair of bars:

687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f32306d735f32306d732e737667 <https://camo.githubusercontent.com/a83b3e8c3aa1824e4ff38b5d0d9136079f431c2e917bb347c15b8b17d64b6a32/687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f32306d735f32306d732e737667>
and
687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f38306d735f38306d732e737667 <https://camo.githubusercontent.com/572896b1f713045d3e146dd32d3431a441d1b6a442e0f3a101d39660094bf93d/687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f38306d735f38306d732e737667>
and
687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f31306d735f3136306d732e737667 <https://camo.githubusercontent.com/0ca81a2fabe48e8fce0f98f8b8347c79d27340684fe0791a3ee6685cf4cdb02e/687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f31306d735f3136306d732e737667>


Why are we even discussing with a straight face that L4S would be ready for prime-time?

So far team L4S has NOT refuted Pete's results in any meaningful way (that is by presenting data showing that the issues are remedied).


	So let me ask bluntly, have you actually looked at the data yourself, or are you relaying on team L4S' subjective responses to the data? Or what makes you believe that L4S is ready for an internet-wide roll-out?


	Anyway talking procedurally, I am opposed to validate such quick* and dirty engineering practices as demonstrated by L4S by granting RFC status (which alas, is not under my control, as I fully realize). The precedence this is going to set is: no matter how incomplete an internet draft or how much at odds with observable data, persistence alone will get you RFC status. IMHO the idea behind a standardization process should not be to simply accept proposals as if 'set in stone', but rather to aim at improving the drafts and technologies behind those drafts, especially in the context of an experiment. 
	Speaking of L4S as an 'experiment', so far there are no clear hypothesis that an L4S experiment is going to explore, nor any objective criteria how to evaluate such an experiment. IMHO the whole 'experiment' is going to be can L4S be successfully rolled-out, without a proper definition of what success is going to be. 

	And with L4S I see a lot of room for improvements, bur mostly I see a lot of work for L4S to even reach its stated promises. The secret to success is under promise, over deliver, L4S appears to aim for the opposite approach.

Best Regards
Sebastian


*) Odd that after a decade iL4S still feels somewhat rushed, but that might be subjective.

P.S.: I am reminded of SpaceX/Tesla here, we as a WG seem to endorse a SpaceX, fail often, fail early, but also evolve quickly approach, but not in the context of the empty skies about Texas, but rather as if Tesla would employ the same haphazard approach to develop the autonomous driving facilities in customer cars in existing traffic... But  IMHO that is what releasing the incomplete L4S design and implementation on the internet. UNLESS our goal is to build a fast priority track for short RTT TCP Prague flows over everything else, in that case current L4S might be a contender. Sorry for the lack of sugar coat here, but the whole episode leaves me somewhat jaded. 




On 20 March 2021 15:33:01 CET, "Livingood, Jason" <[e-mail]> wrote:
i believe nobody is asking for rocks (nor shrubberies) here, this is really about a convincing demonstration that the whole L4S design, consisting of network and protocol bits actually behaves like a coherent design and not two disjoint sub-projects that will never really harmonize?

Maybe I am being too simplistic with my operator mindset, but I'd prefer that any long-term decision be data-driven. And to collect significant data you need to conduct an experiment. Which in turn seems to potentially depend on an experimental RFC. So if we want the convincing demonstration sought & real data to drive a long-term decision then how is an experimental RFC not the logical next step?

Jason