Re: [tsvwg] L4S drafts: Next Steps

[Sebastian Moeller <[e-mail]>]

Hi Jason,


> On Mar 20, 2021, at 16:08, Livingood, Jason <Jason_Livingood=[e-mail]> wrote:
> 
>> The lab studies have shown
> 
> IMO the challenge with lab studies is that there are a lot of variables that are artificial and/or the test environment or test traffic does not fully reflect reality (production).

	[SM] Nobody hindered team L4S to actually vary those parameters.

> 10 or 15 years ago I would have been very focused on extensive QA testing and an elaborate lab test environment. These days its more how to do controlled testing directly in the production network, with a controlled/minimized blast radius in case of problems, easy/quick rollback, rapid code/config iteration.

	[SM] Controlled testing over real networks is actually possible already, without the IETF's blessing, as long as one makes sure the potential fall-out is restricted to voluntarily participating nodes. To claim there is nothing between tests in the lab and "release the kraken" onto the whole internet, does not strike my as aterribly nuanced nor realistic position?


> 
>> far from a promising system that might be ready for a field trial, one that shows worrying failure modes that primarily affect innocent bystanders.
>> What we're asking for is a system that at least behaves reasonably under lab conditions, chosen to represent realistic challenge scenarios likely to occur in the real world.  Only then can we have any confidence that L4S will not cause problems anywhere and everywhere that it is deployed.
> 
> Only one way to find out - test it in a real production experiment with appropriate risk controls.

	[SM] Like a guarding DSCP that makes sure the fall-out is restricted mainly to willingly cooperaring parties? Such common-sense safeguard have been summarily rejected by team L4S... In other words there will be no risk controls, let alone appropriate ones. I come to this conclusion as extrapolation of L4S lab test trajectory. Why do you assume the approach towards safety and functionality engineering is going to change?


> The status quo seems to suggest years more debate without (IMO) sufficient data.

	[SM] So all I need t do to get an incomplete draft to RFC status is to drag my feet? Surely that can not be your serious position?

> An alternative is a controlled production experiment, appropriately risk-managed, between willing participants.

	[SM] +1, but that is not what is under discussion here. We are talking about "release the kraken" based on theoretical considerations, that L4S will work and there will be no incentive to exploit its known failure modes. Engineering by "hope and prayers"...


> I see no downside to allowing that to occur via experimental RFC. We're setting the bar at the proposed standard or standard level, which I think is incorrect. I would say instead enable the experiment to accelerate the learning & improvement & drive data-driven decisions.

	[SM] Again, for a "a controlled production experiment" no buy in of the IETF is required, it never has. So why pretend otherwise?

Sebastian

P.S.: I missed the last meeting, so if team L4S actually presented data that answers all my questions, I am willing to eat my hat, so did they?


> 
> Jason
>