Re: [tsvwg] L4S drafts: Next Steps

[Gorry Fairhurst <[e-mail]>]

On 20/03/2021 15:08, Livingood, Jason wrote:
>> The lab studies have shown
> IMO the challenge with lab studies is that there are a lot of variables that are artificial and/or the test environment or test traffic does not fully reflect reality (production). 10 or 15 years ago I would have been very focused on extensive QA testing and an elaborate lab test environment. These days its more how to do controlled testing directly in the production network, with a controlled/minimized blast radius in case of problems, easy/quick rollback, rapid code/config iteration.
>
>> far from a promising system that might be ready for a field trial, one that shows worrying failure modes that primarily affect innocent bystanders.
>>   What we're asking for is a system that at least behaves reasonably under lab conditions, chosen to represent realistic challenge scenarios likely to occur in the real world.  Only then can we have any confidence that L4S will not cause problems anywhere and everywhere that it is deployed.
> Only one way to find out - test it in a real production experiment with appropriate risk controls. The status quo seems to suggest years more debate without (IMO) sufficient data. An alternative is a controlled production experiment, appropriately risk-managed, between willing participants. I see no downside to allowing that to occur via experimental RFC. We're setting the bar at the proposed standard or standard level, which I think is incorrect. I would say instead enable the experiment to accelerate the learning & improvement & drive data-driven decisions.
>
> Jason

Jason,

Thanks for writing this so clearly!


This sounds like what I was expecting. The labs tests have been useful, 
but the next proposed steps are the deployment experiment, which as far 
as I can see what has driven the development of these drafts in the 
working group.

Gorry