Re: [tsvwg] L4S drafts: Next Steps

[Pete Heist <[e-mail]>]

Hi Jason, one comment below...

On Sat, 2021-03-20 at 15:08 +0000, Livingood, Jason wrote:
> > The lab studies have shown
> 
> IMO the challenge with lab studies is that there are a lot of variables
> that are artificial and/or the test environment or test traffic does
> not fully reflect reality (production). 10 or 15 years ago I would have
> been very focused on extensive QA testing and an elaborate lab test
> environment. These days its more how to do controlled testing directly
> in the production network, with a controlled/minimized blast radius in
> case of problems, easy/quick rollback, rapid code/config iteration.
> 
> > far from a promising system that might be ready for a field trial,
> > one that shows worrying failure modes that primarily affect innocent
> > bystanders.
> >  What we're asking for is a system that at least behaves reasonably
> > under lab conditions, chosen to represent realistic challenge
> > scenarios likely to occur in the real world.  Only then can we have
> > any confidence that L4S will not cause problems anywhere and
> > everywhere that it is deployed.
> 
> Only one way to find out - test it in a real production experiment with
> appropriate risk controls.

Along these lines, there is nothing stopping anyone from using DSCP (as
an appropriate risk control within participating ASs), to perform
experiments to reproduce issues that can be demonstrated in the lab.
Here are a few: https://github.com/heistp/l4s-tests/

It would be nice if someone took the time to set up some production
gear with fq_codel built into it, like routers and/or WiFi access
points, and test various kinds of real-world traffic, through tunnels
and not. One could combine traffic to L4S servers within the
experimental network, along with conventional traffic to anywhere on
the Internet. That seems like a useful step before starting an
Internet-wide experiment. Perhaps even better would be to fix what
safety issues are known so that they can't occur in the first place.

Regards,
Pete

>  The status quo seems to suggest years more debate without (IMO)
> sufficient data. An alternative is a controlled production experiment,
> appropriately risk-managed, between willing participants. I see no
> downside to allowing that to occur via experimental RFC. We're setting
> the bar at the proposed standard or standard level, which I think is
> incorrect. I would say instead enable the experiment to accelerate the
> learning & improvement & drive data-driven decisions.
> 
> Jason
>