Re: [tsvwg] plan for L4S issue #29

[Gorry Fairhurst <[e-mail]>]

See below.

On 30/09/2020 09:00, Sebastian Moeller wrote:
> Dear Greg,
>
> more below in-line, prefixed with [SM]...
>
>> On Sep 30, 2020, at 00:48, Greg White <[e-mail]> wrote:
>>
>>
>>
>> On 9/29/20, 10:05 AM, "tsvwg on behalf of Gorry Fairhurst" <[e-mail] on behalf of [e-mail]> wrote:
>>
>>     See below.
>>
>>     On 29/09/2020 16:49, Rodney W. Grimes wrote:
>>>> On Mon, 28 Sep 2020, Gorry (erg) wrote:
>>>>
>>>>> At some point the working group needs to publish the spec. - This final
>>>>> stage is taking longer than I would hope, and I do hope that will be
>>>>> seeing a WGLC soon.
>>>> Do we actually?
>>     Yes.
>>
>>>> I still haven't ruled out that we decide not to use these bits, for now,
>>>> because we don't know enough how it will affect the entire Internet.
>>     Still possible, if the WG as a whole decides that.
>>
>> We're talking about WGLC to begin an experiment.
> 	[SM] I believe that we are conflating two "experiments" here:
>
> 1) the absolutely required experiments to what degree L4S will realize its claimed characteristics (or its promises) under a number of conditions that are relevant/prevalent in the real world.
> 2) how to deploy L4S in the real world.
>
> If we look carefully at these two experiments it becomes obvious, that an RFC in the experimental track seems necessary for 2), but 1) can and should proceed long before 2) is being addressed. To be blunt, if experiment 1) is not clearly demonstrating improvements over the state of the art, then 2) becomes moot.
>
> @CHAIRS: Could you please let me know, if you agree?
>
>> The interest by the community in achieving the expected benefits of L4S was well represented at the consensus call. I believe that the WG needs to honor the consensus position and move forward with planning the experiment.
> 	[SM] That is a call for experiment 1), but the drafts really aim for experiment 2).
>
>
>> With appropriate guidance I believe the experiment can commence and we can begin to understand whether any of the concerns raised are real, and if they are, how to resolve them.
> 	[SM] I disagree, at the current time, neither the drafts (L4S and operational guidance) are properly tailored for type 1) experiments.
>
>> For the L4S Operational Guidance draft, I would appreciate constructive input from those who are interested in making the experiment a success (thanks already to Sebastian for his comments).  Important aspects include: If a sender wishes to test an L4S congestion control algorithm (Prague or otherwise) what should they be monitoring to understand how much of a benefit are they getting, what impact are they causing to non-L4S flows, etc.? If a network operator wishes to test an L4S compliant network element, what should they monitor?  What actions should either entity take with the resulting information?
> 	[SM] These are all good points. Let me add a few thoughts here.
>
> Let's assume an endpoint wants to participate in testing it will require a few things:
> 1) a L4S-compatible transport that sets ECT(1), L4S-incompatible transports that set ECT(1), as well as standards compliant transports (ECT(0)/ no ECN)
> 2) one/more L4S-compatible remote endpoints
> 3) a L4S AQM at a bottleneck, that will allow monitoring all traffic (the only way endpoints will be able to assess the side-effects of L4S traffic is to actually terminate all such flows).
>
> Point 3) alone makes it clear that for this kind of endpoint driven experiment, only small scale AQMs will be suitable, like AQMs on an end-user internet access link, that are exclusively managing that links traffic and do not interact with other access links' AQM instances.
>
> And given these constraints, all that seems required for type 1) experiments IMHO is to put the access link AQMs under end-user control (preferably with three states: AQM off, all traffic, all traffic is treated like ET(0), full L4S-compliant behavior). If such an AQM defaults to AQM off, this is already save to deploy today, without requiring any additional RFC...
> To be explicit, this opt-in approach allows to control the fall-out quite well; any side-effects of the L4S AQability testingM misbehaving will be mostly restricted to the end-node that opted-in. Sure that is not 100% safe, but it will allow the crucial type 1) experiments to proceed without having to commit to ECT(1) at the IETF RFC level.
>
> Now, realistically there is nothing so far, that would have made that course of action impossible even today, so I wonder why such a setting has not already been used to make the required robustness and reliability tests that L4S is lacking (since almost a decade now?)
>
>
> As is, what I predict is going to happen, is that ECT(1)/L4S is going to end up as RFC without proper robustness and reliability testing, heck without even a clear indication that its promises will realize over anything but short RTT, low hop-count links, by sheer impulse conservation.
>
> Some comments by the chairs seem to indicate that we are already well along that path... but there is still time to do it right. L4S has been slow enough that waiting another X years can not be a showstopper, not that the required type 1) experiments necessarily would take years...
>
>
> Best Regards
>          Sebastian
>
>
>
>> In my opinion, Issue 29 can be closed. RFC3168 detection should continue to evolve and should be referenced in the Operational Guidance draft, but fallback should not be required in the experimental protocol drafts for the reasons outlined in Issue 29, and on the mailing list.
>>
>> -Greg
>>
>> [snip]
>>
>>
Let me try as one chair who has seen several TSVWG experiments over the 
years:

The WG chose to adopt this some time ago, presumably because they saw 
the potential to offer benefit over the state of the art. We now have a 
set of working group documents.

The primary consideration for progression of this type of specification 
is understanding of the safety of the method for at-scale deployment.  
Specifically, for such an experiment, appropiate controls need to be in 
place to actively prevent congestion collapse, and avoid 
security-related pitfalls. The potential set of issues need to be 
teased-out for network operators and the WG to understand how to 
evaluate this.

Technical evaluation of performance, latency, fairness, etc is always 
welcome - including discussion of published papers and presentations of 
results (e.g., in ICCRG). This is however, not the purpose of the TSVWG 
specification.

In the future, those choosing to implement and deploy the specification 
will provide most useful input to any future progression along the 
standards track, and to inform any resulting best current practice - or 
indeed, for this WG to understand whether the experiment is ultimately 
deemed successful or unsuccessful.

Gorry