Re: [tsvwg] plan for L4S issue #29

["De Schepper, Koen (Nokia - BE/Antwerp)" <[e-mail]>]

	[SM] Which deliverables did you agree to and with whom?

I assume Bob means the L4S drafts.

	[SM] Least by which criterion?

My main criterions were: Incentive to get ECN finally widely deployed by having a "big" advantage compared to non-ECT traffic, and having a simple enough network AQM that would be good enough to Classic traffic (better than tail-drop) and very good-enough for low latency (and throughput) to the new-ECN users.

	[SM] Yes, that fleetingly appeared on the list, I commented up it and did not head much of that thereafter.

Thanks for this. Your comments were noted and will certainly be taken into account by Greg (who is editing the draft). I understood from Greg that any contributions of text (filling the todo-gaps and rephrasing the existing text) are welcome.

Koen.


-----Original Message-----
From: tsvwg <[e-mail]> On Behalf Of Sebastian Moeller
Sent: Wednesday, September 23, 2020 10:01 PM
To: Bob Briscoe <[e-mail]>
Cc: [e-mail]
Subject: Re: [tsvwg] plan for L4S issue #29

Dear Bob,

a few questions in-line, below.


> On Sep 22, 2020, at 19:41, Bob Briscoe <[e-mail]> wrote:
> 
> Jonathan, Wes, list,
> 
> Sorry, I've been dead to the L4S conversations on this list for weeks now. But I've resolved to get back involved now. I stopped interacting on L4S over the summer because I became burned out by the L4S conversation being dominated by the same two or three people who do not want the agreed deliverables at all.

	[SM] Which deliverables did you agree to and with whom? 


> I have never had to work on a technology before where I am faced with a continual stream of blocking messages, rather than constructive criticism or solutions being offered.

	[SM] I take it you ignored the IPv4/v6 discussions then?

> 
> It has always been recognized that there are not enough bits in the IP header for a perfect encoding scheme, so that we will have to choose the least worst compromise.

	[SM] Least by which criterion?

> We had a consensus call in April,

	[SM] Did we? As far as I remember the decision back then was always a "vote" of confidence for L4S over SCE, to make team L4S continue with the experient, but it did not (even try to) answer the question whether L4S is ready for deployment.

> in which the questions of Classic ECN fallback (#16) and false positives in the current algo (#29) were well understood by all concerned. The result was a decision not to go with the alternative solution (SCE) that inherently addresses issues #16 & #29, but instead to continue with the solution (L4S) that partially addresses these issues and fully addresses other concerns. The L4S design team were asked to prepare a new ops guidelines draft as part of that decision.

	[SM] Yes, that fleetingly appeared on the list, I commented up it and did not head much of that thereafter.

> 
> I became disheartened about working further on the Classic ECN AQM fallback algo when all the work we'd done was just dismissed as futile.
> It became apparent that no amount of work on it would ever satisfy the few people who have been vocally opposing L4S as a whole.

	[SM] I have a feeling you are targeting in my direction here, but I have always maintained that I can be convinced by data, as long as the data was reasonably representable for to be expected "bad" conditions over the open internet (that is to say pure proofs of principle that something can work under some carefully selected conditions are not enough, no matter under how many iterations known good conditions are shuffled).


> Now that's a shame, because I/we had worked out an additional heuristic to address the large bulk of the false positives that appeared at low link capacity (note that is different from low flow rate). I admit that this solution would not have addressed the few cases that arise in apparently random circumstances. And I admit that, without having tried it, it might not work at all.

	[SM] Not sure that piling up heuristic on top of heuristic is a safe way to engineer robust and reliable solutions, sorry.


> 
> I now prefer to work on Classic ECN AQM detection more than fallback. My logic for this is that L4S can be /unfair/ against Classic flow(s) in a Classic ECN AQM, but it is not /unsafe/. Unsafe means starvation, but Classic flows always still progress against L4S.

	[SM] Could you please give a definition of "starvation", in terms of share of a bottleneck. Without a proper numeric definition that sub thread is going to de-rail quickly. Here is mine: giving a flow less than 1:8 of its equitable share on a bottleneck is starvation in my book. (I accept that too many flows will result in almost unusable throughput for everybody, but that is a different starvation criterion, absolute minimal throughput, which for the discussion at hands seems less suited).


> So, it should be sufficient for L4S senders to /monitor/ for Classic AQMs, at least in the initial stages of the L4S experiment.

	[SM] Or better keep the L4S experiment isolated from the rest of the internet, then you can do whatever you want, no? 

> 
> This can then become part of the monitoring infrastructure that the L4S ops guidance proposes for the L4S experiment. It can highlight where classic ECN AQMs might have been deployed.

	[SM] How that? You will need numbers for your classifiers performance versus the ground truth to assess that, but by running over the uncontrolled internet, you will not have ground truth information. So please elaborate how you intend to use your classifier's output to assess the quality of said classifier itself?


> Then, if it produces false positives,

	[SM] How do you plan to figure out whether a detection is a hit, or a false positive and whether a non-detection is a correct rejection or a miss, if you do not run the experiment over a know network with reliable information about the exact rfc3168-ness of each AQM along the path?

> further digging should discover that there is not actually any classic ECN AQM where it says there is. Then at least false positives don't harm performance.

	[SM] This is IMHO the wrong mindset, do not optimise prematurely for you preferred case, but rather first see how reality looks and then optimize... 

> 
> If it turns out that there are a lot of classic ECN AQMs out there, it then becomes important to improve the detection algo so it can be used in-band for fall-back. On the other hand, if we find there is very little, it becomes preferable to use the network configuration techniques described in the ops guidance to alter the classic ECN AQMs to isolate classic ECN and L4S ECN traffic.

	[SM] This seems all quite optimistic, unless we get an error free rfc3168 detector first to assess the likelihood of rfc3168 AQMs on a typical internet path, no? Really, I must be missing something here, but this does not look like engineering for a proper solution here.

Best Regards
	Sebastian

> 
> Regards
> 
> 
> 
> Bob
> 
> PS. There are also the cases where L4S flows don't perform well (particularly around start-up behaviour) in a Classic ECN AQM within an FQ scheduler (e.g. the experiments Pete did with L4S over FQ_CoDel and with Cake). Over the summer Joakim and I have been working on faster response to marking in the Prague algo, and we need to combine that with delay measurements for these cases.
> 
> I would also like to see a commitment to improving the CoDel and Cobalt control laws to increase ECN marking more rapidly in response to consistently increasing delay. Because this has always been a problem with unresponsive flows, irrespective of whether L4S had ever appeared on the scene. I'd be willing and interested to help with that sort of work,... assuming my day job left sufficient time for this.
> 
> 
> On 31/07/2020 19:03, Jonathan Morton wrote:
>>> On 31 Jul, 2020, at 6:41 pm, Wesley Eddy <[e-mail]> wrote:
>>> 
>>> Hello, ticket #29 for the L4S documents is about classic bottleneck detection misidentifying L4S queues as classic ECN queues.
>>> 
>>> https://trac.ietf.org/trac/tsvwg/ticket/29
>>> 
>>> In contrast to other issues, it doesn't seem like this should block a WGLC on the L4S drafts.
>>> 
>>> 	• It is specific to classic bottleneck detection algorithm, which is planned to be worked on in the Prague ICCRG draft.
>>> 	• The result is sometimes failing to achieve the best possible L4S behavior, but doesn't seem to be an Internet safety issue.  This resulting in people turning off classic bottleneck detection would be a different issue, and something maybe the operator guidelines would address.
>>> 	• It seems like it can be worked on further in the course of L4S experimentation, without negative effects to others.
>>> So, I believe we should track this work in the ICCRG, and close the ticket here.  Please let me know in the next week if I've misunderstood any aspect of this and it should remain open.
>> Presently, the Prague congestion control algorithm is unsafe on the Internet without a properly functioning classic-bottleneck detection heuristic.  Additionally, the classic-bottleneck detection heuristic that has been published and demonstrated does not function properly.  This combination of facts absolutely *is* a blocker for WGLC.
>> 
>> Therefore, this issue should not be closed until it has been concretely addressed.
>> 
>>  - Jonathan Morton
>> 
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
>