Re: [tsvwg] L4S and the detection of RFC3168 AQMs

["[e-mail]" <[e-mail]>]

Martin,Bob, 
 I'm glad you replied, Martin; firstly for your kind words, but also because Bob's reply below was in my spam folder, so I hadn't seen it. More inline [AB]


On Friday, December 11, 2020, 8:11:23 PM GMT, Martin Duke <[e-mail]> wrote: 


Alex,

Thanks for thinking outside the box like this -- it may be this sort of creativity that gets out of the box we're in.


This falls under the "much easier to do in other transports" category, where I could just send a PING or HEARTBEAT marked ECT(0) to test the queue in mid-connection, without affecting the latency of anything that matters. But in the TCP case, I'm not sure how to resolve Bob's second objection (running ECT(0) for a long time would be unacceptable).

On Wed, Dec 9, 2020 at 3:23 PM Bob Briscoe <[e-mail]> wrote:
>  
>  Alex,
> 
> An interesting idea.
> 
> You've given some pro points.
> Here are a few con points:
> 
> 1) Requires L4S FQ AQMs to actively remove Classic ECN support.
> Many FQ AQMs already support Classic ECN. By this proposal, if they wanted to add L4S ECN support (which is pretty simple to do), they would have to remove Classic ECN support, dropping instead of marking ECT(0) packets. It is possible (likely?) that a number of FQ AQM implementers would vote with their feet and not remove Classic ECN support, even if the IETF published an RFC adopting your proposal. Then the certainty that your proposal is designed to create would be lost. Then we would be worse off in two directions: still no way to be certain about ECT(0) markings, while at the same time having reduced what little Classic ECN deployment there is.

[AB] That would be a bad outcome. Perhaps the constraint could be modified to allow that FQ AQMs supporting L4S were allowed to mark ECT(0) following RFC3168, so long as they ONLY do so in buckets which have not seen ECT(1) for some period? I think this would also help with your 3) below. Maybe this would also help with the issues of unfairness with a) tunnels and b) L4S and normal flows hashing to the same bucket, since the queue in that bucket would build up to the point of dropping and then L4S would fall back to reno. It would need some additional storage per bucket, to track those which have recently seen ECT(1), but that could be a 2 bit countdown counter.


> 2) For in-band Classic ECN AQM detection, the proposal requires L4S not to be used for a long time in order to detect whether L4S can be used.
> L4S hosts are the ones that need to check for Classic ECN AQMs. But they don't send ECT(0). So to take advantage of your proposal, they would have to send ECT(1) packets then, if they see one CE marking, they would have to switch to sending ECT(0) packets, and actively send enough ECT(0) to be sure that /lack/ of CE marking was not just lack of congestion. Given CE marking is currently very rare, how long without CE marking do they have to send ECT(0) packets until they can assume it's an L4S AQM? They don't want to alternate ECT0 & ECT1, cos if it is an L4S bottleneck, that will lead to reordering.
> 
> So, lack of CE marking would be ambiguous. Would it mean the bottleneck is L4S? Would it mean the capacity has increased and the flow needs to increase its rate more to find the limit? If there was a loss or losses, that would also be ambiguous. Are they congestion losses?
> 
> All the time that the L4S source is sending ECT(0) packets, the flow is losing the benefit of the low delay it could have been getting if the original CE mark was emitted from an L4S queue. This is a similar problem to an algorithm that gives false positives (like the v2 Classic ECN AQM detection that Asad & I published). It sometimes detected an L4S AQM as Classic. So it was argued that this would probably prevent people from using it. The same would surely be true of an algorithm that requires L4S not to be used in order to detect whether L4S can be used.
> 

[AB] I was indeed assuming that the transport could send ECT(0) probe whenever it wanted, and that probes would not contain data and so would not cause an ordering problem. I admit that I hadn't realised this might be difficult in tcp, although further in the thread Mike Heard and David Black seem to have identified a possible approach.

I am not sure if, supposing a probe can be sent at any time,  this completely addresses your objection. "not using L4S" could be  
a) not using ECT(1) and therefore not getting the benefit of the L queue, or
b) initially assuming that CE marks have the RFC3168 meaning, rather than the L4S meaning. 

If a way is found to send ECT(0) probes without causing reordering, presumably this addresses a)?  b) is more tricky.

Let me take a step back:
 One approach to getting this WG unstuck, would be to find some fallback algorithm which could easily be shown to be safe, even if it has other deficiencies from the point of view of those who want to deploy L4S. That would remove the most serious objection to starting the experiment; and then the WG might agree that since it would now be safe, the experiment could start, so long as there is a reasonable prospect of developing some improved fallback algorithm  . Once the experiment starts, then it would be easier to test and develop better detection algorithms along side one already known to be safe. 

You guys are in a better position to know what kind of deficiencies are tolerable to those who want to use L4S. 

The reason probing seems attractive is that, once you assume L4S doesn't mark ECT(0) , if probe packets are arranged such that each RFC3168 CE mark has the same, independent probability of hitting an ECT(0) nprobe packet, then the mathematics is very simple. At the risk of teaching everyone to suck eggs, if the proportion of probe packets is P then the expected number of CE marks until detection is 1/P. For safety, the number of CE marks until detection seems to me the right metric, since if there are no CE marks it's not a problem, and the higher frequency CE marks are, the more we care about detecting.And to calculate statistics about that, we don't need any more information than P, and any more assumptions other than that the frequency of CE marks is nonzero. 

Making P large  causes overhead in terms of the number of probe packets, which is one deficiency . On the other hand, even the most safety-minded might be placated with a sufficiently large P, and if also reduces the length of time you might need to wait to decide there is no RFC3168 AQM. 

I had actually assumed that the L4S team's  intention was to start by assuming that there were no RFC3168 AQMs. This seems to be the behavior of the pseudocode in the paper. If that is the case (and the WG accepts it) then there are no false positives at startup. The only reason then  to worry about the amount of time to wait without seeing ECT(0)->CE is if there is an RFC3168 AQM, which you previously detected, and now might no longer be the bottleneck. under those conditions it would seem reasonable to be conservative about switching back.

On the other hand if L4S transports should start in 'RFC3168  safe mode' and only switch to L4S if it decides that there is no RFC3168 AQM on the bottleneck, or if it is important to switch quickly back when the bottleneck changes, then you do need to make some assumptions about the frequency of marking  (of this stream). Unfortunately I don't have enough knowledge  of tp  (or tcp-prague) to devise such an assumption. Given some worst-case assumption of the rate at which an actually-marking AQM would mark, it would be straightforward to calculate the probability P(we should have seen ECT(0)->CE by now) by standard bayesian reasoning, and the expected number of packets before this probability would fall below a given threshold. But, I do not know if this would be a number that you would like.

Unfortunately I don't have much  spare effort to develop this, but  I hope that has provided food for thought.

Alex





> 3) Could be used for Out-of-band Classic AQM detection, but...:
> The proposal would help server operators who wanted to run pre-L4S-deployment tests to check for the presence of Classic ECN AQMs. Then ehy could run saturating ECT(0) flows for long enough to be sure that lack of CE meant absence of Classic ECN bottlenecks. But only if all L4S implementers complied (see item #1). The fact that this proposal is only really useful for pre-deployment testing would surely make FQ implementers even less keen on removing Classic ECN support.
> 
> 
> Perhaps this conversation will spark a variant of your idea. However, in its present form, unless I'm missing something, it doesn't seem to stand up to scrutiny.
> 
> Cheers
> 
> 
> Bob
> 
> 
> On 08/12/2020 19:37, Greg White wrote:
> 
> 
>>  
>>  
>> Hi Alex,
>> 
>>  
>> 
>> This does seem worth considering.  FWIW in Low Latency DOCSIS the implementation will be as you describe.  ECT(0) packets will not be marked CE.  This was done for practical reasons, since the classic AQM re-uses the DOCSIS-PIE AQM which is implemented in hardware in many devices, and does not support ECN.  For consistency, any ECT(0) traffic that were to make its way somehow (e.g. DSCP marked as NQB) into the LL queue, will also not be CE marked. 
>> 
>>  
>> 
>> -Greg
>> 
>>  
>> 
>>  
>> 
>>  
>> From: "[e-mail]" <[e-mail]>Reply-To: "[e-mail]" <[e-mail]>Date: Monday, December 7, 2020 at 3:44 PMTo: ""[e-mail]"" <[e-mail]>, ""[e-mail]"" <[e-mail]>, Greg White <[e-mail]>Cc: ""[e-mail]"" <[e-mail]>Subject: L4S and the detection of RFC3168 AQMs
>> 
>> 
>>  
>>  
>> 
>> 
>>  Hi all,  At present the DUALQ draft leaves it to implementers to decide if traffic in the classic queue (IE, ECT(0) traffic that uses currently standard congestion controllers) should be dropped or marked (except, apparently, when an ECT(0) packet for some reason turns up in the L queue?). Perhaps it would be wise to specify that during the experiment, deployments of L4S AQMs (whether DUALQ or the other alternatives mentioned in draft-ietf-tsvwg-l4s-arch) SHOULD NOT (or maybe even MUST NOT) mark ECT(0) traffic, and only drop. This would be to preserve the fact that, as currently, those RFC3168 AQMs which are unaware of L4S, can be identified by the fact that they mark ECT(0) packets with CE. If L4S AQMs are required not to mark ECT(0) as CE, then if an endpoint (or other monitoring method) sees a CE mark on an otherwise ECT(0) flow, then it knows with more or less 100% confidence that an RFC3168-only AQM is on the path. Presumably this is safe, since L4S nodes are required to be able to support Not-ECT traffic, and ECT(0) traffic is required to be able to cope with drop.  At present the only definitive way for endpoints identify an RFC3168 AQM in use, is to observe CE marks on an ECT(0) marked flow.Bob's paper [1] gives various other methods, but this seems to be a research project at present. If any of these approaches were found to be reliable, then the above requirement could be relaxed fairly easily; the reverse is not so easy. There are, as is probably obvious, a number of reasons for wanting to identify RFC3168 AQMS: - Current efforts to gather statistics on the number of RFC3168 AQMs which might encounter problems when L4S traffic passes through them. - The the ops draft (sec 3.1)  envisages that CDN operators should test for the presence of RFC3168 AQMs, but doesn't yet specify how this should be achieved - Within a L4S transport protocol, in order to fall back to RFC3168 behavior All of these would presumably be assisted by being able to classify observed ECT(0)->CE transitions unambiguously being the result of an L4S-unaware node. Unless I'm missing something it seems very much worth considering to restrict L4S marking behavior of ECT(0) for the time being? I am not sure which drafts would need updating; DUALQ at least, but presumably the ops draft and maybe L4S-arch. regards, Alex [1] https://arxiv.org/pdf/1911.00710.pdf
>> 
>> 
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
>