[tcpPrague] L4S and BBR (was: [iccrg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text)

[Bob Briscoe <[e-mail]>]

Christian,

I've changed the subject line given it's no longer appropriate.
See inline tagged [BB]...

On 01/11/2020 01:07, Christian Huitema wrote:
>
> I am reading the L4S ECN-AQM proposal with an eye on responding to it 
> in an implementation of QUIC,
>

[BB] That's good news.

> and I have a couple of questions regarding use of ECN marking with QUIC.
>
> The document does not mention QUIC, yet QUIC is already used in a 
> large fraction of Internet traffic. QUIC does specify support for ECN, 
> and QUIC acknowledgements may carry counts of each category of ECN 
> marks received from the peer -- three counters for ECT(0), ECT(1) and 
> CE. In theory, QUIC implementations could take advantage of L4S -- in 
> fact, at least one implementation supports DC-TCP like CC already. Is 
> there interest in specifying L4S for QUIC?
>

[BB] Certainly.
Nonetheless, when I search for the string "QUIC" in L4S ECN-AQM 
proposal, I find it (assuming you mean draft-ietf-tsvwg-aqm-dualq-coupled ).
But you won't find much else about QUIC there, 'cos that doc is about 
the AQM.

There are three main L4S docs.
You need to be reading the doc with the requirements for L4S transports: 
draft-ietf-tsvwg-ecn-l4s-id 
<https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id>
And the L4S architecture: draft-ietf-tsvwg-l4s-arch 
<https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch>
Both of which explain that QUIC already provides the necessary feedback.

Quentin De Coninck wrote an initial implementation of a "QUIC Prague" 
congestion control based on your picoQUIC code. It can be accessed via 
the L4S landing page here:
     https://riteproject.eu/dctth/#code
He started it at an IETF hackathon. I don't think it's been maintained 
since it was first written, but it might be a start.


> My next question regards the interaction of the proposed L4S ECN-AQM 
> with CC algorithms like BBR that attempt to discover the bottleneck 
> packet rate for the connection, and use pacing to send packets at that 
> rate. I observed that BBR is never mentioned in the draft, yet BBR is 
> used in a sizeable part of Internet traffic. Do we have data on how a 
> non-L4S aware implementation of BBR interacts with the proposed L4S AQM?
>

[BB] Again, if you're looking at the L4S AQM doc, you won't find much 
about CC.
For BBR, take a look at section 5.2 of l4s-arch 
<https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-5.2>.

When BBRv2 was released it included initial support for L4S ECN. Neal 
and Yuchung presented BBRv2's L4S ECN queue delay results when they 
first talked about BBRv2 in iccrg. I'm sure more work is needed on the 
L4S ECN part, but I guess that will rise in priority once ISPs start 
deploying L4S AQMs (e.g. Greg White reported that cable implementations 
were being interop tested some time ago, and Nokia released an L4S WiFi 
AP a while back).

So, if there is an L4S AQM at the bottleneck, the resulting ECN signals 
would make BBRv2 morph into an L4S CC.

However, your question is about how a non-L4S aware BBR interacts with 
an L4S AQM.
* For that, I'll have to take 'BBR' to mean something like BBRv2, but 
without any support for ECN - i.e. without the L4S ECN part.
* And just to be super-precise, I'll take the 'L4S AQM' to mean the 
DualQ Coupled AQM (not an FQ one).
Then the answer is, I think, only a few tests of that combination have 
been done. Am I right, Olga, Asad, Olivier, Neal, Yuchung, anyone? I 
suspect this mainly because there doesn't seem to be a reason why the 
L4S ECN part of BBRv2 would be disabled.

I can say what I suspect would happen in what I think is your 
hypothetical case. The DualQ Coupled AQM design relies on a flow that is 
classified into the Classic queue (e.g.. the non-L4S aware BBRv2 you ask 
about) responding to loss and/or delay in a way that would be friendly 
to Reno. The v2 changes to BBR were intended to be more friendly to 
other flows, in particular the response to >1% loss. Then, these non-L4S 
aware BBRv2 flows that you posit should co-exist with L4S flows in some 
way. But whether they would coexist well isn't something I've looked 
into, 'cos I think it's rather a hypothetical question (if I've 
understood you correctly).

Before BBRv2 when BBRv1 had no response to loss (and no ECN support), I 
can say that it usually starved L4S flows over the same DualQ AQM for 
the same reason it usually starved Cubic and other flows in a FIFO. But 
hopefully that is in the past now.


> My last question regards potential use of ECT(1) marking. Most current 
> implementations set ECT(0), but setting ECT(1) instead is trivial. 
> This should elicit an L4S compatible response in L4S-AQM, and the BBR 
> implementation might be modified to use the signals as part of the 
> bottleneck bandwidth tracking. But there is a small issue there. With 
> BBR, QUIC packets are supposedly paced at just under the bottleneck 
> rate, except during "probe" periods in which they probe for 1 RTT at a 
> slightly higher rate. The L4S AGM might degenerate in a form of ON-OFF 
> control -- no feedback at all most of the time, then a bunch of CE 
> marks if the probe rate exceeds the bottleneck bandwidth. As anyone 
> experimented with that?
>

[BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'allowed' 
to send packets over the Internet as ECT(1). Then, yes, an L4S DualQ 
Coupled AQM will classify BBRv2 packets into the L4S queue. This should 
have a very shallow ECN marking threshold (500us - 1ms), so even if the 
flow (whether QUIC or TCP) is flying just under the available capacity, 
bunching of packets means it is unlikely to completely avoid ECN marking 
between probes. If it could avoid ECN marking, you are right that it 
would get a bump of ECN marks during the probe. I haven't studied the 
code, but when it experiences ECN marking I believe it switches into an 
L4S ECN mode for a while, and uses ECN rather than delay probing to 
track available capacity. I assume it switches back to BBR's delay 
probing mode if it gets no ECN for a while (e.g. the bottleneck might 
have moved). But I haven't looked at BBRv2's ECN behaviour in detail.

I'm sure someone on one of the lists that this is posted to will be able 
to answer tho.

HTH


Bob


> -- Christian Huitema
>
>
> On 10/31/2020 2:54 AM, Bob Briscoe wrote:
>> Folks,
>>
>> The co-authors of ECN L4S ID have been reviewing the correctness of 
>> the normative 'Prague' requirements.
>>     See 
>> https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-10#section-4.3
>> This is the second of 2 emails, about 2 of the requirements that we 
>> think ought to be reworded a little.
>>
>> If you agree with the rationale, but think the new wording still 
>> doesn't fully capture the requirement, pls suggest sthg better.
>> If you disagree with the rationale, pls discuss.
>>
>> 4.3.  Prerequisite Congestion Response
>> ...
>> CURRENT:
>>
>>     o  A scalable congestion control MUST react to ECN marking from a
>>        non-L4S but ECN-capable bottleneck in a way that will coexist with
>>        a TCP Reno congestion control [RFC5681  <https://tools.ietf.org/html/rfc5681>] (seeAppendix A.1.4  <https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-10#appendix-A.1.4>  for
>>        rationale).
>>
>>        Note that a scalable congestion control is not expected to change
>>        to setting ECT(0) while it falls back to coexist with Reno.
>>   
>> PROPOSED:
>>    o  A scalable congestion control MUST implement monitoring in order
>>       to detect a likely non-L4S but ECN-capable AQM at the bottleneck.
>>       On detection of a likely ECN-capable bottleneck it SHOULD be
>>       capable (dependent on configuration) of automatically adapting its
>>       congestion response to coexist with TCP Reno congestion controls
>>       [RFC5681] (see Appendix A.1.4 for rationale and a referenced
>>       algorithm).
>>
>>       Note that a scalable congestion control is not expected to change
>>       to setting ECT(0) while it falls back to coexist with Reno.
>>
>> RATIONALE:
>> 1/ The requirement as currently written says what an omniscient 
>> sender MUST do. So there's an implied requirement that a sender MUST 
>> be omniscient, which is of course impossible.
>> 2/ The requirement needs to be recast to require a sender to aim to 
>> be as knowledgeable as possible. Then, what it does as a result needs 
>> to take into account the a priori likelihood of there being a non-L4S 
>> bottleneck present.
>> 3/ This includes the possibility that the operator of the host knows 
>> that the network it serves has not deployed any single queue classic 
>> ECN AQM (e.g. in a CDN case they're doing out of band testing, or 
>> they've asked the ISP). So we've included the possibility of 
>> fall-back being disabled by configuration.
>> 4/ Nonetheless, as has been pointed out on the list, there is still a 
>> possibility that there is a Classic ECN AQM somewhere else on the 
>> path (to continue the CDN example, perhaps beyond the ISP in a home 
>> network). The 'MUST monitor' requirement still stands to ensure the 
>> operator doesn't miss these cases.
>> 5/ Then, if the server operators have disabled fall-back for their 
>> deployment, they can reconsider their policy or at least do more 
>> focused testing if they are frequently detecting a single-queue 
>> Classic ECN AQM.
>>
>> Items 3-5 are the "react via management" model that I've talked about 
>> on the list, given the unfairness doesn't amount to starvation, and 
>> it is possible that the prevalence of the problem is very low.
>>
>>
>> Finally, after the bullet list of requirements in section 4.3, (which 
>> are prerequisites for setting the ECT1 codepoint), we propose to add 
>> the following requirement, as suggested on the tsvwg list:
>>
>>       To participate in the L4S experiment, a scalable congestion 
>> control MUST
>>       be capable of being replaced by a Classic congestion control (by
>>       application and by administrative control). A Classic 
>> congestion control
>>       will not tag its packets with the ECT(1) codepoint.
>>
>> Cheers
>>
>>
>> Bob
>>
>>
>> -- 
>> ________________________________________________________________
>> Bob Briscoehttp://bobbriscoe.net/
>>
>> _______________________________________________
>> tcpPrague mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/tcpprague
>
> _______________________________________________
> iccrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/iccrg

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/