Re: [tcpPrague] [aqm] L4S status update

"Bless, Roland (TM)" <> Thu, 24 November 2016 20:48 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C23E9129CD5; Thu, 24 Nov 2016 12:48:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.2
X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Iao9TOjApXZS; Thu, 24 Nov 2016 12:48:05 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id D7D5C129CBD; Thu, 24 Nov 2016 12:48:04 -0800 (PST)
Received: from ([] by with esmtp port 25 iface id 1cA0w7-0006CM-EE; Thu, 24 Nov 2016 21:48:03 +0100
Received: from [IPv6:::1] (localhost []) by (Postfix) with ESMTPS id 26ABDB0033C; Thu, 24 Nov 2016 21:48:25 +0100 (CET)
From: "Bless, Roland (TM)" <>
To: Bob Briscoe <>
References: <> <> <>
Organization: Institute of Telematics, Karlsruhe Institute of Technology (KIT)
Message-ID: <>
Date: Thu, 24 Nov 2016 21:48:02 +0100
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20060111 Thunderbird/1.5 Mnenhy/
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: 8bit
X-ATIS-Timestamp: 1480020483.
Archived-At: <>
Cc: tcpm IETF list <>, AQM IETF list <>, tsvwg IETF list <>, TCP Prague List <>
Subject: Re: [tcpPrague] [aqm] L4S status update
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 24 Nov 2016 20:48:08 -0000

Hi Bob,

see comments inline, please.

Am 22.11.2016 um 20:09 schrieb Bob Briscoe:
> I share your concern about cc-specific AQMs. But that is not a good
> characterization of what we're doing.

Yep, but it's part of what you're proposing.

> On the current Internet, everything is meant to be somewhat "friendly"
> to the original TCP cc (now spec'd in RFC5681). All sorts of cc's work
> alongside that, with slightly different "fairness" properties, and only
> one AQM is needed to cover them all. Nonetheless, Reno is the "lamest",
> so everyone has to try to "Do (not much) Harm" to the lamest.

Yes, and that's actually a deployment problem we agree on. Congestion
control (CC) innovation is obstructed by the old loss-based CC so some
separation mechanism would be good to let room for low-delay CCs.

> *Is any AQM CC-neutral?**
> *Note rule 5 <> in the
> AQM Guidelines [RFC7567]
>       "AQM algorithms SHOULD NOT interpret specific transport protocol
> behaviors."
> In general, the advice in that section is sound, but I don't think we
> realized at that time just how subtle this issue is.

I think that AQMs could be in fact designed CC neutral to a large
extent, but every CC reacts probably different to the drop/ECN signals.
So the dropping strategy implemented by the AQM will always cause a
CC-specific reaction. Some AQMs therefore may achieve better results if
they are tuned to specific CC behavior (but my e2e argument would also
apply here), however, right now it's not that easy to change AQM
implementations, esp. those implemented in hardware.

> Since then, I discovered that the autotuning parameter table in the PIE
> algorithm is designed very precisely around the 1/sqrt(p) rule of Reno
> (see Fig 5 in the PI^2 paper <>).
> Similarly, the sqrt control law in Codel claims to be dependent on Reno
> {Note 1}.

> The point is that these AQMs still work fine with Cubic, Compound,
> Westwood, etc, because all these ccs were designed to interwork with
> Reno. {Note 2}
> The idea of L4S (and specifically the DualQ Coupled AQM and the L4S ID
> spec) is to enable a shift to a completely different "norm", but still
> coexist with all the 'Classic' cc's that coexisted around the old
> "norm". The new norm is intended to be just as fuzzy as the old norm
> {Note 3}. The idea is two fuzzy clouds of congestion controls, around an
> old and a new norm that are related together.

Yes, and I support that basic idea to introduce network support for
separating flows with different CC schemes. However, I'd like to have a
solution that doesn't build in a specific coupling law, otherwise we
will end up with having either TCP friendly or DCTCP friendly CC schemes.

> *BBR**
> *I believe BBR attempts to be 'friendly' to loss-based flows when
> competing in the same queue. But it's still research, and we don't yet
> know how good it is at that in all scenarios, although we do have code

I'm not sure how friendly/fair BBR is actually to other CCs,
but L4S is also still research...

> to test now. Given BBR currently sets Not-ECT, it would classify itself
> into the Classic queue of a DualQ AQM, and if it coexists with Reno it
> /should/ coexist with L4S traffic in the other queue. See Koen's recent
> posting
> <>
> about this.

Hmm, from what I understood so far BBR reacts differently to packet loss
than Reno/Cubic etc. So that coupling by dropping probability may not
work correctly in this case, because BBR will not react according to
1/sqrt(p) (cf. the FAQ from the BBR slide set presented at IETF97,

> There would be nothing to stop someone designing a variant of BBR that
> coexisted in the L4S queue with Scalable CCs like DCTCP (the point being
> that if the bottleneck was not DualQ it would keep delay low and if if
> the bottleneck was DualQ it would benefit even more from the lower
> queuing delay there). However, it would have to be a bit more careful
> about its whole round trip of queue probing, to avoid increasing the
> delay in the L4S queue. You'll see that I suggested to Neil Cardwell
> that they consider probing with a few packets rather than a whole
> window, e.g. the chirping
> <> technique that Mirja
> and I looked into back in 2010 was designed to find the same knee
> between rate increase and delay increase, with far fewer packets. I
> thought of a better way of using chirping a few weeks ago, so I will be
> returning to that too.

That would be interesting...

> *Specs**
> *There is no statement that all L4S cc's MUST adhere to a 1/p rule. The
> L4S ID draft says:
>   "The inverse proportionality requirement above is worded
>    as a 'SHOULD' rather than a 'MUST' to allow reasonable flexibility
>    when defining these specifications."

I found two MUSTs in these contexts:

   In order to prevent
   starvation of Classic traffic by scalable L4S traffic (e.g.  DCTCP)
   the drop probability of Classic traffic MUST be proportional to the
   square of the marking probability of L4S traffic, In other words, the
   power to which p_L is raised in Eqn. (1) MUST be 2.
   The likelihood that an AQM drops a Not-ECT Classic packet (p_C) MUST
   be roughly proportional to the square of the likelihood that it would
   have marked it if it had been an L4S packet (p_L).  That is

      p_C ~= (p_L / k)^2

> I hope that 'SHOULD' is fuzzy enough - I suspect adding more words would
> make it less fuzzy. But I would welcome wording to make it even more
> fuzzy if you would like to engage in wordsmithing.
> Nonetheless, we are trying to steer a path between a rock and a hard
> place. Because, to shift to a much calmer waters beyond the rocks, we
> have to define some number to relate L4S to Classic. I am wary because
> when ECN was specified, there were attempts to define ECN as different
> to drop. However, ECN originally ended up "the same as drop" because
> no-one could muster enough backing behind any particular number to
> relate the two, so the number '1' won by default (ie. ECN = 1 * drop^1 ).

That's a different discussion.

> Bob
> {Note 1} I have never got a good answer to my questions on aqm@ietf as
> to why a sqrt that controls the shrinkage of the spacing between dropped
> packets has something to do with the steady state law of Reno,
> particularly because the law leads to linear growth in p over time.

Yep. However, according to our experiments Codel's dropping law seems to
achieve a quite reasonable loss desynchronization.

> {Note 2} Actually, I don't believe PIE would work that well with Cubic
> at v high BDP, once it was far from Reno-friendly, but that's only
> intuition from the stability analysis, not from actual testing.

We tested PIE at 10GB/s and it worked with Cubic similarly as at
low speeds.

> {Note 3} Indeed, at the moment, when DCTCP is on its own in the L4S
> queue of the DualQ AQM as coded now, it hits up against a step
> threshold, which makes it behave as 1/p^2, not 1/p. For now, that's just
> because we didn't want to change too much about DCTCP at one time. But
> it's also got some nice properties. This will all need to be discussed
> as the DualQ AQM is specified more deeply.

So probably it would be good to find out what is the common denominator
of _new_ L4S-like CCs (not only considering DCTCP) and what is a common
denominator for CCs in the other class.


> On 21/11/16 14:27, Bless, Roland (TM) wrote:
>> Hi Bob and all,
>> see below.
>> Am 01.11.2016 um 01:02 schrieb Bob Briscoe:
>>> A few people have been working away to specify and document all the
>>> aspects of the new Low Latency, Low Loss, Scalable throughput (L4S)
>>> service, which held a successful BoF in Berlin. As the decision was to
>>> try to work across multiple WGs, I thought it would be useful to give ...
>> Thanks for putting this together.
>>>   * Dual Queue Coupled AQM
>>>       o With Curvy RED for Linux (access available shortly)
>>>       o With PI2 for Linux <> [*UPDATED*]
>> I'll repeat my concerns that I already expressed at the L4S BOF in Berlin:
>> While I agree that we probably need to separate low-delay congestion
>> control schemes from traditional "queue-filling" congestion schemes,
>> I strongly suggest to avoid putting a congestion control-specific
>> coupling scheme into the network (a classic case for applying the
>> "end-to-end arguments in system design").
>> The current Dual queue coupled AQM proposal has got a coupling based on
>> a congestion control specific dropping law p_C=(p_L/2)². So if
>> congestion control schemes change then this coupling needs to be
>> adapted. For example, the currently proposed scheme may fail if that
>> vast majority of TCP traffic is using BBR other some other forthcoming
>> CC scheme instead of Cubic, Reno, Compound etc. The same applies to
>> draft-briscoe-tsvwg-ecn-l4s-id, section 2.5, where the dropping
>> likelihood is defined.
>> Regards,
>>  Roland