Re: [tcpPrague] [aqm] L4S status update

Dave Täht <> Wed, 30 November 2016 04:09 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 0C8531294B7; Tue, 29 Nov 2016 20:09:23 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.398
X-Spam-Status: No, score=-3.398 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.497, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Rnu_8K0EPQOC; Tue, 29 Nov 2016 20:09:21 -0800 (PST)
Received: from ( [IPv6:2a01:7e00::f03c:91ff:feae:7028]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id D01E1129447; Tue, 29 Nov 2016 20:09:20 -0800 (PST)
Received: from dair-2506.local ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPSA id 1F02021341; Wed, 30 Nov 2016 04:09:17 +0000 (UTC)
To: Bob Briscoe <>, Jonathan Morton <>
References: <> <> <> <> <>
From: Dave Täht <>
Message-ID: <>
Date: Tue, 29 Nov 2016 20:09:16 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: 8bit
Archived-At: <>
Cc: tcpm IETF list <>, AQM IETF list <>, tsvwg IETF list <>, "Bless, Roland (TM)" <>, TCP Prague List <>
Subject: Re: [tcpPrague] [aqm] L4S status update
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 30 Nov 2016 04:09:23 -0000

On 11/25/16 6:23 AM, Bob Briscoe wrote:
> Jonathan,
> On 22/11/16 20:37, Jonathan Morton wrote:
>>> On 22 Nov, 2016, at 21:09, Bob Briscoe <> wrote:
>>> {Note 1} I have never got a good answer to my questions on aqm@ietf as to why a sqrt that controls the shrinkage of the spacing between dropped packets has something to do with the steady state law of Reno, particularly because the law leads to linear growth in p over time.
>> If you have intervals between events which follow a 1/sqrt(N) sequence, where N is the number of preceding events, you get an event frequency which increases linearly with time.  

We do have a tendency to talk past each other regarding codel's rate.
the 1/sqrt(N) only applies during codel's initial seeking toward an
ideal drop rate. after that -

for 99.99999% of the time, after that, there is no definable *rate* in
codel - it varies based on the workload and the time spent above and
below the target. I really wish I'd produced an animation of this rather
than the first graph I did at stanford.


as for what that initial ramp has to do with reno or tcp's behaviors, it
seems to work, and we've tried other things that didn't work as well.
Give cake's variant a shot. You tell me.

> Yes, I pointed that out originally
> <>. And
> Toke confirmed in response that it did indeed happen in practice.
>> This applies directly to Codel’s signalling strategy, which is to start at one mark per (assumed) RTT, and to increase the marking frequency if that was insufficient to control the queue.
>> When you have multiple Reno flows sharing a single queue, there is a sqrt(N) factor in several of the characteristics, where N is the number of flows.  When such a shared link becomes saturated, all of the flows must be signalled to slow down, but for stochastic reasons it’ll probably take more than N signalling events to do so.  Increasing the signalling frequency while the queue remains insufficiently controlled has a good chance of quickly finding all the flows, while dropping relatively few packets (of non-ECN flows).
> Just throwing a square root in "somewhere", doesn't mean it is the
> correct "somewhere".
> A stated goal of the sqrt in the CoDel control law is to match the
> 1/sqrt(p) in TCP Reno's window formula. Quite aside from whether that is
> a correct goal, it isn't even doing that:
> * ACK-clocked load (number of flows, N) is proportional to 1/cwnd, ie.
> proportional to sqrt(p) [see 2nd para of section 4 of PI2 paper
> <>]
> * because CoDel applies a sqrt to the interval between drops, it results
> in a linear increase in p with time (because the sqrt effectively gets
> squared - see the simple maths in my original posting about this
> <>).
> So, the question is: "Why is a linear increase in p (starting from 0)
> good for controlling load from N flows, where N is proportional to sqrt(p)?"

Why do you keep insisting that the rate is this rate? See above.

>> I’m reasonably convinced that Codel is a near-optimal solution to congestion signalling on TCP-friendly flows.  
> The word "optimal" has a precise meaning. I think you mean simply "I am
> predisposed to CoDel."

I have yet to see one convincing benchmark or paper of a single queued
aqm that outperforms fq_codel.

We've done thousands over here. And shared the test data. And the tool.
What you got?

> Whether the control law increases p linearly with time or by the sqrt,
> or by any function of time, is not the point anyway. Time is not the
> correct unit for this control law. The CoDel control law has no
> variables in it (other than the point at which it starts) that depend on
> any feature of the traffic. Once the CoDel control law starts, it just
> blindly increases until it reaches a high enough drop level to control
> the traffic. So the higher p needs to be, the longer it takes. And the
> lower p needs to be, the more it will overshoot within a round trip.


> A constant increase in p with time, and no dependency on the traffic is
> just plain wrong.
> No way will that result in anything that anyone could prove was
> "optimal" even if you put caveats around it like "near-optimal" or
> "reasonably convincingly near-optimal".
> Perhaps this helps you to see why claims of near-optimality say more
> about the political or religious zeal of the person making the claim,
> than they do about CoDel itself.

Benchmarks, here. Not zeal.

>> Regrettably, the latter are not the only type of traffic actually found on the Internet.
>> Really, the central assumption of Codel is that each flow requires only one congestion signal event per RTT to cause it to back off.  Codel stops working well on traffic which doesn’t obey that assumption (a linear increase in drop frequency is inadequate to mitigate a flood - you need to work with drop probabilities for that), but it *does* work acceptably well with multiple flows sharing a queue, due to this operating-point search.
> Similarly, isn't the phrase "acceptably well" another way of saying "We
> don't need to consider any other AQM that might be better at handling a
> wider range of load scenarios"? Even though there is already an
> alternative available that increases the drop level dependent on how
> fast the queue is growing.
> This religious belief in a particular technology is not healthy. Please
> can we have some objectivity here.

I really, really feel like this is really going overboard.

It took me this long to reply to this mail because of this bluster.

We have, on this side, published all the code, published every
benchmark, published the vast majority of our test data, and at every
point, shown fq_codel to be the win it is. Cake is a few percentage
points better in some respects. If some technology arrives worth
benchmarking then by god we'll benchmark it.

I don't know how to fall on our sword more than this.

So far as I know, to date, you - and very few on these lists - have
bothered to even try the production quality code we've made readily
available in now dozens of router distributions, and cheap hardware, in
any scenario. But I suspect millions now have. With no complaints. In
fact, near universal praise.

The religion is on your side, bob, not ours. I will always benchmark any
code you give me with the tools I have. My sole mission in life is to
beat bufferbloat in my lifetime.

I have tired of the dialog on these lists because since the bufferbloat
effort started, several billion machines has shipped without decent
queue management,
and I would much rather solve the problems as best I can, for those that
are willing to try the solutions we have developed so far.

- and especially - as we learn from the deployment, what other real
problems exist, to feed forward into the process.

Even then, even with the make-wifi-fast fq_codel and airtime fairness
patches for the ath9k still landing, we will at best, improve the
performance of only a few million more devices, in the hands of early
adopters, next year, a small fraction of the sold total. There are
hundreds of chipsets left to fix.

It's a big Internet. There's room for everyone's ideas on it. I look
forward to seeing yours reach a deployable state.

> Regards
> Bob
>>  - Jonathan Morton
>> _______________________________________________
>> tcpPrague mailing list
> -- 
> ________________________________________________________________
> Bob Briscoe                     
> _______________________________________________
> aqm mailing list