Re: [aqm] Last Call: <draft-ietf-aqm-fq-codel-05.txt> (FlowQueue-Codel) to Experimental RFC

Bob Briscoe <> Wed, 23 March 2016 19:37 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 081E5127058; Wed, 23 Mar 2016 12:37:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id d9-nQwJXbQhT; Wed, 23 Mar 2016 12:37:37 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 9DE6B12D7F4; Wed, 23 Mar 2016 12:37:37 -0700 (PDT)
Received: from ([]:59929 helo=[]) by with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.86_1) (envelope-from <>) id 1aiob1-0001CZ-1T; Wed, 23 Mar 2016 19:37:35 +0000
Subject: Re: [aqm] Last Call: <draft-ietf-aqm-fq-codel-05.txt> (FlowQueue-Codel) to Experimental RFC
To: Dave Taht <>
References: <> <> <> <> <>
From: Bob Briscoe <>
Message-ID: <>
Date: Wed, 23 Mar 2016 19:37:34 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
X-Mailman-Approved-At: Thu, 24 Mar 2016 08:05:48 -0700
Cc:, =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= <>,,, Martin Stiemerling <>, "" <>
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: IETF-Discussion <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 23 Mar 2016 19:37:41 -0000


Quick reply for now...

DualQ and DCTCP is a separate issue to this question, because it has 
only just started development.
That's why I only mentioned PIE & FQ_CoDel, which have been maturing for 
some time.

Safe means "without unintended side-effects". The word doesn't alter its 
meaning when the thing in question is really cool in other respects.

If a teleworker is using a VPN for their everyday work from home, and 
every time their kids upload or download an elephant they squeeze all 
the VPN traffic to nearly nothing, that is enough to require a support 
call, and explanation of how to switch FQ_CoDel to a safer (but less 
performant) AQM (such as PIE, say).

This is not an academic argument. There are a few billion people using 
the Internet. Even if 0.01% are hit by one of the listed limitations, 
that's a huge volume of support calls. This is why it is important to 
use the word "safe" precisely. Yes, FQ_CoDel has awesome performance. 
But it is awesome and unsafe in certain circumstances.

If that muddies the marketing message, well then seriously consider 
marketing PIE instead. There is no room for "not invented here" in this 
Otherwise, FQ_CoDel will get bad press later. Then, after riding the 
hype curve of coolness it will fall over the cliff of disillusionment.


On 22/03/16 04:41, Dave Taht wrote:
> I don't even know where to start bob. This part of the language has
> been in the draft for 2 years, and you are the only person to object
> that I can recall.
> It's an experimental RFC. By "safe" we mean that deploying it, within
> the guidelines, won't break anything to any huge extent to enormous
> benefits. "unsafe", for example, would be promoting use of dctcp while
> it still responds incorrectly to packet loss.
> Verses your decades long quest for better variable rate video, we've
> had over a decade of the bufferbloat problem to deal with on all
> traffic, particularly along the edge, and even after solutions started
> to appear in mid 2012, we haven't made a real dent in what's deployed,
> except for the small select group of devs, academics, ISPs, and
> manufacturers willing to try something new.  I'd like to imagine
> things are shifting to the left side of the green line here, but under
> load, most users are still experiencing latency orders of magnitude in
> excess of what can be achieved
> I've been testing the latest generation of wifi APs of late, and the
> "best" of them, under load, in a single direction, has over 2 seconds
> at the lower rates. Applying any of these algorithms to wifi is
> proving hard, and it's where the bottlenecks are shifting to at least
> in my world, where the default download speed is hovering at around
> 75mbit, and wifi starts breaking down long before that is hit.
> ...
> I tore apart that HAS experiment you cited here:
> - where I was, at least, happy
> to see fq_codel handle the onslought of dctcp traffic, gracefully. (It
> makes me nervous to have such tcps loose on the internet where a
> configuration mistake might send that at the wrong people. fq_codel,
> "safe" - not, perhaps, optimal - in the face of dctcp.)
> my key objections to nearly all the experiments on your side are
> non-reproducability, no competing traffic (not even bothering to
> measure web PLT in
> that paper, for example), no competing upload traffic, and no
> inclusion of the typical things that are latency sensitive at all
> (voip, dns, tcp neg, ssl neg, etc).
> with competing download and upload traffic, fq_codel *dramatically*
> improves the responsiveness and utilization of the link, for all
> traffic. Above 5mbits pretty much the only thing that matters for web
> traffic is RTT, the google cite for this is around somewhere.
> I tend to weigh low latency for every other form of traffic...
> today... over marginal improvements in a contrived video download
> scenario someday.
> As for pie vs fq_codel, despite making pie widely available (for
> example, it's shipped in the ubuntu 15.10 release) and just as easily
> configurable in the sqm-scripts as fq_codel and cake are, it's been
> very difficult to get users to stick with it for a decent a/b test.
> This is perhaps the best comparison of the two I know of:
> The best hope I have for a set of subjective QoE experiences, is
> perhaps this summer we'll have real pie and fq_codel capable modems
> and be able to do a blind "taste test". The pie that we have to play
> with is not enough like the draft to trust the results from. We're
> still screwed in that the CMTSes are not fixed and in most of my test
> sites, download bufferbloat is experienced more often than upload, in
> one by a factor of 6000x1, so we need to compare three things -
> default cmts behavior down + pie/fq_codel up. And with the cmts
> throttled down (as we do it today) via htb + pie/fq_codel.
> My core sadness in the flowqueue codel draft is that we spend way more
> time talking about the caveats than the observed benefits.
> In the case of torrent traffic, utp (especially) works well for me, on
> my workloads. I run through a vpn all day without bothering to do
> anything special about it. GF uploads big fat images all day without
> ever bothering me. The gamers on campus are generally happy. (The only
> prioritization I do is dns traffic).
> The HUGE benefit of having the fq-aqm tech in place is how much better
> it is than anything that exists in the field. I cannot remember the
> last non-wifi buffering event I've had on any streaming media, nor the
> last time netflix paused. I can't remember the last time I got annoyed
> in a ssh or mosh shell. I hold videoconferences on a high rate
> conference bridge (60mbit down/8 up) all the time on a 75mbit link
> with "cake" while testing other stuff... but that is perhaps just me
> and a rag-tag group of people that have bothered to actually install
> and configure the software, and their workloads.
> Without more implementations of all these bufferbloat fighting
> technologies, made deployable, with more people trying them, we're not
> going to get anywhere. There's still billions of boxes left to deploy
> and a long tail of deployed gear that will probably last 10 years or
> more.

Bob Briscoe