Re: [aqm] analysis paper on PIE...

[Dave Taht <[e-mail]>]

On Wed, Nov 12, 2014 at 4:03 PM, Scheffenegger, Richard <[e-mail]> wrote:
> Hi Martin,
>
> I believe these papers may qualify that requirement:
>
> http://ipv6.cablelabs.com/wp-content/uploads/2014/06/DOCSIS-AQM_May2014.pdf

This documents the docsis-pie implementation which has rather a few
basic improvements on pie by itself, notably bytemode, some predictive
stuff, and drop-semi-derandomization.

It also uses overlarge and not-recomended-by-the-inventors constants
for codel and sfq_codel, and lumps together all results at all
bandwidths, where, as we've shown, current aqm implementations perform
differently at different bandwidths and RTTs.

The earlier paper covered the bandwidth scenarios more broadly and in
depth, with less twiddling of the constants.

I also had a very long post on this list going into the problems with
the testing done here. which I'll search for unless someone beats me
to it.

Some of the tools used in this evaluation landed in ns2 earlier this
year, and I would certainly like these tests reproduced independently,
with sane values for codel and fq_codel, and preferably against a
simulator I trust more, like ns3.

> http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6925768

Yep. Toke points to codel's falloff at higher rates in his paper also
- this is mostly due to a problem in the control law introduced in the
linux version that isn't in the ns2 version, and is nearly invisible
in the fq_codel version. I do fear a similar problem is in PIE when
dealing with TSO packets but have not tested.

And the load transients are a problem with any straight aqm system.

> https://www.duo.uio.no/handle/10852/37381

I have picked apart this paper elsewhere also. I would have liked it,
if in particular, fq_codel had been used throughout the tests in
comparison, particularly with ecn....

To add to the comparisons:

http://caia.swin.edu.au/reports/140630A/CAIA-TR-140630A.pdf

While quite good, this was kind of limited to steady state
performance, and at rates below 10mbit.

I was delighted to see someone actually use cerowrt for it's intended
purpose, evaluating
each new algorithm on real hardware in their bachelor's thesis, but I
can't find that url right now.

> tl;dr - both pie and codel camps did some independent implementations and testing of the respective other algorithm,

I tracked codel, fq_codel, pie, improvements to red (ared), sfq,
sfqred, and SFB closely for the past 4 years. All of these are in
linux 3.14 or later (with pie entering last). The ability to do basic
testing of everything on the table is a download away, on nearly every
linux distro, and cerowrt has 'em all - and openwrt several...

I spent GSOC2014 getting ns3 up to speed. For some reason I don't seem
to have any time to write papers myself...

>with discussions and denting out some poorly described aspects in that process. It's my understanding that this lead to a better quality of the drafts in both instances.

Ya know, I'm not part of the codel or pie "camp". I'm pretty firmly in
the low knobs "fq + an aqm with ecn support" camp.There aren't a lot
of people in a codel-only camp. The algorithm is a toolkit, much like
pie is a toolkit for docsis-pie, and setting up the debate as codel vs
pie feels like an exercise in dialectical dualism for sake of
excluding the third alternative.

Certainly codel the algorithm, and codel the stand-alone aqm can be
improved (and I have patches for that available for a long time now)
but it has taken a long time to have an adaquate suite of test tools
to be able to analyze the often microscopic differences between
versions of the base aqm algorithms, be the pie or codel or red
derived. (for example I was unaware until I got a preview of toke's
paper of the degree of decline in effectiveness in codel alone at
100Mbit, being focused primarily on finding improvements at 5mbit or
below (the speed most of the edge of the internet runs at - and the
hardware I use to test peaking out at about 60mbits before htb rolls
over and dies on a cpu designed in 1989. So instead of fiddling with
codel I've been fiddling with a better rate shaper embedded into
fq_codel)

I do certainly hope that work can move forward on the evaluation
guidelines based on a wide variety of scenarios. In my own case, my
biases are towards managing slow start better (vs steady state TCPs),
capturing all the latency sources (e.g DNS, tcp syns), and enabling
voip, gaming, web, and videoconferencing traffic better to the expense
of full single flow goodput, at rates well below 100mbit, and
typically at baseline physical rtts in the 4-50ms range.


>

I utterly agree that more testing is needed. However the only open
source test suite (netperf-wrapper) only implements a few of the tests
in the pie and docsis-pie papers, making reproduction difficult, and
the earlier
>
>
> Richard Scheffenegger
>
>
>
>
>> -----Original Message-----
>> From: aqm [mailto:[e-mail]] On Behalf Of Martin Stiemerling
>> Sent: Mittwoch, 12. November 2014 13:47
>> To: [e-mail]
>> Subject: Re: [aqm] analysis paper on PIE...
>>
>> [writing not as AD, but as random IETF participant]
>>
>> Sorry for this blunt question:
>>
>> Is there any other analysis made by an independent source, i.e., where not
>> the PIE authors are running an analysis of PIE?
>>
>> Thanks,
>>
>>    Martin
>>
>> Am 10.11.14 um 21:15 schrieb Rong Pan (ropan):
>> > Please see our analysis paper on PIE...
>> >
>> > Thanks,
>> >
>> > Rong
>> >
>> >
>> >
>> > _______________________________________________
>> > aqm mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/aqm
>> >
>>
>> _______________________________________________
>> aqm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/aqm
>
> _______________________________________________
> aqm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/aqm



-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks