Re: [aqm] RED implementation on Linux 3.14.22

[Dave Taht <[e-mail]>]

On Mon, Dec 15, 2014 at 5:41 AM, Jim Gettys <[e-mail]> wrote:
>
>
> On Mon, Dec 15, 2014 at 2:51 AM, Simone Ferlin-Oliveira <[e-mail]>
> wrote:
>>
>> All,
>>
>> I am doing some work with shared bottleneck detection that requires
>> some evaluation with different AQM, in particular, RED. Since I
>> haven't been following the evolution of the implementation,  I would
>> like to ask about your experience with the code on Linux 3.14 (and
>> newer).
>
>
> I know that Dave Taht ran into bugs in RED a while back, which I believe
> have been fixed for quite a while.

The power of git to answer questions like this is unparalleled. Taking
a look at my current kernel tree and doing a:

git log net/sched/sch_red.c

shows eric fixed 2 bugs in Linux RED in

commit 1ee5fa1e9970a16036e37c7b9d5ce81c778252fc
Author: Eric Dumazet <[e-mail]>
Date:   Thu Dec 1 11:06:34 2011 +0000

   sch_red: fix red_change

...

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=1ee5fa1e9970a16036e37c7b9d5ce81c778252fc

ARED was added slightly later, and sfqred (a first attempt at blending
fq + AQM together) shortly after that. (which doesn't do ared (never
made it to mainline as fq_codel landed soon after that)

Advice: Keep track of net-next, do git pulls regularly, and watch "git
log net" for changes.

> You should always be looking at whether code has been patched in the current
> kernel.org system for a module like that you are interested in, so do a diff
> between 3.14 and the current Linux system. 3.14 is recent enough that it may
> be viable for experiments, for the time being.  Planning to keep up with
> Linux development is wise long term in any case, as the rate of
> improvement/change in the networking stack is very high at the moment as
> draining the bufferbloat swamp and other performance work continues.

Important changes since 3.14: pie added, DCTCP added, gso/tso offloads
seriously reworked and made gentler, sch_fq's pacing improved. The
last kernel rounds (3.18,3.19) were seriously productive: hystart
improved at longer RTTs, still more TSO/gso improvements, and
xmit_more support was added for some devices. Also support for per
route congestion control settings (primarily targetted at DCTCP) was
just added.

I believe some of the long RTT falloff we saw in toke's paper was due
to hystart issues, as I have been unable to duplicate some of his
results with this upcoming release.

I have basically thrown out all my 3.14 results at this point and am
starting over with the soon-to-stablize 3.19 release. (Well, in fact,
I ended up starting over 3 times in the last 2 months as each of the
new features above landed in the kernel) (but as for red, no changes
except in the underlying TCPs and device drivers)

Relevant commits were:

Hystart change:

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=42eef7a0bb0989cd50d74e673422ff98a0ce4d7b

xmit_more:

http://netoptimizer.blogspot.com/2014/10/unlocked-10gbps-tx-wirespeed-smallest.html
very good lwn article on it http://lwn.net/Articles/615238/

one of several GSO fixes:

commit d649a7a81f3b5bacb1d60abd7529894d8234a666
Author: Eric Dumazet <[e-mail]>
Date:   Thu Nov 13 09:45:22 2014 -0800

    tcp: limit GSO packets to half cwnd

... etc. Do a git log net. :)

preso that convinced systemd to switch to fq_codel:
http://lwn.net/Articles/616241/

> Also note that underlying device drivers may have (sometimes lots) of
> buffering out of control of the Linux queue discipline.  For Ethernet
> devices, you should ensure that that the drivers have BQL support
> implemented to minimize this buffering.  Other classes of drivers are more
> problematic, and may have lots of buffering to surprise you.

+10 (or rather, -10). It's up to 25 devices now. I note that TSO/GSO used to
interact very badly with soft rate limiting (htb), it seems better now.

>
> Also be aware that ethernet flow control may move the bottleneck from where
> you expect to somewhere else, and that switches in networks also have to be
> well understood.  Most consumer switches have this *on* by default, and
> mixed 1G/100Mb networks can be particularly "entertaining" in this regard.
> Cable modems, unfortunately, typically do not implement flow control, but
> some DSL modems do (putting the bottleneck into your router, rather than in
> the modem).

I should probably put red back into my test matrixes. I stopped
benchmarking it and pfifo_fast a long time ago. A netperf-wrapper data
set that predates the hystart fix, testing 3 RTTs:

http://snapon.lab.bufferbloat.net/~d/comprehensive.puck/
or:
http://snapon.lab.bufferbloat.net/~d/comprehensive_puck.tgz


>>
>> *Any* help is appreciated.
>
>
> Hope this helps.
>
>>
>>
>> Thanks,
>> Simone
>>
>> _______________________________________________
>> aqm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/aqm



-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks