Re: [aqm] Gathering Queue Length Statistics

Dave Taht <dave.taht@gmail.com> Wed, 25 February 2015 23:00 UTC

Return-Path: <dave.taht@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 70CB91A904F for <aqm@ietfa.amsl.com>; Wed, 25 Feb 2015 15:00:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.4
X-Spam-Level:
X-Spam-Status: No, score=-1.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, J_CHICKENPOX_32=0.6, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id stJfayI1B9UR for <aqm@ietfa.amsl.com>; Wed, 25 Feb 2015 15:00:06 -0800 (PST)
Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com [IPv6:2607:f8b0:4003:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ED15C1A904C for <aqm@ietf.org>; Wed, 25 Feb 2015 15:00:05 -0800 (PST)
Received: by mail-oi0-f50.google.com with SMTP id v1so6203655oia.9 for <aqm@ietf.org>; Wed, 25 Feb 2015 15:00:05 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=K1JNbHLagpxkhgnxgkMC1y2WIX4fenL165l7nssIiMA=; b=AL9GMD6yJp3qJ00K/Aj7dPPKF82OZTufEz6HF9dwzquxf0lLJBlio6zLQUywWCJRyV 0CEu3U87eilxj2DUjZamU68nCZFXNN62WhzvWnNH0nnJlUTKqWRBV243t+FS0pRocxjB EFpAp2XxOg25Z4cQT+eTQ7KLLHPVDmbwK8OxPBsmJHRTgFoY4TFGOIoc5rjHpt/ISc8o mUCqFZdi2u4PhOJRvdSCCfyb2bkigndGU4uMcvHL0pZ70WA4js2jil9e3VDH5eTWFVSZ lH3c/T1Ja4cUbLtwmvz5g3468pIN8V3TzOWk2Vo9pda3RUB0SwwVaFoOr5yym6s2Q3+A ilBQ==
MIME-Version: 1.0
X-Received: by 10.202.62.70 with SMTP id l67mr3729603oia.59.1424905205178; Wed, 25 Feb 2015 15:00:05 -0800 (PST)
Received: by 10.202.51.66 with HTTP; Wed, 25 Feb 2015 15:00:05 -0800 (PST)
In-Reply-To: <20150226101512.6bbcc126@redhat.com>
References: <CAA93jw5dPhLmamcSHxD92b50XxmkkZNy28SWMkOSRFzBY3keBA@mail.gmail.com> <1424891800.38947.YahooMailBasic@web141601.mail.bf1.yahoo.com> <20150226101512.6bbcc126@redhat.com>
Date: Wed, 25 Feb 2015 15:00:05 -0800
Message-ID: <CAA93jw7STEyw5227jLu9_COXY_5PQ5+FBPWszXw9wTQknSr1Dw@mail.gmail.com>
From: Dave Taht <dave.taht@gmail.com>
To: Jesper Dangaard Brouer <brouer@redhat.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Archived-At: <http://mailarchive.ietf.org/arch/msg/aqm/Pi7wufG10pz9gD3KCdlLCswOS1s>
Cc: Ryan Doyle <rpdoyle@live.unc.edu>, "aqm@ietf.org" <aqm@ietf.org>, Daniel Havey <dhavey@yahoo.com>
Subject: Re: [aqm] Gathering Queue Length Statistics
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Feb 2015 23:00:08 -0000

On Wed, Feb 25, 2015 at 1:15 PM, Jesper Dangaard Brouer
<brouer@redhat.com> wrote:
> Hi Ryan,
>
> Follow Daniel's advice, just use netperf-wrapper. I wrote an easy setup
> guide for netperf-wrapper here:
>  http://netoptimizer.blogspot.co.nz/2014/09/mini-tutorial-for-netperf-wrapper-setup.html

In looking at your tutorial I note that netperf in svn has added a new
EXTREMELY nice feature (that netperf-wrapper does not use yet),
notably

d@nuc-client:~/git/netperf$ DUMP_TCP_INFO=1 netperf -H 172.21.2.1
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
172.21.2.1 () port 0 AF_INET : demo
tcpi_rto 202000 tcpi_ato 0 tcpi_pmtu 1500 tcpi_rcv_ssthresh 29200
tcpi_rtt 1558 tcpi_rttvar 290 tcpi_snd_ssthresh 52 tpci_snd_cwnd 54
tcpi_reordering 3 tcpi_total_retrans 7
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 87380  16384  16384    10.03     320.94


So I would kind of recommend using the svn version rather than the 2.6
version at this point. (big thanks to eric dumazet for the suggestion)

I also note that netperf-wrapper has a man page, which apparently a
few have missed.

Patches are always welcomed to improve the documentation. Also there
are a whole bunch of new tests that I would like to add (or prefer!
find people to write and add) involving wrapping a few other tools
that are easier to use and more robust than dit-g, which I can write
up specs for up here if anyone is interested... and some new stuff for
wifi.

and I have a new analytical model for looking the bifurcation
characteristics of multiple flows moving in and out of slow start...
(but I will be damned if I publish the core ideas at the moment, or
explain less obliquely, as I hope to finally get a paper out of it in
collaboration with someone willing to do the experiments. :)
Volunteers?


)

> You will love the GUI mode:
>  netperf-wrapper --gui


> Most powerful thing in the GUI is that you can easy "load" result sets
> on-top-of each-other, which allow you to easy compare. And you can save
> these as PNG and add them easily to your report :-)

I so love being able to compare noisy data from multiple test runs...

And I tend to use .svgs or .eps or .pdf rather than .pngs as you
capture more scalable detail that works better at multiple
resolutions. At least some latex tools can deal with that well.

>
> And the secret to (solving) bufferbloat is not the queue length, but
> the time-spend in the queue, which is actually easier to derive ;-)

+10, but it is mildly more tricky than that. ;)

> --
> Best regards,
>   Jesper Dangaard Brouer
>   MSc.CS, Sr. Network Kernel Developer at Red Hat
>   Author of http://www.iptv-analyzer.org
>   LinkedIn: http://www.linkedin.com/in/brouer
>
>
>
> On Wed, 25 Feb 2015 11:16:40 -0800 Daniel Havey <dhavey@yahoo.com> wrote:
>
>> Jeez Dave, don't scare the kid!  :^)  He's only an undergrad!
>>
>> Look Ryan, Dave really does have your best interest at heart and that
>> is why he wants you to use CDFs instead of averages.  And he is
>> right, I think you should use CDFs also.  In fact, save yourself a
>> pile of repetitve work and just use Toke's tools.  They will produce
>> better results and require less work from you :)
>>
>> Trust me on this.  I have done both and Toke's tools are waaaaay
>> better and easier than anything you (or I) will come up with.
>> Install Toke's tools and then you will have a cornucopia of awesome
>> and repeatable results that you can just plug in to your final
>> paper.  Go for it :^)
>>
>> thanxs ;^)
>> ...Daniel
>>
>>
>> --------------------------------------------
>> On Wed, 2/25/15, Dave Taht <dave.taht@gmail.com> wrote:
>>
>>  Subject: Re: [aqm] Gathering Queue Length Statistics
>>  To: "Ryan Doyle" <rpdoyle@live.unc.edu>
>>  Cc: "aqm@ietf.org" <aqm@ietf.org>
>>  Date: Wednesday, February 25, 2015, 10:24 AM
>>
>>  On Wed, Feb 25, 2015 at
>>  9:53 AM, Ryan Doyle <rpdoyle@live.unc.edu>
>>  wrote:
>>  > Hello,
>>  >
>>  > I am a senior undergraduate student at the
>>  University of North Carolina at
>>  > Chapel
>>  Hill and am studying the effectiveness of AQMs. I have set
>>  up a lab
>>  > network and plan on running
>>  different sets of experiments with different
>>  > AQMs. My router machines are running Linux
>>  kernel version 3.16.0.
>>  >
>>  > I am using the fq_codel, codel, and pie
>>  qdiscs for my research and am
>>  > wondering
>>  if there is a way to collect statistics regarding the
>>  average
>>  > queue length since a qdisc was
>>  enabled? I have looked at tc's "-s" flag
>>  for
>>  > statistics, but they show nothing
>>  about queue length and I have been unable
>>  > to find anything else that might help me
>>  get queue length statistics.
>>
>>  Oh, god. I am getting incredibly sensitive
>>  about average queue length,
>>  and I realize
>>  that that is not what you meant. But since not enough
>>  people have seemingly read this or any of the
>>  related materials, here
>>  it is again.
>>
>>  http://www.pollere.net/Pdfdocs/QrantJul06.pdf
>>
>>  And I of course always
>>  recommend van´s talk on the fountain model for
>>  thinking about closed loop servo systems.
>>
>>  http://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos
>>
>>  In the bufferbloat
>>  project...
>>
>>  We have
>>  developed many tools that drive aqms hard, see
>>  netperf-wrapper
>>  on github for that, which
>>  measures e2e delay without requiring any
>>  tools on the routers inbetween. e2e delay in my
>>  mind is way more
>>  important than average
>>  queue length. And you can derive the queue
>>  length(s) from tcp timestamps in those
>>  netperf-wrapper tests, from
>>  additional
>>  packet captures, if you must.
>>
>>  If you absolutely MUST derive average queue
>>  length from the box, you
>>  can poll the
>>  interface frequently with tc -s qdisc show as well as
>>  with ifconfig- and parse out the number of
>>  packets and the number of
>>  bytes. But you can
>>  do MUCH more valid statistical analysis than that,
>>  with that sort of data set - and if you poll
>>  too frequently you will
>>  heisenbug your
>>  tests, as those data collection calls take locks that
>>  interfere with the path. and we have all sorts
>>  of advice about traps
>>  for the unwary
>>  here:
>>
>>  http://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel
>>
>>
>>  Please use
>>  things like CDFs to see the range of delays, rather than
>>  averages. It is what happens at above 90% of
>>  the range that makes
>>  bufferbloat maddening
>>  to ordinary users.
>>
>>  I am
>>  summarily rejecting any papers that I review that report
>>  average
>>  queue length as if it meant
>>  anything. And for a few other reasons. You
>>  have been warned. I really lost my temper after
>>  the last paper I
>>  reviewed last weekend and
>>  the resulting flamage is all over the bloat
>>  list and codel lists, and starts here:
>>
>>  https://lists.bufferbloat.net/pipermail/codel/2015-February/000872.html
>>
>>
>>  > Best,
>>  > Ryan Doyle
>>  >
>>  >
>>  _______________________________________________
>>  > aqm mailing list
>>  > aqm@ietf.org
>>  > https://www.ietf.org/mailman/listinfo/aqm
>>  >
>>
>>
>>
>>  --
>>  Dave
>>  Täht
>>  Let's make wifi fast, less jittery
>>  and reliable again!
>>
>>  https://plus.google.com/u/0/107942175615993706558/posts/TVX3o84jjmb
>>
>>  _______________________________________________
>>  aqm mailing list
>>  aqm@ietf.org
>>  https://www.ietf.org/mailman/listinfo/aqm
>>
>>
>> _______________________________________________
>> aqm mailing list
>> aqm@ietf.org
>> https://www.ietf.org/mailman/listinfo/aqm
>
>
>
>
> --
> Best regards,
>   Jesper Dangaard Brouer
>   MSc.CS, Sr. Network Kernel Developer at Red Hat
>   Author of http://www.iptv-analyzer.org
>   LinkedIn: http://www.linkedin.com/in/brouer



-- 
Dave Täht
Let's make wifi fast, less jittery and reliable again!

https://plus.google.com/u/0/107942175615993706558/posts/TVX3o84jjmb