Re: [aqm] Gathering Queue Length Statistics
Jesper Dangaard Brouer <brouer@redhat.com> Wed, 25 February 2015 21:15 UTC
Return-Path: <brouer@redhat.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2D8761A8775 for <aqm@ietfa.amsl.com>; Wed, 25 Feb 2015 13:15:30 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.012
X-Spam-Level:
X-Spam-Status: No, score=-6.012 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_32=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qmy8wRwlyd31 for <aqm@ietfa.amsl.com>; Wed, 25 Feb 2015 13:15:27 -0800 (PST)
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 765801A88A7 for <aqm@ietf.org>; Wed, 25 Feb 2015 13:15:21 -0800 (PST)
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1PLFJmg001127 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 25 Feb 2015 16:15:19 -0500
Received: from localhost (ovpn-112-47.phx2.redhat.com [10.3.112.47]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1PLFEK6004538; Wed, 25 Feb 2015 16:15:15 -0500
Date: Thu, 26 Feb 2015 10:15:12 +1300
From: Jesper Dangaard Brouer <brouer@redhat.com>
To: Daniel Havey <dhavey@yahoo.com>, Toke Høiland-Jørge nsen <toke@toke.dk>
Message-ID: <20150226101512.6bbcc126@redhat.com>
In-Reply-To: <1424891800.38947.YahooMailBasic@web141601.mail.bf1.yahoo.com>
References: <CAA93jw5dPhLmamcSHxD92b50XxmkkZNy28SWMkOSRFzBY3keBA@mail.gmail.com> <1424891800.38947.YahooMailBasic@web141601.mail.bf1.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
Archived-At: <http://mailarchive.ietf.org/arch/msg/aqm/HW7QJGiZZQL8r5OjUssJz02GJHM>
Cc: Ryan Doyle <rpdoyle@live.unc.edu>, "aqm@ietf.org" <aqm@ietf.org>, Dave Taht <dave.taht@gmail.com>, brouer@redhat.com
Subject: Re: [aqm] Gathering Queue Length Statistics
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Feb 2015 21:15:30 -0000
Hi Ryan, Follow Daniel's advice, just use netperf-wrapper. I wrote an easy setup guide for netperf-wrapper here: http://netoptimizer.blogspot.co.nz/2014/09/mini-tutorial-for-netperf-wrapper-setup.html You will love the GUI mode: netperf-wrapper --gui Most powerful thing in the GUI is that you can easy "load" result sets on-top-of each-other, which allow you to easy compare. And you can save these as PNG and add them easily to your report :-) And the secret to (solving) bufferbloat is not the queue length, but the time-spend in the queue, which is actually easier to derive ;-) -- Best regards, Jesper Dangaard Brouer MSc.CS, Sr. Network Kernel Developer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer On Wed, 25 Feb 2015 11:16:40 -0800 Daniel Havey <dhavey@yahoo.com> wrote: > Jeez Dave, don't scare the kid! :^) He's only an undergrad! > > Look Ryan, Dave really does have your best interest at heart and that > is why he wants you to use CDFs instead of averages. And he is > right, I think you should use CDFs also. In fact, save yourself a > pile of repetitve work and just use Toke's tools. They will produce > better results and require less work from you :) > > Trust me on this. I have done both and Toke's tools are waaaaay > better and easier than anything you (or I) will come up with. > Install Toke's tools and then you will have a cornucopia of awesome > and repeatable results that you can just plug in to your final > paper. Go for it :^) > > thanxs ;^) > ...Daniel > > > -------------------------------------------- > On Wed, 2/25/15, Dave Taht <dave.taht@gmail.com> wrote: > > Subject: Re: [aqm] Gathering Queue Length Statistics > To: "Ryan Doyle" <rpdoyle@live.unc.edu> > Cc: "aqm@ietf.org" <aqm@ietf.org> > Date: Wednesday, February 25, 2015, 10:24 AM > > On Wed, Feb 25, 2015 at > 9:53 AM, Ryan Doyle <rpdoyle@live.unc.edu> > wrote: > > Hello, > > > > I am a senior undergraduate student at the > University of North Carolina at > > Chapel > Hill and am studying the effectiveness of AQMs. I have set > up a lab > > network and plan on running > different sets of experiments with different > > AQMs. My router machines are running Linux > kernel version 3.16.0. > > > > I am using the fq_codel, codel, and pie > qdiscs for my research and am > > wondering > if there is a way to collect statistics regarding the > average > > queue length since a qdisc was > enabled? I have looked at tc's "-s" flag > for > > statistics, but they show nothing > about queue length and I have been unable > > to find anything else that might help me > get queue length statistics. > > Oh, god. I am getting incredibly sensitive > about average queue length, > and I realize > that that is not what you meant. But since not enough > people have seemingly read this or any of the > related materials, here > it is again. > > http://www.pollere.net/Pdfdocs/QrantJul06.pdf > > And I of course always > recommend van´s talk on the fountain model for > thinking about closed loop servo systems. > > http://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos > > In the bufferbloat > project... > > We have > developed many tools that drive aqms hard, see > netperf-wrapper > on github for that, which > measures e2e delay without requiring any > tools on the routers inbetween. e2e delay in my > mind is way more > important than average > queue length. And you can derive the queue > length(s) from tcp timestamps in those > netperf-wrapper tests, from > additional > packet captures, if you must. > > If you absolutely MUST derive average queue > length from the box, you > can poll the > interface frequently with tc -s qdisc show as well as > with ifconfig- and parse out the number of > packets and the number of > bytes. But you can > do MUCH more valid statistical analysis than that, > with that sort of data set - and if you poll > too frequently you will > heisenbug your > tests, as those data collection calls take locks that > interfere with the path. and we have all sorts > of advice about traps > for the unwary > here: > > http://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel > > > Please use > things like CDFs to see the range of delays, rather than > averages. It is what happens at above 90% of > the range that makes > bufferbloat maddening > to ordinary users. > > I am > summarily rejecting any papers that I review that report > average > queue length as if it meant > anything. And for a few other reasons. You > have been warned. I really lost my temper after > the last paper I > reviewed last weekend and > the resulting flamage is all over the bloat > list and codel lists, and starts here: > > https://lists.bufferbloat.net/pipermail/codel/2015-February/000872.html > > > > Best, > > Ryan Doyle > > > > > _______________________________________________ > > aqm mailing list > > aqm@ietf.org > > https://www.ietf.org/mailman/listinfo/aqm > > > > > > -- > Dave > Täht > Let's make wifi fast, less jittery > and reliable again! > > https://plus.google.com/u/0/107942175615993706558/posts/TVX3o84jjmb > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm > > > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm -- Best regards, Jesper Dangaard Brouer MSc.CS, Sr. Network Kernel Developer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer
- [aqm] Gathering Queue Length Statistics Ryan Doyle
- Re: [aqm] Gathering Queue Length Statistics Fred Baker (fred)
- Re: [aqm] Gathering Queue Length Statistics Dave Taht
- Re: [aqm] Gathering Queue Length Statistics Daniel Havey
- Re: [aqm] Gathering Queue Length Statistics Jesper Dangaard Brouer
- Re: [aqm] Gathering Queue Length Statistics Dave Taht
- Re: [aqm] Gathering Queue Length Statistics grenville armitage
- Re: [aqm] Gathering Queue Length Statistics Ryan Doyle
- Re: [aqm] Gathering Queue Length Statistics Mikael Abrahamsson
- Re: [aqm] Gathering Queue Length Statistics Toke Høiland-Jørgensen
- [aqm] Is bufferbloat a real problem? Daniel Havey
- Re: [aqm] Is bufferbloat a real problem? Curtis Villamizar
- Re: [aqm] Is bufferbloat a real problem? Daniel Havey
- Re: [aqm] Is bufferbloat a real problem? Daniel Havey
- Re: [aqm] Is bufferbloat a real problem? David Collier-Brown
- Re: [aqm] Is bufferbloat a real problem? KK
- Re: [aqm] Is bufferbloat a real problem? David Collier-Brown
- Re: [aqm] Is bufferbloat a real problem? Mikael Abrahamsson
- Re: [aqm] Is bufferbloat a real problem? Curtis Villamizar
- Re: [aqm] Is bufferbloat a real problem? Curtis Villamizar
- Re: [aqm] Is bufferbloat a real problem? Curtis Villamizar