Re: [bmwg] Mean vs Median

Paul Emmerich <emmericp@net.in.tum.de> Thu, 12 November 2015 12:38 UTC

Return-Path: <emmericp@net.in.tum.de>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3197F1A879D for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 04:38:59 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.85
X-Spam-Level:
X-Spam-Status: No, score=-2.85 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, GB_SUMOF=1, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Md4N8ciaCAbu for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 04:38:57 -0800 (PST)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D9EF51A879C for <bmwg@ietf.org>; Thu, 12 Nov 2015 04:38:56 -0800 (PST)
Received: from dyn94st.net.in.tum.de (dyn94st.net.in.tum.de [131.159.14.94]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 70BE9187FA09 for <bmwg@ietf.org>; Thu, 12 Nov 2015 13:38:54 +0100 (CET)
To: bmwg@ietf.org
References: <6b20c5aba195.56384250@naist.jp> <6a60d6ebaa6a.56384561@naist.jp> <6a80d3baddd6.5638459e@naist.jp> <6aa08a52c1ca.563845da@naist.jp> <6aa09799f4a7.563846ca@naist.jp> <6b60a07c9bbf.56384707@naist.jp> <6c109c80bfc2.56384743@naist.jp> <6a60e1ff9170.56384780@naist.jp> <6a60f4388bab.563847bc@naist.jp> <6bd0f10697e2.563847f8@naist.jp> <6a409179ad4a.56384835@naist.jp> <6a80cfd8c72d.56384871@naist.jp> <6c30b15ad280.563848ae@naist.jp> <6c30f0e98215.563848ea@naist.jp> <6c10c39aeff9.56384926@naist.jp> <6ab08659b996.56384963@naist.jp> <6ab0ea4dfdd6.563849a0@naist.jp> <6ab0be62e098.563849dc@naist.jp> <6aa0abb5b14b.56384a19@naist.jp> <6aa0e679a9c8.56384a55@naist.jp> <6b60e1babb96.56384a93@naist.jp> <6b60fdd88897.56384acf@naist.jp> <6a509431f711.56384c39@naist.jp> <6a50aab7bf13.5638cb72@naist.jp> <CAPrseCo-E82O+tSvRC=4x-yXYTMEHUW6UjeQK6HBRZwXey=sKg@mail.gmail.com> <5640DA91.30502@net.in.tum .de> <9C1BEDBD-2338-4E1B-8C98-E9479FE01423@is.naist.jp> <56434C78.6090502@net.in.tum.de>
From: Paul Emmerich <emmericp@net.in.tum.de>
Message-ID: <5644885E.5050400@net.in.tum.de>
Date: Thu, 12 Nov 2015 13:38:54 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <56434C78.6090502@net.in.tum.de>
Content-Type: text/plain; charset="windows-1252"; format="flowed"
Content-Transfer-Encoding: 7bit
Archived-At: <http://mailarchive.ietf.org/arch/msg/bmwg/mm6mOD_mrf4Iuz1tekqxeFGrjLc>
Subject: Re: [bmwg] Mean vs Median
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Nov 2015 12:38:59 -0000

Hi,

On 10.11.15 02:18, Marius Georgescu wrote:
>  > Can you give us more context (test setup; physical/virtualized
> tester/DUT; one tester/sender_receiver tester ... ) on these measurements?

I also found an example for a non-normally distributed latency on a 
hardware switch.

Test setup: two flows of traffic coming from different ports on the 
switch that are directed to a single output port. The sum of two 
incoming flows is less than the capacity of the outgoing link, so this 
is not an overload scenario but a typical load scenario for a switch.

This setup results in a bimodal latency distribution, each packet either 
has to wait for a short time if there is a packet from the other flow 
currently being transmitted, or it can be transmitted immediately if the 
link is free.

The buffers on the switch are virtually empty as I used constant 
inter-packet spacings.


The histogram of a 1 Gbit/s flow is available at [1]. The second 
competing flow was 4 Gbit/s in this test setup, however, the slower flow 
shown in the graph was prioritized (which doesn't matter as the output 
was not overloaded). Changing the relative rates changes which of the 
two peaks dominates, but the basic building blocks are still the two 
normal distributions.

A median value is often completely meaningless in such a scenario.


Paul

[1] https://www.dropbox.com/s/xbgelt336ade0dx/two-flows-pica8.pdf?dl=1