IETF Logo Mail Archive

Re: [bmwg] Mean vs Median

Paul Emmerich <emmericp@net.in.tum.de> Thu, 12 November 2015 16:55 UTC

Return-Path: <emmericp@net.in.tum.de>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 565F71ACDC1 for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 08:55:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.85
X-Spam-Level:
X-Spam-Status: No, score=-3.85 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iHukTmii4Sqz for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 08:55:53 -0800 (PST)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7AC9D1B2B25 for <bmwg@ietf.org>; Thu, 12 Nov 2015 08:55:53 -0800 (PST)
Received: from dyn94st.net.in.tum.de (dyn94st.net.in.tum.de [131.159.14.94]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 422AB1883002 for <bmwg@ietf.org>; Thu, 12 Nov 2015 17:55:51 +0100 (CET)
To: bmwg@ietf.org
References: <6b20c5aba195.56384250@naist.jp> <6a60e1ff9170.56384780@naist.jp> <6a60f4388bab.563847bc@naist.jp> <6bd0f10697e2.563847f8@naist.jp> <6a409179ad4a.56384835@naist.jp> <6a80cfd8c72d.56384871@naist.jp> <6c30b15ad280.563848ae@naist.jp> <6c30f0e98215.563848ea@naist.jp> <6c10c39aeff9.56384926@naist.jp> <6ab08659b996.56384963@naist.jp> <6ab0ea4dfdd6.563849a0@naist.jp> <6ab0be62e098.563849dc@naist.jp> <6aa0abb5b14b.56384a19@naist.jp> <6aa0e679a9c8.56384a55@naist.jp> <6b60e1babb96.56384a93@naist.jp> <6b60fdd88897.56384acf@naist.jp> <6a509431f711.56384c39@naist.jp> <6a50aab7bf13.5638cb72@naist.jp> <CAPrseCo-E82O+tSvRC=4x-yXYTMEHUW6UjeQK6HBRZwXey=sKg@mail.gmail.com> <9C1BEDBD-2338-4E1B-8C98-E9479FE01423@is.naist.jp> <56434C78.6090502@net.in.tum.de> <CAPrseCqY1FFQv8yuASVC5xMYQ7w4+KQCnMhE1cfV7Bjtowovqg@mail.gmail.com> <5644 8E4C.9060307@n et.in.tum.de> <!&!AAAAAAAAAAAuAAAAAAAAAEFA/wpwMJJMh/vN2IEkLQQBANlTnCJhprtFudq2LHCBs8EBACQA//8AABAAAACF68SkjwoLQZ5jYexomO8NAQAAAAA=@is.naist.jp>
From: Paul Emmerich <emmericp@net.in.tum.de>
Message-ID: <5644C496.5050901@net.in.tum.de>
Date: Thu, 12 Nov 2015 17:55:50 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <!&!AAAAAAAAAAAuAAAAAAAAAEFA/wpwMJJMh/vN2IEkLQQBANlTnCJhprtFudq2LHCBs8EBACQA//8AABAAAACF68SkjwoLQZ5jYexomO8NAQAAAAA=@is.naist.jp>
Content-Type: text/plain; charset="windows-1252"; format="flowed"
Content-Transfer-Encoding: 7bit
Archived-At: <http://mailarchive.ietf.org/arch/msg/bmwg/oYTgeDF3T6XTMNxyQ6QA5AD7J4w>
Subject: Re: [bmwg] Mean vs Median
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Nov 2015 16:55:55 -0000

On 12.11.15 16:19, Marius Georgescu wrote:
> [MG]  Thanks for sharing your paper. I am not sure if this solution would
> make a good comparison base. As I see it, the test report has to be
> synthetic enough to allow easy comparison. Reporting a single number might
> not be enough, but reporting 10 numbers is too much imo.

Maybe the latency report can be split into two categories: typical 
latency and worst-case latency. The former being something like average 
and standard deviation (or median and 1st/3rd quartile), the latter the 
90th and 99th percentile? A full test report should (optionally?) 
include the full CDF or histogram as a graph.


  Paul

v2.23.0 | Report a Bug | By Email