IETF Logo Mail Archive

Re: [bmwg] Mean vs Median

"MORTON, ALFRED C (AL)" <acmorton@att.com> Thu, 12 November 2015 18:32 UTC

Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83D4C1B2D8E for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 10:32:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.211
X-Spam-Level:
X-Spam-Status: No, score=-4.211 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2LzyDm2R1GGp for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 10:32:50 -0800 (PST)
Received: from mail-pink.research.att.com (mail-pink.research.att.com [204.178.8.22]) by ietfa.amsl.com (Postfix) with ESMTP id 79A481B2ABF for <bmwg@ietf.org>; Thu, 12 Nov 2015 10:32:50 -0800 (PST)
Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id 4CA4F121C4E; Thu, 12 Nov 2015 13:33:45 -0500 (EST)
Received: from exchange.research.att.com (njfpsrvexg0.research.att.com [135.207.255.124]) by mail-blue.research.att.com (Postfix) with ESMTP id 11C7CF04F1; Thu, 12 Nov 2015 13:32:50 -0500 (EST)
Received: from NJFPSRVEXG0.research.att.com ([fe80::108a:1006:9f54:fd90]) by NJFPSRVEXG0.research.att.com ([fe80::108a:1006:9f54:fd90%25]) with mapi; Thu, 12 Nov 2015 13:32:50 -0500
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Paul Emmerich <emmericp@net.in.tum.de>, "bmwg@ietf.org" <bmwg@ietf.org>
Date: Thu, 12 Nov 2015 13:32:48 -0500
Thread-Topic: [bmwg] Mean vs Median
Thread-Index: AdEdawkoDeTn28KbSTiUfuOFExAyqQADIXwQ
Message-ID: <4AF73AA205019A4C8A1DDD32C034631D0F294857BF@NJFPSRVEXG0.research.att.com>
References: <6b20c5aba195.56384250@naist.jp> <6a60e1ff9170.56384780@naist.jp> <6a60f4388bab.563847bc@naist.jp> <6bd0f10697e2.563847f8@naist.jp> <6a409179ad4a.56384835@naist.jp> <6a80cfd8c72d.56384871@naist.jp> <6c30b15ad280.563848ae@naist.jp> <6c30f0e98215.563848ea@naist.jp> <6c10c39aeff9.56384926@naist.jp> <6ab08659b996.56384963@naist.jp> <6ab0ea4dfdd6.563849a0@naist.jp> <6ab0be62e098.563849dc@naist.jp> <6aa0abb5b14b.56384a19@naist.jp> <6aa0e679a9c8.56384a55@naist.jp> <6b60e1babb96.56384a93@naist.jp> <6b60fdd88897.56384acf@naist.jp> <6a509431f711.56384c39@naist.jp> <6a50aab7bf13.5638cb72@naist.jp> <CAPrseCo-E82O+tSvRC=4x-yXYTMEHUW6UjeQK6HBRZwXey=sKg@mail.gmail.com> <9C1BEDBD-2338-4E1B-8C98-E9479FE01423@is.naist.jp> <56434C78.6090502@net.in.tum.de> <CAPrseCqY1FFQv8yuASVC5xMYQ7w4+KQCnMhE1cfV7Bjtowovqg@mail.gmail.com> <5644 8E4C.9060307@n et.in.tum.de> <!&!AAAAAAAAAAAuAAAAAAAAAEFA/wpwMJJMh/vN2IEkLQQBANlTnCJhprtFudq2LHCBs8EBACQA//8AABAAAACF68SkjwoLQZ5jYexomO8NAQAAAAA=@is.naist.jp> <5644C496.5050901@net.in.tum.de>
In-Reply-To: <5644C496.5050901@net.in.tum.de>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: <http://mailarchive.ietf.org/arch/msg/bmwg/ILYWrRpHl0cWGnUs_a6rnqeAj5M>
Subject: Re: [bmwg] Mean vs Median
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Nov 2015 18:32:55 -0000

> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Paul Emmerich
> Sent: Thursday, November 12, 2015 11:56 AM
> To: bmwg@ietf.org
> Subject: Re: [bmwg] Mean vs Median
> 
> On 12.11.15 16:19, Marius Georgescu wrote:
> > [MG]  Thanks for sharing your paper. I am not sure if this solution
> > would make a good comparison base. As I see it, the test report has to
> > be synthetic enough to allow easy comparison. Reporting a single
> > number might not be enough, but reporting 10 numbers is too much imo.
> 
> Maybe the latency report can be split into two categories: typical
> latency and worst-case latency. The former being something like average
> and standard deviation (or median and 1st/3rd quartile), the latter the
> 90th and 99th percentile? A full test report should (optionally?)
> include the full CDF or histogram as a graph.
> 
> 
[ACM] 
I have been suggesting (in many places and drafts) that the "worst case"
delay should be characterized with RFC5481 PDV, so that it is a view of
delay variation and more easily comparable across test runs. The High
percentile as a single measure of PDV is built-in, and of course the
CDF or histogram is a possibility:
https://tools.ietf.org/html/rfc5481#section-4.2

Al
(as a participant)

v2.23.0 | Report a Bug | By Email