IETF Logo Mail Archive

Re: [bmwg] Mean vs Median

Marius Georgescu <liviumarius-g@is.naist.jp> Fri, 13 November 2015 00:57 UTC

Return-Path: <liviumarius-g@is.naist.jp>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7BB9C1B3B7D for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 16:57:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.102
X-Spam-Level:
X-Spam-Status: No, score=-0.102 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2nEt8bcts6j2 for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 16:57:51 -0800 (PST)
Received: from mailrelay22.naist.jp (mailrelay22.naist.jp [163.221.80.91]) by ietfa.amsl.com (Postfix) with ESMTP id 4EA601B3B74 for <bmwg@ietf.org>; Thu, 12 Nov 2015 16:57:51 -0800 (PST)
Received: from mailpost22.naist.jp (mailscan22.naist.jp [163.221.80.59]) by mailrelay22.naist.jp (Postfix) with ESMTP id 6414391F for <bmwg@ietf.org>; Fri, 13 Nov 2015 09:57:49 +0900 (JST)
Received: from naist-wavenet125-047.naist.jp (naist-wavenet125-047.naist.jp [163.221.125.47]) by mailpost22.naist.jp (Postfix) with ESMTPSA id 4FF5491E for <bmwg@ietf.org>; Fri, 13 Nov 2015 09:57:49 +0900 (JST)
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
From: Marius Georgescu <liviumarius-g@is.naist.jp>
In-Reply-To: <4AF73AA205019A4C8A1DDD32C034631D0F294857BF@NJFPSRVEXG0.research.att.com>
Date: Fri, 13 Nov 2015 09:57:26 +0900
Content-Transfer-Encoding: quoted-printable
Message-Id: <A0FFD6C2-0325-4E81-8C84-5D81DEB80F58@is.naist.jp>
References: <6b20c5aba195.56384250@naist.jp> <6a60e1ff9170.56384780@naist.jp> <6a60f4388bab.563847bc@naist.jp> <6bd0f10697e2.563847f8@naist.jp> <6a409179ad4a.56384835@naist.jp> <6a80cfd8c72d.56384871@naist.jp> <6c30b15ad280.563848ae@naist.jp> <6c30f0e98215.563848ea@naist.jp> <6c10c39aeff9.56384926@naist.jp> <6ab08659b996.56384963@naist.jp> <6ab0ea4dfdd6.563849a0@naist.jp> <6ab0be62e098.563849dc@naist.jp> <6aa0abb5b14b.56384a19@naist.jp> <6aa0e679a9c8.56384a55@naist.jp> <6b60e1babb96.56384a93@naist.jp> <6b60fdd88897.56384acf@naist.jp> <6a509431f711.56384c39@naist.jp> <6a50aab7bf13.5638cb72@naist.jp> <CAPrseCo-E82O+tSvRC=4x-yXYTMEHUW6UjeQK6HBRZwXey=sKg@mail.gmail.com> <9C1BEDBD-2338-4E1B-8C98-E9479FE01423@is.naist.jp> <56434C78.6090502@net.in.tum.de> <CAPrseCqY1FFQv8yuASVC5xMYQ7w4+KQCnMhE1cfV7Bjtowovqg@mail.gmail.com> <"5644 8E4C.9060307"@n> <et.in.tum.de@naist.jp> <!&!AAAAAAAAAAAuAAAAAAAAAEFA/wpwMJJMh/vN2IEkLQQBANlTnCJhprtFudq2LHCBs8EBACQA//8AABAAAACF68SkjwoLQZ5jYexomO8NAQAAAAA=@is.na ist.jp> <5644C496.5050901@net.in.tum.de> <4AF73AA205019A4C8A1DDD32C034631D0F294857BF@NJFPSRVEXG0.research.att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
X-Mailer: Apple Mail (2.2104)
X-TM-AS-MML: No
X-TM-AS-Product-Ver: IMSS-7.1.0.1392-8.0.0.1202-21938.003
X-TM-AS-Result: No--5.992-5.0-31-10
X-imss-scan-details: No--5.992-5.0-31-10
X-TMASE-MatchedRID: I5Fk1qs03HaPvrMjLFD6eB5+URxv1WlBGcfGM6EiL4aqvcIF1TcLYI+1 bntEYE/11Fc61VCGvh1cQYBu5oPw5Xz5lEEBuvacjZlmG3r9xqgS12tj9Zvd86W8iLE/8mvBlH6 REwfgTaNtIWcaVVtscD53gHGvjFuhto7DfGoLirwliarNEjJ/QQqjkC/M03hzbcPp/oilssilQa x3ye0WRS09lkVV8miASvTsMFqiMb1PDPfmo+ftx3PkIViiTNODirCcxxL1pV3McZL/toKwUg/Wp 5XFtOkbo4xrv9ku/fVftuJwrFEhTbew1twePJJBMM4ioayl4t151TqU12Mb2uloYgzjonrsxlbl qLlYqXK6w1pVBdgBaGvw3ttTwo+UhhFGtjF/H6doZ9I02DKqaPXu8yUtznWan9XJO8DXhU3t8+1 IilvgLJruBQvCa2AAvPFqlSVYiWWpezT1GcXxKeGducDKWycfemWGoyQu3Xt1VcR45IUWT9rm0q I/WtFn
Archived-At: <http://mailarchive.ietf.org/arch/msg/bmwg/B0M7xKWkCdgftE1HXhfg2rKAMxE>
Subject: Re: [bmwg] Mean vs Median
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Nov 2015 00:57:52 -0000




> On Nov 13, 2015, at 03:32, MORTON, ALFRED C (AL) <acmorton@att.com> wrote:
> 
>> -----Original Message-----
>> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Paul Emmerich
>> Sent: Thursday, November 12, 2015 11:56 AM
>> To: bmwg@ietf.org
>> Subject: Re: [bmwg] Mean vs Median
>> 
>> On 12.11.15 16:19, Marius Georgescu wrote:
>>> [MG]  Thanks for sharing your paper. I am not sure if this solution
>>> would make a good comparison base. As I see it, the test report has to
>>> be synthetic enough to allow easy comparison. Reporting a single
>>> number might not be enough, but reporting 10 numbers is too much imo.
>> 
>> Maybe the latency report can be split into two categories: typical
>> latency and worst-case latency. The former being something like average
>> and standard deviation (or median and 1st/3rd quartile), the latter the
>> 90th and 99th percentile? A full test report should (optionally?)
>> include the full CDF or histogram as a graph.
>> 
>> 
> [ACM] 
> I have been suggesting (in many places and drafts) that the "worst case"
> delay should be characterized with RFC5481 PDV, so that it is a view of
> delay variation and more easily comparable across test runs. The High
> percentile as a single measure of PDV is built-in, and of course the
> CDF or histogram is a possibility:
> https://tools.ietf.org/html/rfc5481#section-4.2
MG: I like the idea of using the 99.9th percentile as a single measure for the “Worst case” latency.

v2.23.0 | Report a Bug | By Email