IETF Logo Mail Archive

Re: [bmwg] Mean vs Median

Marius Georgescu <liviumarius-g@is.naist.jp> Fri, 13 November 2015 00:43 UTC

Return-Path: <liviumarius-g@is.naist.jp>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8ECDF1B3A9F for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 16:43:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.598
X-Spam-Level:
X-Spam-Status: No, score=0.598 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UHEmhGjZUw6w for <bmwg@ietfa.amsl.com>; Thu, 12 Nov 2015 16:43:40 -0800 (PST)
Received: from mailrelay22.naist.jp (mailrelay22.naist.jp [IPv6:2001:200:16a:50::91]) by ietfa.amsl.com (Postfix) with ESMTP id 4EB621B3A3E for <bmwg@ietf.org>; Thu, 12 Nov 2015 16:43:40 -0800 (PST)
Received: from mailpost22.naist.jp (mailscan22.naist.jp [163.221.80.59]) by mailrelay22.naist.jp (Postfix) with ESMTP id 49239893 for <bmwg@ietf.org>; Fri, 13 Nov 2015 09:43:39 +0900 (JST)
Received: from naist-wavenet125-047.naist.jp (naist-wavenet125-047.naist.jp [163.221.125.47]) by mailpost22.naist.jp (Postfix) with ESMTPSA id 340B1892 for <bmwg@ietf.org>; Fri, 13 Nov 2015 09:43:39 +0900 (JST)
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
From: Marius Georgescu <liviumarius-g@is.naist.jp>
In-Reply-To: <5644C496.5050901@net.in.tum.de>
Date: Fri, 13 Nov 2015 09:43:16 +0900
Content-Transfer-Encoding: quoted-printable
Message-Id: <22142303-00FF-4649-A50F-F655B94F99D2@is.naist.jp>
References: <6b20c5aba195.56384250@naist.jp> <6a60e1ff9170.56384780@naist.jp> <6a60f4388bab.563847bc@naist.jp> <6bd0f10697e2.563847f8@naist.jp> <6a409179ad4a.56384835@naist.jp> <6a80cfd8c72d.56384871@naist.jp> <6c30b15ad280.563848ae@naist.jp> <6c30f0e98215.563848ea@naist.jp> <6c10c39aeff9.56384926@naist.jp> <6ab08659b996.56384963@naist.jp> <6ab0ea4dfdd6.563849a0@naist.jp> <6ab0be62e098.563849dc@naist.jp> <6aa0abb5b14b.56384a19@naist.jp> <6aa0e679a9c8.56384a55@naist.jp> <6b60e1babb96.56384a93@naist.jp> <6b60fdd88897.56384acf@naist.jp> <6a509431f711.56384c39@naist.jp> <6a50aab7bf13.5638cb72@naist.jp> <CAPrseCo-E82O+tSvRC=4x-yXYTMEHUW6UjeQK6HBRZwXey=sKg@mail.gmail.com> <9C1BEDBD-2338-4E1B-8C98-E9479FE01423@is.naist.jp> <56434C78.6090502@net.in.tum.de> <CAPrseCqY1FFQv8yuASVC5xMYQ7w4+KQCnMhE1cfV7Bjtowovqg@mail.gmail.com> <"5644 8E4C.9060307"@n> <et.in.tum.de@naist.jp> <!&!AAAAAAAAAAAuAAAAAAAAAEFA/wpwMJJMh/vN2IEkLQQBANlTnCJhprtFudq2LHCBs8EBACQA//8AABAAAACF68SkjwoLQZ5jYexomO8NAQAAAAA=@is.na ist.jp> <5644C496.5050901@net.in.tum.de>
To: bmwg@ietf.org
X-Mailer: Apple Mail (2.2104)
X-TM-AS-MML: No
X-TM-AS-Product-Ver: IMSS-7.1.0.1392-8.0.0.1202-21938.003
X-TM-AS-Result: No--5.237-5.0-31-10
X-imss-scan-details: No--5.237-5.0-31-10
X-TMASE-MatchedRID: xJ3t8OetYhGPvrMjLFD6eB5+URxv1WlBGcfGM6EiL4aqvcIF1TcLYI+1 bntEYE/17s3FANKtoAWFMgrGNxC2HxgZ7Tcj60Qbma6DzXaohvN+S5m2/8VLmvTCq1LILNRVWvO 5qFysnpUhSMpzgOAsc1tdxppxOLHv7iCc2+9bOG8dxBAG5/hkW4+YVJqrrEz4E7RwSuKQQaiFDj mu9JFekA1Z2WZVyZItX7bicKxRIU1JBCh8s9UcDWc9AABlmIKioOaKLxFhZimOhzOa6g8KrcHIq V0WTGsL/2AZDReuq6RgPYpvVYskBtLiAIlb1HCYJUY46WIRoG0=
Archived-At: <http://mailarchive.ietf.org/arch/msg/bmwg/2PBjuH5PQqCIAwpVcSWvjBqsEeo>
Subject: Re: [bmwg] Mean vs Median
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Nov 2015 00:43:41 -0000

> On Nov 13, 2015, at 01:55, Paul Emmerich <emmericp@net.in.tum.de> wrote:
> 
> On 12.11.15 16:19, Marius Georgescu wrote:
>> [MG]  Thanks for sharing your paper. I am not sure if this solution would
>> make a good comparison base. As I see it, the test report has to be
>> synthetic enough to allow easy comparison. Reporting a single number might
>> not be enough, but reporting 10 numbers is too much imo.
> 
> Maybe the latency report can be split into two categories: typical latency and worst-case latency. The former being something like average and standard deviation (or median and 1st/3rd quartile), the latter the 90th and 99th percentile? A full test report should (optionally?) include the full CDF or histogram as a graph.
> 

MG: That sounds like a good idea to me. In the light of recent discussions, maybe median + 1st&3rd quartiles would be a better choice for the "typical latency” (i was thinking regular/common latency could be more descriptive name). A  MAY for fine grain analysis with histograms sounds good as well.

v2.23.0 | Report a Bug | By Email