Re: [bmwg] Mean vs Median

[Stenio Fernandes <[e-mail]>]

On Thu, Nov 12, 2015 at 7:57 PM, Marius Georgescu <[e-mail]
> wrote:

>
>
>
>
> > On Nov 13, 2015, at 03:32, MORTON, ALFRED C (AL) <[e-mail]>
> wrote:
> >
> >> -----Original Message-----
> >> From: bmwg [mailto:[e-mail]] On Behalf Of Paul Emmerich
> >> Sent: Thursday, November 12, 2015 11:56 AM
> >> To: [e-mail]
> >> Subject: Re: [bmwg] Mean vs Median
> >>
> >> On 12.11.15 16:19, Marius Georgescu wrote:
> >>> [MG]  Thanks for sharing your paper. I am not sure if this solution
> >>> would make a good comparison base. As I see it, the test report has to
> >>> be synthetic enough to allow easy comparison. Reporting a single
> >>> number might not be enough, but reporting 10 numbers is too much imo.
> >>
> >> Maybe the latency report can be split into two categories: typical
> >> latency and worst-case latency. The former being something like average
> >> and standard deviation (or median and 1st/3rd quartile), the latter the
> >> 90th and 99th percentile? A full test report should (optionally?)
> >> include the full CDF or histogram as a graph.
> >>
> >>
> > [ACM]
> > I have been suggesting (in many places and drafts) that the "worst case"
> > delay should be characterized with RFC5481 PDV, so that it is a view of
> > delay variation and more easily comparable across test runs. The High
> > percentile as a single measure of PDV is built-in, and of course the
> > CDF or histogram is a possibility:
> > https://tools.ietf.org/html/rfc5481#section-4.2
> MG: I like the idea of using the 99.9th percentile as a single measure for
> the “Worst case” latency.
>

[SF] it seems an interesting approach to keep it simple. ECDFs or
histograms would also help, but in the case of data that follows
heavy-tailed distributions, the graphs would be really weird. They are
better presented in a log-log scale and sometimes as an inverted ECDF.
Anyway, the general approach for further analysis is within the scope of
Exploratory Data Analysis (EDA), which includes all descriptive statistics
and graphical methods. So, I agree with the idea of having typical and
worst-case analysis, and for the optional full report a suggestion could be
to make an EDA on the data. And I would suggest NIST's Engineering
Statistics Handbook (free, online) for further reference (cf. chapter on
EDA: http://www.itl.nist.gov/div898/handbook/eda/section1/eda1.htm)