Re: [bmwg] Mean vs Median

[Paul Emmerich <[e-mail]>]

Hi,

On 11.11.15 19:46, Stenio Fernandes wrote:
> The discussions so far have led me to conclude that in the context here,
> there is no need to make any assumptions on the sample set. Any specific
> measure of centrality or dispersion might not be precise enough in some
> cases. As stated by others, mean/median would work for well-behaved
> (e.g., normally distributed) data, but would not work for multi-modal or
> heavy-tailed ones. Recall that heavy-tailed distributions are usually
> characterized by the shape and location parameters instead of mean and
> variance. Regarding the number of samples, it is really tough to
> characterize heavy-tailed or multi-modal distributions with a few
> samples, even using advanced algorithms for maximum-likelihood estimation.
my suggestion would be characterizing the latency by providing several 
percentiles. I'd suggest using 10, 20, ... 90, 99, (and maybe 99.9 if we 
define the required number of samples as large enough to capture long 
tails). This is still a small number of data points that can easily be 
included in a test report. But it is effectively a quantized CDF and 
therefore captures the whole distribution including its shape reasonably 
well.


Paul