Re: [bmwg] Mean vs Median

[Paul Emmerich <[e-mail]>]

Hi,

On 03.11.15 09:45, Stenio Fernandes wrote:
> a word of caution here... a number of phenomena in computer networks
> follows a heavy-tailed probability distribution function, which means
> that there is a non-negligible probability that a random variable will
> take huge values. these values might be erroneously considered as outliers.

this is a really important point. I have benchmarked software where the 
99th percentile of the latency is twice the average/median and the 
99.9th percentile ten times the average/median.
This is an important performance characteristic for latency-sensitive 
applications that isn't captured by taking just 20 measurements. So I'd 
really like to see a standard that calls for thousands of latency 
measurements to capture this properly.

You can also get interesting insights into a black-box device by looking 
at histograms/probability density functions. For example, you can figure 
out if the device processes packets in batches, estimate the batch size, 
figure out at which rates interrupt moderation algorithms change etc. 
(This is, of course, not really a performance metric, just an 
interesting insight.)


Paul

-- 
Paul Emmerich
Technical University of Munich (TUM)
Department of Informatics
Chair for Network Architectures and Services