Re: [bmwg] Mean vs Median

[Paul Emmerich <[e-mail]>]

Hi,

On 10.11.15 08:15, Marius Georgescu wrote:
> For physical testing, the distribution tends to be normal (at least for my testing experiences). In this context the difference between mean and median would be negligible.
> Since in the virtual world (the future) bimodal is the new normal (if i may say so :), median seems like a better choice.
> I fully agree we should have an additional statistic, accounting for the variance somehow. I proposed the Margin of error. It can be the standard deviation, the standard error …
normally distributed latencies are actually pretty rare when testing 
devices that do any part of the processing in software. It doesn't even 
have to be virtualized. The main reason for this is batch processing of 
packets which drastically improves the performance.

Simple batching leads to a uniform distribution. However, there are 
different algorithms trying to optimize for different things in a 
typical software router which leads to really weird distributions.
For example, Linux uses polling in the NAPI to reduce interrupts and 
increase the batch size, but many drivers like Intel's ixgbe also 
feature a second interrupt moderation algorithm that further reduces 
interrupts. Both systems interact and you get weird distributions.
You can read more about interrupt handling for network processing in 
Linux in my paper about it at [1] if you are interested. (However, the 
paper does not yet look at distributions, it only states a few percentiles.)

I've uploaded an excerpt from one of my papers here [2] which shows the 
latency of a Linux system running Open vSwitch without any 
virtualization. An excerpt from the same paper with virtualization is 
available at [3], this looks more like a normal distribution with a long 
tail. (The paper is currently under peer review for a journal and will 
hopefully be published soon, I can share a pre-print privately with 
anyone who is interested in the full paper.)


Paul

[1] 
http://www.net.in.tum.de/fileadmin/bibtex/publications/papers/SPECTS15NAPIoptimization.pdf
[2] 
https://www.dropbox.com/s/dl9tmslubyd2j7m/Screenshot%202015-11-11%2010.17.05.png?dl=1
[3] 
https://www.dropbox.com/s/tia2lwhlt08hl8l/Screenshot%202015-11-11%2010.24.57.png?dl=1