[tsvwg] NQB, what rate is permissable for flows to still "be" NQB

[Sebastian Moeller <[e-mail]>]

Dear list,


in last Friday's tsvwg meeting Greg asked what recommendation the NQB draft should give senders about maximally permissible rates for NQB flows. (At least I left the session with the feeling that input on this topic was requested).

	One proposal, already under discussion on the list was to base this on some statistics of access rate, with the proposal being to take 10% of Ookla's idea of global average access rates (see https://www.speedtest.net/global-index "Speedtest Global Index") which reports global average (and per country sub-averages) of upload, download and latency numbers split into mobile and fixed broadband.

	These are taken from random user measurement against Ookla's speedtest.net (which in itself is a collection of mostly volunteered measurement nodes that just need to fulfill a number of criteria (see https://support.ookla.com/hc/en-us/articles/234578628) to enter the speedtest network and are mostly operated by entities other than Ookla). The required minimum internet access speed (1 Gbps) makes it very likely that these measurement nodes are located in data-centers. During a typical measurement the system will try to automatically match clients with "close by" servers (but allows clients to manually select servers as an alternative).
	
	The global index (from 2019 on) will include data from all countries that "have at least 300 unique user test results for mobile or fixed broadband in the reported month" (https://www.speedtest.net/global-index/about) Neither for the global average, nor for the per country numbers any information about the sample size is given (above the minimum), nor any measure of variance. So this source only gives (somewhat under-qualified) average numbers that are likely to be not really representative (of the spatial distribution of all users within a given country/the world) and  will be biased towards higher speeds (as averages of positively skewed distributions do). If we compare Ookla's current result for Germany https://www.speedtest.net/global-index/germany#fixed
with the data from the official German speedtest site (lower plot on https://breitbandmessung.de/interaktive-darstellung-jahresvergleich *) it should become clear what a per country, or global average hides: the composition of access speeds in Germany as seen by that CDF clearly is more complicated than an average implies (it is not a smooth curve as there clearly are speed-tiers that ISPs can and do offer with their available technology). 
	For answering the NQB problem something related to the experience a given number of endusers will have seems more appropriate than an average. So, IMHO, something like the lower 10% or 25%-percentile of the access speed (taken from a per-country CDF of reliable data sources like in ehe EU the national regulators' reports**) might be a much better number to use for the NQB recommendation than Ookla's averages. Qantiles/Median at least allow to estimate which fraction of users can expect rate issues with compliant NQB-senders. (This is not to dismiss Ookla's numbers, it is great that they compile and offer those, it is just that these are not terribly well suited for the NQB purpose).

	But overall, given that such a recommendation can only ever be approximate and will not in itself guarantee safety for any individual connection but can just be a rough guideline for senders, I would say the current recommendation to not NQB mark flows with instantaneous rates >> 1Mbps seems just as appropriate as any statistics taken from non-official/non-representative numbers like Ookla's.


	As written this recommendation will however run into "conflict" with gaming traffic (not cloud gaming with its video streaming but client server gaming where mainly control packets are exchanged), as games tend to send/receive packets on a "clock" (often 30 or 60 Hz, but 128 Hz is not unheard of) with one to multiple packets per clock-tick (see e.g. https://www.sciencedirect.com/science/article/abs/pii/S1084804511001731) and since back-2-back packets will create bursts at link speed (assuming the burst travels intact to the bottleneck and the bottleneck does not employ a scheduler that splits bursts up) these can exceed the current 1 Mpbs recommendation (I am grossly simplifying here): "Other types of QB flows include those that frequently send at a high burst rate (e.g. several consecutive packets sent well in excess of 1 Mbps) even if the long-term average data rate is much lower."

	Assuming NQB is really intended for gaming control packets, this might be remedied as Greg proposed by giving a more permissive definition of "several consecutive packets" (like say expand several to >= 10 or similar). Whether that is justified on a slow connection within NQB's internal logic is a different question.

Regards
	Sebastian


*) These numbers are not as current as Ookla's and hence a direct comparison seems not advised (https://www.speedtest.net/insights/blog/icymi-ookla-data-and-analysis-from-december-2020/ gives Ookla's fixed broadband global average for December 2020 as 96/52). But these graphs show the development over a number of years including the respective N for each curve and allow to see the countries dominant speed tiers (50 and 100 Mbps) and alsi allow to see the median of the 2019-2020 period (51.1 Mbps down, ~10.7 up) and that there was a small fraction of links with considerably higher speeds (500 and 1000 Mbps) which will make sure that the average (which appears purposefully not reported by the national regulator) would be noticeably larger than the median. BTW the 25%-ile was around 22.7 Mbps down 5.2 up...


**) Unlike Ookla's "mostly-for-fun" speedtest*** (no disrespect intended) the official EU speedtests like the one in Germany that can be used as evidence at court has considerably more stringent specifications than Ookla's and generally seems to be designed and operated carefully, including approaches like having a user report the contracted speed before starting a measurement to make sure the measurement back-end has sufficient resources to saturate that speed.

***) To repeat, I do not want to criticize Ookla here, it is just that their numbers are not suitable for some purposes, especially purposes the test was never designed for.