Re: [tsvwg] NQB, what rate is permissable for flows to still "be" NQB

[Greg White <[e-mail]>]

Hi Sebastian,

Thank you for the constructive post.

I do agree with you that using global average statistics isn't ideal.  If we had a global CDF of access network rates, sampled randomly and updated periodically, that would be better.  And, as you indicated, we could then use the 10th or 25th percentile number as guidance, rather than some fraction of the mean.

Given this, perhaps we use language that indicates more clearly that the limits on rate are expected to evolve over time, but keep the recommended limit – labeled somehow as "current" – of a few packets per RTT or 1 Mbps.  This leaves it for future discussion and assessment based on up-to-date knowledge at the time, rather than trying to nail it down precisely today.

On the other topic you mentioned – applications that might send a few packets each "tick" – that is a good point.  The applications we're referring to are isochronous (sending state updates on a fixed interval), but each update "message" could exceed the network MTU and thus be broken into multiple packets, usually sent back-to-back.  This raises the question of how big of a burst should be allowed?  The language currently implies than an MTU-sized "burst" is ok (i.e. it doesn't limit the packet size to something smaller than that).  But, upon what foundation is this based?  Surely an application that sends a 1500 B packet, once every 30 ms, isn't significantly more NQB than an application that sends a pair of 800 B packets every 32 ms.    For that matter, an application that sends a pair of 200 B packets back-to-back likely would be more NQB than an application that sends a single 1500 B packet, yet the former application would fail the test in this recommendation.

In my opinion we only need reasonable guidance here, not precision. NQB is not a guaranteed service after all.  Since this is just a recommendation (and we provide some further qualitative description about what NQB is meant to be) perhaps it is ok to leave it to the reader to convince themselves whether or not they are (e.g.) fine marking an online game's traffic as NQB even if it occasionally sends two back-to-back packets, if the data rate on an inter-burst basis is still well less than 1 Mbps.   There will always be borderline cases and exceptions.  Perhaps a particular application indicates to its users that the minimum requirement for internet connectivity is 20 Mbps. Based on this, they might feel as if they are not taking too much of a risk by marking a smooth 3 Mbps stream as NQB. 

At the end of the day, I think the intent is that the application developer understands that by using NQB marking they take on some risk that their traffic will be sanctioned by a Traffic Protection algorithm (which could result in out-of-order delivery or loss) in exchange for a protected, low-latency queue. The lower (and smoother) the data rate, the lower that risk is. We should provide suitable guidance so that an application developer can relatively easily make the decision.   

Another aspect I'm considering is whether the draft should provide some guidance for "very low rate" links.  Guidance could be that very low rate links (e.g. links that are less than the 10th percentile of global access network rates) should implement a Traffic Protection mechanism that is more tolerant of burstiness (i.e. a temporary queue) than higher rate links might be.

-Greg


On 11/13/21, 10:41 AM, "tsvwg on behalf of Sebastian Moeller" <[e-mail] on behalf of [e-mail]> wrote:

    Dear list,


    in last Friday's tsvwg meeting Greg asked what recommendation the NQB draft should give senders about maximally permissible rates for NQB flows. (At least I left the session with the feeling that input on this topic was requested).

    	One proposal, already under discussion on the list was to base this on some statistics of access rate, with the proposal being to take 10% of Ookla's idea of global average access rates (see https://www.speedtest.net/global-index "Speedtest Global Index") which reports global average (and per country sub-averages) of upload, download and latency numbers split into mobile and fixed broadband.

    	These are taken from random user measurement against Ookla's speedtest.net (which in itself is a collection of mostly volunteered measurement nodes that just need to fulfill a number of criteria (see https://support.ookla.com/hc/en-us/articles/234578628) to enter the speedtest network and are mostly operated by entities other than Ookla). The required minimum internet access speed (1 Gbps) makes it very likely that these measurement nodes are located in data-centers. During a typical measurement the system will try to automatically match clients with "close by" servers (but allows clients to manually select servers as an alternative).

    	The global index (from 2019 on) will include data from all countries that "have at least 300 unique user test results for mobile or fixed broadband in the reported month" (https://www.speedtest.net/global-index/about). Neither for the global average, nor for the per country numbers any information about the sample size is given (above the minimum), nor any measure of variance. So this source only gives (somewhat under-qualified) average numbers that are likely to be not really representative (of the spatial distribution of all users within a given country/the world) and  will be biased towards higher speeds (as averages of positively skewed distributions do). If we compare Ookla's current result for Germany https://www.speedtest.net/global-index/germany#fixed
    with the data from the official German speedtest site (lower plot on https://breitbandmessung.de/interaktive-darstellung-jahresvergleich *) it should become clear what a per country, or global average hides: the composition of access speeds in Germany as seen by that CDF clearly is more complicated than an average implies (it is not a smooth curve as there clearly are speed-tiers that ISPs can and do offer with their available technology). 
    	For answering the NQB problem something related to the experience a given number of endusers will have seems more appropriate than an average. So, IMHO, something like the lower 10% or 25%-percentile of the access speed (taken from a per-country CDF of reliable data sources like in ehe EU the national regulators' reports**) might be a much better number to use for the NQB recommendation than Ookla's averages. Qantiles/Median at least allow to estimate which fraction of users can expect rate issues with compliant NQB-senders. (This is not to dismiss Ookla's numbers, it is great that they compile and offer those, it is just that these are not terribly well suited for the NQB purpose).

    	But overall, given that such a recommendation can only ever be approximate and will not in itself guarantee safety for any individual connection but can just be a rough guideline for senders, I would say the current recommendation to not NQB mark flows with instantaneous rates >> 1Mbps seems just as appropriate as any statistics taken from non-official/non-representative numbers like Ookla's.


    	As written this recommendation will however run into "conflict" with gaming traffic (not cloud gaming with its video streaming but client server gaming where mainly control packets are exchanged), as games tend to send/receive packets on a "clock" (often 30 or 60 Hz, but 128 Hz is not unheard of) with one to multiple packets per clock-tick (see e.g. https://www.sciencedirect.com/science/article/abs/pii/S1084804511001731) and since back-2-back packets will create bursts at link speed (assuming the burst travels intact to the bottleneck and the bottleneck does not employ a scheduler that splits bursts up) these can exceed the current 1 Mpbs recommendation (I am grossly simplifying here): "Other types of QB flows include those that frequently send at a high burst rate (e.g. several consecutive packets sent well in excess of 1 Mbps) even if the long-term average data rate is much lower."

    	Assuming NQB is really intended for gaming control packets, this might be remedied as Greg proposed by giving a more permissive definition of "several consecutive packets" (like say expand several to >= 10 or similar). Whether that is justified on a slow connection within NQB's internal logic is a different question.

    Regards
    	Sebastian


    *) These numbers are not as current as Ookla's and hence a direct comparison seems not advised (https://www.speedtest.net/insights/blog/icymi-ookla-data-and-analysis-from-december-2020/ gives Ookla's fixed broadband global average for December 2020 as 96/52). But these graphs show the development over a number of years including the respective N for each curve and allow to see the countries dominant speed tiers (50 and 100 Mbps) and alsi allow to see the median of the 2019-2020 period (51.1 Mbps down, ~10.7 up) and that there was a small fraction of links with considerably higher speeds (500 and 1000 Mbps) which will make sure that the average (which appears purposefully not reported by the national regulator) would be noticeably larger than the median. BTW the 25%-ile was around 22.7 Mbps down 5.2 up...


    **) Unlike Ookla's "mostly-for-fun" speedtest*** (no disrespect intended) the official EU speedtests like the one in Germany that can be used as evidence at court has considerably more stringent specifications than Ookla's and generally seems to be designed and operated carefully, including approaches like having a user report the contracted speed before starting a measurement to make sure the measurement back-end has sufficient resources to saturate that speed.

    ***) To repeat, I do not want to criticize Ookla here, it is just that their numbers are not suitable for some purposes, especially purposes the test was never designed for.