Re: [tsvwg] NQB recommendations for WiFi. [was: Qs on your 5G L4S slides]

[Sebastian Moeller <[e-mail]>]

Hi Greg,

More below [SM3]

On 17 March 2021 00:28:32 CET, Greg White <[e-mail]> wrote:
> See [GW3]
> 
> On 3/16/21, 9:03 AM, "Sebastian Moeller" <[e-mail]> wrote:
> 
>   Hi Greg,
> 
>   thanks for fixing the topic. More below prefixed [SM2]
> 
>> On Mar 16, 2021, at 15:27, Greg White <[e-mail]> wrote:
>> 
>> Changing the subject line because this is no longer related to
> Ingemar's 5G L4S slides.
>> 
>> See [GW2].
>> 
>> On 3/15/21, 4:37 PM, "Sebastian Moeller" <[e-mail]> wrote:
>> 
>>   Hi Greg,
>> 
>>   see [SM] below.
>> 
>> 
>>> On Mar 15, 2021, at 22:57, Greg White <[e-mail]> wrote:
>>> 
>>> Inline [GW].
>> [snip]
>>> [GW] The NQB draft does NOT make any recommendations on treatment of
> L4S-ECN marked traffic.  In addition, for NQB traffic it recommends to
> map it into a separate queue in the best effort access category
> (AC_BE).
>> 
>>   	[SM] Which is sweet, but for almost 100% of deployed APs that is
> not going to happen, not eve the tinker friendly opensource OpenWrt APs
> allow to change the weights of the ACs.
>> 
>> [GW2] What do you mean?
> 
> 	[SM2] As I said, OpenWrt does not allow users to modify the EDCA
> parameters easily, sure, you can build your own firmware from source
> and add any customization you want, and you and I can do this, but te
> rest of my family can not. I simply assume tat the technical skills to
> do so are not ubiquitous in the population enough to relay in that.
> 
> [GW3] I didn't build any firmware.  I used easily obtainable open
> firmware binaries in each case.  

[SM3] Then please tell me how you persistently changed the EDCA parameters on OpenWrt, so I can add that to the OpenWrt wiki. With OpenWrt being the base of some commercial routers tsuch documentation might be useful over the pure OpenWrt community.


But, this is beside the point. It is
> not my expectation that end-users will read the RFC and then make these
> adjustments themselves (even if there was a brightly colored button on
> top of the AP that said "NQB Mode"). The deployed hardware *does*
> support the ability to use different EDCA parameters than the default
> values.  

[SM3] Yes, I asked for tests whether a secondary AC_BE was feasible at all, you reported succesful tests and I conceded that point, I am not talking about whether a secondary AC_BE is possible, but how feasible a change of units already in the field is. IMHO this is an important point.



> The guidance in the draft is that ISP equipment MUST use a
> DSCP that maps to AC_BE, unless they've got safeguards in place. 

[SM3] Greg, how is that supposed to be safe? NQB is a codepoint that will be set by endpoints not by an an intermediary node like an ISP.  
	But an ISP will not necessarily be aware that NQB traffic passes through its pipes, or be aware of the NQB draft/RFC.... Add to this that ISPs are also not required to do any remapping of DSCPs at all, in fact we actively encourage ISPs to leave DSCPs alone so they can work end to end; IIRC your draft does that as well. And in that case there is simply no awifi-safety left, no admission control, no rare control, no nothing.
	IMHO the consequence is that NQB needs to be WiFI-safe by default, requiring any additional action to make it safe is NOT good engineering practice, period. Because if we require participating ISPs to actually actively do something, instead of de-fanging NQB we could also ask them to actually arm NQB instead, which will be significantly safer. THe fact that I need to spell this out, does not reflect well on the safety considerations in the draft, sorry. I am not impressed.



> Configuring EDCA is *one* of the safeguards that they could use, and it
> is 100% possible in ISP-deployed APs to do so, and so providing this
> guidance is both appropriate and useful.

[SM3] Okay, I read this as a general recommendation, instead of a bespoken recommendation for NQB-aware ISPs that deploy APs in the field. Also I think I have an addition proposal for that group:
0) have the draft propose an NQB that maps to AC-BE by default
1) Ask these ISPs to filter/steer NQB market packets into a secondary AC_BE wifi queue
2) instruct them to use e.g. hostapd's qos_map to have all stations connected to that AP also map NQB into that secondary AC_BE queue

For participating ISPs this will result in the same outcome while being safe by default for non-participating wifi networks.




> 
>> I've personally done it with multiple off-the-shelf consumer grade
> devices (using multiple chipsets), and measured per-AC throughput with
> the EDCA params configured as described in the draft.  There was only
> one model that I came across where setting the EDCA parameters was not
> possible, and that implementation was clearly broken because it gave BE
> higher priority than VI by default.  I sent you an email with this
> result a year ago. 

[SM3] Yes, and I ceded that point and am not trying to relitigate that. The discussion has moved on, from is this possible, to how feasible that is.



> Plus, I believe that the majority of broadband
> providers in the world either deploy an integrated gateway or provide a
> WiFI router to their customers so they (the provider) have the ability
> to set these params themselves.  

[SM3] Believe is good to construc an initial hypothesis, but then we need data to confirm/falsify that hypothesis. Anecdotally, I hope you agree, that there is also a lot of end-user supplied wifi gear in use, which also needs to deal gracefully with NQB.


> 
> 	[SM2] Do you have a break-down how many of the world's APs fall into
> the oned by ISP ans owned by enduser categories?
> 
> [GW3] I only have anecdotes from large ISPs in multiple countries, by
> no means a complete survey.  But, again, you are missing the point.

[SM3] Mmmh, am I? I think we talking past each other  here. As above my point is not all gear is under control of ISPs. Side point, looking how long it took ISPs to deal with e.g. intel's puma6 bug on docsis modems I am not convinced, that ISPs can generally be understood as goid stewards of their end users' interests, but I digress.
	NQB needs to be safe for all of the existing internet, including end-user operated severely out-of-date WiFi gear.

> 
>> 
>>> It only utilizes the video access category as a way to interoperate
> with existing WiFi gear (including RFC8325 gear*), and in that case it
> recommends that the EDCA parameters be configured so that AC_VI gets
> the same bandwidth priority as AC_BE.
>> 
>>   	[SM] But we all know that this recommendation will not actually
> to changes in a noticeable number of already deployed APs. Let's not
> kid ourselves that the update problem magically disappears just because
> it would be nic; this does not work for safety fixes, why should it
> work any better for new features?
>> 
>> [GW2]  You are way out on a limb here.  The recommendations in the
> draft currently are that ISPs use a codepoint at interconnection that
> (if left unmodified) would map to AC_BE.  
>> It then mandates that, by default, access network gear ensure that
> such a codepoint is used. It only recommends that an ISP re-mark with a
> codepoint that maps to AC_VI when appropriate safeguards are in place,
> which could be a combination of policing, EDCA configuration, or full
> NQB PHB compliance.  What is your remaining safety concern!?
> 
> [SM2] Same as before opt-out versus opt-n for the AC_VI component. My
> proposal, use DSCP2 and have participating nodes that know whet they
> are doing and that it is safe to do so, re-map DSCP2 to DSCP42, is
> inherently safer than the use a DSCP with known side-effects and relay
> on other to clean up the mess. If there are no others or they are
> oblivious to NQB, but are nice enough not to bleach DSCPs where is the
> safety? 
> In reality explicit opt-in solutions tend to be safer by far than those
> solutions that require explicit opt out. We have been through this
> before.
> 
> [GW3] From a practical perspective, that is the result of what is in
> the draft currently. There is no opportunity for remapping of upstream
> DSCP until after it has already crossed the WiFi link,

[SM3] Sorry, there is, hostap's qos_maps allowd an AP to instruct connected stations which DSCP to AC mapping they should use. The fact that you do not seem to know that does not reflect well on the amount of research that went into the WIFi section of this draft. 



> and you had
> admitted previously that since there is no policing of *any* upstream
> DSCP or AC available in WiFi today (and currently there are plenty of
> recommendations already around using AC_VI and AC_VO, and some
> applications and operating systems that already do so) so recommending
> that NQB compliant applications use AC_VI in the upstream is consistent
> with existing recommendations and introduces no additional risk.

[SM3] Sorry, I disagree on the risk part, any recommendation that leads to a higher fraction of AC-VO/VI being used increases the risk of unfairness and air-time starvation. And just because everybody else so far behaved like an a**hole, is no excuse to also behave like one (one of the orinciples I try to teach my kids, "two wrongs do not make a right"). 


> Also,
> I think you agree that these pre-existing recommendations about DSCP
> and AC usage do not result in your scary scenario of a sender on one
> network flooding another user's WiFi network with inappropriately
> marked traffic via their ISP (or if it is happening, then I'd argue
> that the ISP is negligent already).  

	[SM3] That is because in practice, end-hosts rarely set DSCPs other than zero at all. You intend to change that by encouraging applications to set NQB.
Airtime crunch already happens to day, just look at the reports of users that practically abandoned 2.4 GHz wifi due to extremely low throughput. What is easily visible is, how many other SSIDs/APs are active on the same band (or overlapping ones), but airtime crunch due to different AC usage is pretty much invisible to end users, you need to make packet captures over wifi including the wifi headers to see this, which is outside of the scope of normal ed users.
That is exactly what I tried to explain before, the failure mode you introduce is a silent one, that will be hard ot diagnose correctly.


>  Why do you think it would be
> "nice" for an ISP to forward all DSCPs from the Internet into their
> customers' WiFi networks without any policing?

[SM3] Because I would very much like DSCPs to be end to end, so applications can convey their intent, and my ingress gateway can than select the correct implementation DSCP to map to the desired priority band inside my network. My proposal is to split the 6 DSCP bits equitable, the 3 CS bits go to the transport to be remapped transiently as they please, the other three bits go end to end to convey intent. Whether I honor intent or not is not my ISPs business, but I digress.


> 
> What NQB adds is the recommendation that the NQB DSCP gets carried
> end-to-end, and your concern previously was

[SM3] And still is, because you still have not solved that realistic situation, at all.


> that doing so could
> blindside a WiFi user if the sender's ISP followed NQB PHB guidance and
> passed the value 42 into an interconnecting ISP, and the
> interconnecting ISP was both unaware of NQB and blindly passed DSCPs
> from interconnection to the user's WiFi network (i.e. the negligent
> ISP).  

[SM3] Where is the requirement in any PS RFC that ISPs needs to remap unknown DSCPs to zero? As far as I can tell the recommendation is to pass DSCPs along that an ISP does not need to use for their own purposes, no? Neither https://tools.ietf.org/html/rfc2474 nor https://tools.ietf.org/html/rfc2475 seem to require or even recommend DS domains to re-map unkown DSCPs to zero, so your point about negligence seems simply your personal assessment/opinion.


> Per the recommendations in draft 05, that won't happen.

[SM3] How? You still need someone to actively change the NQB dscp to make it safe, and as we hopefully agrre the existence of said benevolent party that takes that action is simply not guaranteed.
There are studies showing the survival rate of DSCPs end to end, ande these are >> 0% for basically all dscps, so I am not convinced that NQB betting on all AS/DS domains re-mapping unkonwn DSCPS to zero ist sane or safe. We keep going in circles, but I have data supporting my position (Custura, Ana, Raffaello Secchi, and Gorry Fairhurst. ‘Exploring DSCP Modification Pathologies in the Internet’. Computer Communications 127 (1 September 2018): 86–94. https://doi.org/10.1016/j.comcom.2018.05.016. show that ~ >= 40% of all dscps survive multihop paths through the internet, I can almost guarantee that not all of of those where covered by SLAs/TCAs between the networks), while you level insults at ISPs (negligent) and hope for strict re-mapping. 
So, I disagree with your assessment, that my situation of concern will not happen, simply becsuse you added a re-marking requirement to the draft text. That is not how safety works, Greg, and you know that, so please do not pretend otherwise.

> 
>>> 
>>> *this has gotten me thinking that it would be worth further
> discussion on NQB recommendations for RFC8325 gear.  Since the
> recommendation in the NQB draft would amount to a change to the
> implementation, perhaps the draft should recommend that 8325 gear (if
> possible) maps NQB to a separate queue in AC_BE, and only provides the
> AC_VI option as a backstop in case the implementation can’t provide a
> separate queue.
>> 
>>   	[SM] Again sounds sane, unless we look at the deployed base.
>> 
>> [GW2] What do you mean?
> 
> [SM2] The deployed base will ONLY do the DSCP42 to AC_VI
> prioritization, none other gear exists, and will, if it ever appears,
> take a long time to reach prevalence in the field. What you call a
> backstop will be the principle mode of operation for NQB for a long
> time.
> 
> [GW3] Are you talking about RFC8325 gear?  

	[SM3] I am talking about 802.11n and greater, whichh all supports WMM, and it is WMM that contains the maap DSCPs to ACs methods that cause the issue, I do not care about any RFC text here, I care about the existing wifi APs and stations in the existing internet/home networks. Getting two RFCs to harmonize only requires crafty wording, but getting the deployed base and a new proposal to harmonize requires accepting the conditions in the field and making sure the text matches the reality (or show a path of you the reality can be gradually changed, without major disturbance).


> 
> 
>> AFAIK there are no RFC8325 devices deployed currently.  Are you
> saying that it is insane for anyone to implement RFC8325?  Or are you
> saying that it is insane for us to provide recommendations to them? 
> Surely you don't think that all of the WiFi gear that will ever be
> deployed is already in the field.  
> 
> [SM2] No, I wanted to convey that think the argument you brought is
> sane and rationale, except for the timeframe in which the backstop will
> be operational.
> 
> [GW3] Fine. The NQB PHB is intended as standards track, for use going
> forward, not just for the next year or two.  It is understood that it
> will take time for it to be implemented and deployed, and until then
> some real benefits can be provided using existing gear via the
> interoperability aspects (and associated safeguards).

[SM3] Which safe-guards again? Really, you admit, that proper NQB-over-wifi support (which you envision to take a few years to realize) will require changes to APs, but at the same time you rule out the obvious safe solution, of only giving NQB-ver-wifi special treatment on those APs that are already NQG-aware and contain the necessary safety mechanisms.
I am at a loss, why you fight claw nail and tooth against the obvious save and deployable solution (which will give an advantage: better service to your main market, participating ISPs) UNLESS this is really just a power-play to gain IETF blessing for hogging everybody's airtime.



>>> 
>>> L4S is also walking into the WiFi environment as it finds it. With
> today's non-L4S products, I would also recommend that the L4S-ECN
> codepoints are mapped to the video access category, if possible. 
>>> Nokia's latest WiFi products (in the 'Beacon' range) already include
> an L4S DualQ Coupled AQM. And as other L4S WiFi products come out, the
> coupling will introduce the recommended congestion signals that can be
> used as back-pressure against the priority scheduler. Users don't want
> to abuse scheduling priority at the expense of the balance between
> their own applications. But they have no choice until there's a
> mechanism that allows their applications to balance against other apps.
>>> 
>>> 
>>> [GW] In my view the preferred option is for the dual-queue coupled
> AQM to be implemented within a single access category (e.g. AC_BE). 
> Utilizing AC_VI for L4S-ECN traffic would eliminate the ability to
> provide backpressure, since the BE queue in one STA can’t easily be
> coupled to the VI queue in another.
>> 
>>   [SM] Not that I want to defend ab-using AC_VI for L4S (as I said
> NQB with its aim for low rate flows has some justification for using
> AC_VI, assuming sufficiently strict admission control), but
> back-pessure in WiFI between all entities only happens via airtime
> access and that means you mainly compete with senders using the same
> AC, no?
>> 
>> [GW2] Well that statement isn't exactly true (unless I misunderstood
> you), but the point is that dual-q coupled AQM gives a scheduling
> weight to L4S traffic, but then subjects that traffic to CE marking
> driven by the classic queue (the backpressure).
> 
> [SM2] Yes, a design that has a number of failure modes, in fact I
> prefer to call a spade a spade and call this a 1:16 priority scheduler
> with a few heuristics to ameliorate the sharing behavior under a number
> of conditions, but I digress.
> 
>> If you wanted to use default-configured AC_VI for L4S in order to
> provide the scheduling weight part, you wouldn't be able to get the
> backpressure since there is no way in WiFi for the classic (i.e. BE)
> queue in one device to directly influence the CE marking of traffic in
> another device's VI queue.  So, the better approach would be to only
> give L4S traffic scheduling weight within the device (i.e. a separate
> queue in BE) where you can provide the backpressure.  
> 
> [SM2] I agree that L4S should not use AC_VI (or other priority EDCA
> parameter sets). My point is that even if you would do back pressure
> right in one SSID, this will not help the neighbors. Assume network a
> on 802.11n's channel 6 uses AC_VI for L4S traffic but properly detects
> back pressure from its own AC_BE queue, but there is only L4S traffic
> active which all maps into AC_VI without issue for that SSID.
> 	Now, a neighbor also uses 802.11n channel 6, that neighbor will get
> almost no air-time on channel 6 for traffic in its own AC_BE. And that
> pretty much affects all other radios on channel 6 in reception range. 
> 
> [GW3] Multiple SSIDs doesn't add anything new to this situation.  Even
> if all of the devices are on a single SSID there is no way in existing
> equipment (802.11n, .11ac, or .11ax) for the BE queue in one device to
> communicate to the VI queue in another.  

[SM3] T


> BTW, what do you mean by
> "almost no air-time"? BE and Vi will share the medium with a 1:4 split
> by default.

[SM3] In theory, in practice this differs. 

Here is a link to a flent rrul test from a wired linux host (happy-horse) to a wireless client (hms-beagle, macbook pro, und macosx)
rrul uses 4 flows per direction on each of BK (CS1), BE (CS0), CS5, EF, which map respectively to AC_BK, AC_BE, AC_VI, AC_VI

https://abload.de/image.php?img=rrul_-_happy-horse_fqaekyt.png

Please note that, albeit the AP (OpenWrt) and the client exercise 3 of the 4 ACs, the client practically starves the AP for txops.

And here are the average Station-to-AP rates for the four flows
BK:		4.88 Mbps
BE:	 	3.64 Mbps
CS5:	25.27 Mbps
EF:		24.82 Mbps
and here the AP -to-station rates:
BK:		0.03 Mbps
BE:	 	0.11 Mbps
CS5:	0.41 Mbps
EF:		0.39 Mbps

This also demonstrates that your assumption of a 1:4 BE/VI split might be correct in theory, but not in practice.


Really, Greg, I am getting offended a bit, by your clear assumption that standards text is more relevant than what is out in the field. NQB needs to operate in the existing internet, with existin gear, warts and all. And the current draft is, IMHO simply unfit to do so, at least in the one section, where I have some expertise, WiFi. 


> 
> [SM2] As I said for properly managed NQB use, with rate-limiting that
> situation will be rare/unlikely (but still possible, WiFi rates depend
> on conditions and whether a traffic class is sparse or saturating
> depends on the ratio  between the classes instantaneous rate and the
> available instantaneous rate*)
> When scheduling on a shared medium, you really need to take the full
> medium into account.; but you know that way better than I do ;)
> 
> [GW3] True.

	[SM3] And that bugs me a bit, what you are proposing for "my" wifi environment here, is something you would not entertain for one second to deploy in the other shared medium in which you are an expert....


> 
>   Best Regards
>   	Sebastian
> 
> 
> *) I made this point in an earlier review, non-queue-buiding is IMHO a
> misnomer, non-capacity-seking is a far better description.
> 
> [GW3] I don't agree.  Non-capacity-seeking applications could still be
> Queue-Building.
> Imagine a video codec that runs at 20 Mbps & 30 fps but
> dumps each video frame (~83kB) onto the link as back-to-back packets on
> a GigE interface.  For a 40 Mbps bottleneck (which the video codec is
> not seeking to fill to capacity) that would still certainly look like
> QB traffic. 

	[SM3] Non-capacity-seeking is a property you can easily test for and is a property in the hand of the protocol/application, queue-building however is in the hand of the network as you nicely demonstrate. You can have a network so fast, that TCP is not going to saturate it and hence will build no queue. And yes, you are repeating my point for me, a sending application can not know whether its traffic is going to build a queue or not independent on its rate. So IMHO non-queue-building still is not a good description or name. It is however simple

Best Regards
	Sebastian



> 
> 
>> 
>> 
>>