Re: [tsvwg] draft-ietf-tsvwg-nqb, more questions

[Sebastian Moeller <[e-mail]>]

> On Nov 8, 2019, at 00:20, Greg White <[e-mail]> wrote:
> 
> You've still missed the point.
> 
>    	[SM] It will keep one expected major source of DSCP marked traffic on a sane(r) policy.
> 
> No. It won't. 
> 
> What will keep that source on a sane policy is that they have an interest in having a sane policy.  This IETF draft does not create that interest, nor would it prevent them from running amok.   
> 
> -Greg

	[SM] Sorry with source, I was not talking about individual senders, but about the "NQB/L4S complex" that I expect to cause a lot of NQB marked traffic to appear on the ingress side of home users ionce it gets off the ground. And my point is very much to give my best arguments to make sure you do not end up writing a bad policy into an RFC. What ever you do in the real world is outside of the scope of the IETF, but taking that as an argument in favor of bad engineering is simply a bad argument.
	And regarding your interpretation of individual senders, it is clear that remote senders have a snowballs chance in hell to react timely to the variable bitrates on a wifi segment, hence the wifi elements AP (and ideally STAs) will need to enforce sanity and make sure tp avoid starvation. This is not a new requirement, but the status quo is that most traffic is AC_BE, so the lack of enforcement was sort of tolerable; but either NQB withers away and will not be used (and all of this discussion does not matter) or it will be used and then you better make sure that NQB will not be found out as a reason why non-NQB on-line gaming starts to suck. But really looking at the AC stats from my laptop:

laptop:~ user$ sudo netstat -I en0 -qq
en0:
     [ sched:  FQ_CODEL  qlength:    0/128 ]
     [ pkts:          0  bytes:          0  dropped pkts:    185 bytes:  55724 ]
=====================================================
     [ pri: VO (1)	srv_cl: 0x400180	quantum: 600	drr_max: 8 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 171322	bytes: 15967234 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
=====================================================
     [ pri: VI (2)	srv_cl: 0x380100	quantum: 3000	drr_max: 6 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 1081254	bytes: 95307849 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
=====================================================
     [ pri: BE (7)	srv_cl: 0x0	quantum: 1500	drr_max: 4 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 41792980	bytes: 10120070005 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 9	feedback: 9	stalls: 8	failed: 0 ]
     [ drop overflow: 0	early: 5	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
=====================================================
     [ pri: BK (8)	srv_cl: 0x100080	quantum: 1500	drr_max: 2 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 4392109	bytes: 1258595132 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 2	feedback: 2	stalls: 2	failed: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]




and my OpenWrt router:

root@router:~# cat /sys/kernel/debug/ieee80211/phy1/ath9k/xmit
                            BE         BK        VI        VO

MPDUs Queued:             4933        120       595     47628
MPDUs Completed:          2765        331      1045    110627
MPDUs XRetried:           2792         30       381       216
Aggregates:            4053416     109096     11152         0
AMPDUs Queued HW:            0          0         0         0
AMPDUs Completed:     29761018     931822    280598         0
AMPDUs Retried:         484203      23327      4543         0
AMPDUs XRetried:          8606        149       754         0
TXERR Filtered:             97          7        32        37
FIFO Underrun:               0          0         0         0
TXOP Exceeded:               0          0         0         0
TXTIMER Expiry:              0          0         0         0
DESC CFG Error:              0          0         0         0
DATA Underrun:               0          0         0         0
DELIM Underrun:              0          0         0         0
TX-Pkts-All:          29775181     932332    282778    110843
TX-Bytes-All:        672115293 1008162267 199579558  20417055
HW-put-tx-buf:               8          8         8         8
HW-tx-start:          21266021     570626    256944    110799
HW-tx-proc-desc:      21266026     570629    257004    110839
TX-Failed:                   0          0         0         0


Summary packet numbers:
AC	Laptop
BE:		 41792980	29775181
BK:		   4392109	    932332
VI:		   1081254	    282778
VO:		     171322	    110843
Total:	 47437665	31101134


So my status quo is that all four AC are used with the fraction of AC_VI+AC_VO at 
100*(1081254+ 171322)/47437665 = 2.6 % (laptop)
100*(282778 + 110843)/31101134 = 1.3 % (router)

This is with relative high streaming media usage.


Best Regards
	Sebastian






> 
> 
> 
> On 11/7/19, 4:12 PM, "Sebastian Moeller" <[e-mail]> wrote:
> 
>    Hi Greg,
> 
> 
>> On Nov 7, 2019, at 23:44, Greg White <[e-mail]> wrote:
>> 
>> Sebastian,
>> 
>> I agree that this isn't rocket science.  In the existing WiFi systems that we're discussing, there are 16 DSCPs that get mapped to AC_VI, and 16 DSCPs that get mapped to AC_VO.   No one polices any of them in residential WiFi networks!  There is nothing that prevents anyone from writing an application today that sends bulk traffic marked as CS7 (a value that the IETF considers to be reserved).  
> 
>    	[SM] Sure, but thanks to ISP often bleaching DSCPs there currently is very little in and outgoing traffic that actually maps to anything but AC_BE, you propose to change that in a major way, with your attempt at making NQB end to end (with one end potentially being in a close data center with an SLA allowing for NQB survival). Just because no one has massively abused this so far, is NOT a justification to do so now.
> 
> 
>> 
>> I believe that the reason that RFC 8325 simply wrings its hands over this issue, as opposed to solving it, is that A) it is a *really* hard problem to solve in an unmanaged network, and B) while it is certainly a problem in theory, there is no evidence that it is a real problem in practice.
> 
>    	[SM] The problem is that there is simply no data, and interpreting this as there is no problem is neither solid science nor engineering. If you have measurements showing that this is no problem with the expected traffic rates with the NQB marking, please feel free to post them to the list or as PM.
> 
>> 
>> 
>> Blocking the IETF from assigning NQB the 0x2A value does *nothing* to prevent an application from using 0x2A (or any of the other 31 code points that you are concerned about) for *any* purpose that the application developer sees fit, it does *nothing* to prevent a network operator from using 0x2A for NQB traffic, and it does *nothing* to help solve the main issue that you are concerned about.
> 
>    	[SM] It will keep one expected major source of DSCP marked traffic on a sane(r) policy. Most users and applications do not bother paying with dscps so the current amount of abuse is limited (and if I configure my internal skype and ssh applications to even use AC_VO I can control their activity and usage much better than I can traffic rushing in from the internet). Sure I can re-map 0x2A to a sane value on ingress to my network, but that is not going to help that much if my neighbors do not do this, and their APs start hogging all tx_ops on the shared channel.
> 
>> 
>> If you want to solve this problem, my suggestion is that you start a new draft proposing a technology that will holistically protect WiFi networks from all potential DSCP abuses. 
> 
>    	[SM] First rule of holes, put the shovel away when you find yourself inside one. Here, I am not concerned so much in fixing all wifi AC abuse and more in giving my best to avoid this unfortunate and undesirable condition getting worse.
> 
> 
> 
>    Sebastian
> 
> 
> 
>> 
>> Greg
>> 
>> 
>> 
>> 
>> On 11/7/19, 3:00 PM, "Sebastian Moeller" <[e-mail]> wrote:
>> 
>>   Greg,
>> 
>>> On Nov 7, 2019, at 22:11, Greg White <[e-mail]> wrote:
>>> 
>>> Sebastian,
>>> 
>>> In the interest of moving this forward and not going in circles,
>> 
>>   	[SM] My impression is not that we going in circles, it is a rather slow road to understanding and accepting the properties and limitations of wifi's unfortunate prioritization scheme in conjunction with your intended use of the NQB dscp. Me shutting up, would not change the underlaying facts that need to be considered here if the issue is to be solved.
>> 
>> 
>>> I think you summarized your position as: "To be explicit, I do not object on principle to using AC_VI or even AC_VO as long as this does not eat significantly into the tx_ops for AC_BE, the current draft improves  in that direction. Would it be possible to make this point even stronger?"  
>>> 
>>> In my view, as long as the applications that we're recommending to be marked NQB are no more bandwidth intensive than video conferencing (the applications that the IETF already recommends be marked with a DSCP that maps to AC_VI), then we have achieved what is needed.  Do you agree?
>> 
>>   	To state the obvious, "no more bandwidth intensive than video conferencing" is virtually meaningless as it will give very little guidance on acceptable either single flow nor aggregate NQB traffic share or absolute bandwidth (yes rfc8325 is also pretty silent on this, but "look mum, the other guys are doing it too" is not the best of defenses IMHO). I realize that this is an issue for you as you explicitly target NQB as requiring no prioritization or rate-limiting, I get that; but I believe that on wifi since you effectively propose to use a prioritization system you really really should require rate-limiting. 
>> 
>>   rfc8325, section 8.2:
>>   "The wireless LAN presents a unique DoS attack vector, as endpoint
>>      devices contend for the shared media on a completely egalitarian
>>      basis with the network (as represented by the AP).  This means that
>>      any wireless client could potentially monopolize the air by sending
>>      packets marked to preferred UP values (i.e., UP values 4-7) in the
>>      upstream direction.  Similarly, airtime could be monopolized if
>>      excessive amounts of downstream traffic were marked/mapped to these
>>      same preferred UP values.  As such, the ability to mark/map to these
>>      preferred UP values (of UP 4-7) should be controlled.
>> 
>>      If such marking/mapping were not controlled, then, for example, a
>>      malicious user could cause WLAN DoS by flooding traffic marked CS7
>>      DSCP downstream.  This codepoint would map by default (as described
>>      in Section 2.3) to UP 7 and would be assigned to the Voice Access
>>      Category (AC_VO).  Such a flood could cause Denial-of-Service to not
>>      only wireless voice applications, but also to all other traffic
>>      classes.  Similarly, an uninformed application developer may request
>>      all traffic from his/her application be marked CS7 or CS6, thinking
>>      this would achieve the best overall servicing of their application
>>      traffic, while not realizing that such a marking (if honored by the
>>      client operating system) could cause not only WLAN DoS, but also IP
>>      network instability, as the traffic marked CS7 or CS6 finds its way
>>      into queues intended for servicing (relatively low-bandwidth) network
>>      control protocols, potentially starving legitimate network control
>>      protocols in the process."
>> 
>>   To me this clearly indicates that the achievable wifi rate betwnn AP and each station needs to be taken into account when considering which fraction of > AC_BE traffic is acceptable. So very much incompatible with your goal of having endpoints set the NQB dscp based on their own best assessment of traffic type without some sanitization at the AP level (only the AP will have a reasonable chance of predicting immediate achievable rates with some confidence, neither endpoints nor intermediate low latency queue enabled AQMs have sufficient knowledge to judge what traffic rate is acceptable for AC_VI). I consider it to be quite unfortunate that rfc8325 has no references to research on real-world data on the consequences of actually following through with its recommendations, nor that it has robust recommendations of recommended maximal fractions of the achievable rate each AC should be constrained to.
>> 
>>   All of this is not really rocket science, and the obvious solution, following both the principle of least surprise and a "first, do no harm" maxim is to select the NQB dscp such that the unfortunate default DSCP to AC mappings keep NQB marked traffic in the AC_BE queue, UNLESS the AP is NQB-aware in which case the AP should be mandated to at least reliably avoid starvation of ACs.
>>   The amount of back and forth seems to indicate that this is not an outcome you consider desirable.
>> 
>> 
>>   Sebastian
>> 
>>> 
>>> -Greg
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 11/5/19, 1:58 AM, "Sebastian Moeller" <[e-mail]> wrote:
>>> 
>>>  Hi Greg,
>>> 
>>> 
>>>> On Nov 5, 2019, at 01:28, Greg White <[e-mail]> wrote:
>>>> 
>>>> Hi Sebastian,
>>>> 
>>>> Interoperability with existing WiFi equipment is an important aspect, since WiFi latency can be considerable. By default, many existing APs only support 4 priority queues, and thus it is not possible to meet all of the requirements of the NQB PHB (at least in this default configuration).
>>> 
>>>  	[SM] I agree the question is how to deal with that "impedance mismatch".
>>> 
>>>> Nonetheless, it is possible to utilize two of the four queues in order to meet some of the requirements, and thus provide some of the benefits of the NQB PHB.  
>>> 
>>>  	[SM] Unless you opt for selecting AC_BK for the NQB traffic, for most users the value of NQB will be mostly in the priority boost on wifi and the resulting air-time access advantage (which results in both lower latency and potentially higher bandwidth).
>>> 
>>>> With proper configuration and/or policies, this can be done safely.  
>>> 
>>>  	[SM] Sure, I am concerned about the status quo wich does not entail "proper configuration and/or policies", and hence I believe the NQB special treatment on WIFI should be opt-in and not "opt-out" (in quotes as most endusers will not be able to opt-out). For thid reaon I believe that the proposal to use a code point that by default is mapped to AC_BK is the only correct solution (as a bonus it seems that such a code point also has a better chance to survive transit over the internet). NQB-aware APs then simply treat that NQB-codepoint however they want. If for example a priority boost is desired such an AP can easily implement the required rate-limiting so that AC_BE traffic does not get starved out. In short, I fully agree that special treatment requires "proper configuration and/or policies" and the desirable strategy if that can not guaranteed should be "do no harm".
>>> 
>>>> The final SHOULD is intended to address your concern about prioritization (since it results in segregation without prioritization).
>>> 
>>>  	[SM] Ah, in that case the AP needs to be be NQB aware anyway, would it then not be better to use an appropriate scheduler/AQM in front of the AC_BE queue and keep all traffic in the same priority class? The disadvantage of setting AC_VI to the same EDCA values as AC_BE is then that applicatons that expect an airtime access boost from using AC_VI will not get it any more (not necessarily a deal-breaker but certainly unexpected enough to merit clear communication of that side-effect).
>>> 
>>>> Absent this requirement (or the ability to comply with it operationally), the operator would need to consider (and perhaps limit) which applications are allowed to be marked as NQB.  This aspect isn't discussed in the draft, but I will add it based on your input.
>>> 
>>>  	[SM] Great! I would guess the safest would be to have the NQB-aware scheduler in an AP apply some (proportional) rate-limiting if NQB traffic is getting preferential air-time access.
>>> 
>>>> 
>>>> Network operators understand the value of segregating NQB traffic on WiFi links, and will almost certainly select a DSCP in practice that achieves that goal.  
>>> 
>>>  	[SM] That is exactly part of my concern with the default mapping to AC_VI approach, I expect that very quickly a lot of traffic will utilize the AC_VI queue potentially starving normal AC_BE traffic in the process.
>>> 
>>>> Assigning a different DSCP in this draft would do nothing to prevent them from doing so.
>>> 
>>>  	[SM] Sure, but is that really a good justification for proposing a DSCP with known side-effects? As far as I am concerned an RFC should propose sane defaults and hope for the best.
>>> 
>>>> Instead, what we need to do is clearly articulate how to make best use of the existing WiFi tools, and how to avoid conflicts. 
>>> 
>>>  	[SM] I believe the last two are mutually exclusive...
>>> 
>>>> 
>>>> In existing RFCs, the IETF already recommends that video conferencing applications mark their traffic as either AF4x or CS4, all of which get mapped to AC_VI.  The remaining language in the NQB draft describes sparser flows than these.
>>> 
>>>  	[SM] as an implementer I read "relatively low data rates", without further guidance I have very little intuition what to use as reference. Could this be made more explicit? This is orthogonal to the question whether such a limit should be enforced in any way, here the question really is about getting a feel what is considered acceptable for NQB treatment.
>>> 
>>>> 
>>>> Based on your comments, I attempted to remove all text that could be interpreted as recommending that high-data-rate traffic be marked NQB.  
>>> 
>>>  	[SM] Thanks, as long as the aggregate NQB traffic is relative sparse compared to the available WiFi bandwidth (or the number of tx_ops) most of my WiFi concerns get less and less relevant. To be explicit, I do not object on principle to using AC_VI or even AC_VO as long as this does not eat significantly into the tx_ops for AC_BE, the current draft improves  in that direction. Would it be possible to make this point even stronger?
>>> 
>>>> It appears that I missed one instance (in the Introduction it gives "interactive voice and video" as an example). Aside from this (which I can correct), I think the draft currently recommends that NQB only be used for sparse traffic.  That said, the section where this guidance is intended to be given is still lacking in specificity, and poses some open questions that may need to be addressed in a subsequent revision.
>>> 
>>>  	[SM] Sounds great. Now this then cycles back to one of the other open topics, "enforcement". Ideally NQB-aware APs should monitor both queues and re-assign flows between them based on flow-behavior in relation to time-variant bandwidth experienced by that flow. 
>>> 
>>>  Best Regards
>>>  	Sebastian
>>> 
>>>> 
>>>> Best Regards,
>>>> Greg
>>>> 
>>>> 
>>>> On 11/4/19, 3:25 PM, "tsvwg on behalf of Sebastian Moeller" <[e-mail] on behalf of [e-mail]> wrote:
>>>> 
>>>> Regarding https://datatracker.ietf.org/doc/draft-ietf-tsvwg-nqb/?include_text=1
>>>> 
>>>> 7.3.  WiFi Networks
>>>> 
>>>>    WiFi networking equipment compliant with 802.11e generally supports
>>>>    either four or eight transmit queues and four sets of associated EDCA
>>>>    parameters (corresponding to the four WiFi Multimedia Access
>>>>    Categories) that are used to enable differentiated media access
>>>>    characteristics.  Implementations typically utilize the IP DSCP field
>>>>    to select a transmit queue, but should be considered as Non-
>>>>    Differentiated Services-Compliant Nodes as described in Section 4 of
>>>>    [RFC2475].  As a result this document discusses interoperability with
>>>>    WiFi networks, as opposed to PHB compliance.
>>>> 
>>>>    As discussed in [RFC8325], most existing implementations use a
>>>>    default DSCP to User Priority mapping that utilizes the most
>>>>    significant three bits of the DiffServ Field to select "User
>>>>    Priority" which is then mapped to the four WMM Access Categories.  In
>>>>    order to increase the likelihood that NQB traffic is provided a
>>>>    separate queue from QB traffic in existing WiFi equipment, the 0x2A
>>>>    codepoint is preferred for NQB.  This would map NQB to UP_5 which is
>>>>    in the "Video" Access Category.
>>>> 
>>>>    Systems that utilize [RFC8325], SHOULD map the NQB codepoint to UP_5
>>>>    in the "Video" Access Category.
>>>> 
>>>>    In order to preserve the incentives principle, WiFi systems SHOULD
>>>>    configure the EDCA parameters for the Video Access Category to match
>>>>    those of the Best Effort Access Category.
>>>> 
>>>> 
>>>> [SM] This last section is puzzling: if the wifi system configures AC_VI with EDCA parameters that match the AC_BE parameters, AC_VI ceases to be different from AC_BE, in that case picking a codepoint that automatically maps to CS0 and hence to AC_BE  seems much safer, simpler and straight forward to me. 
>>>> Especially since essentially none of the millions deployed WiFi APs out there will a) have this configured like proposed already and b) none of the consumer APs I know actually allow to easily adjust EDCA parameters at all. I guess I must be missing something and would be delighted to be shown why the proposed text is the right thing.
>>>> My take on this still is, if NQB traffic is sufficiently sparse using AC_VI can be justified, but without any rate limits this has the potential of being quite unfair to concurrent APs on the same channel (as well as the neighboring channels that overlap with the selected). 
>>>> I do not want to sound alarmist, but given the number of cable-ISP WiFi-APs (as indicated by a SSID containing the ISPs name) in my city, I believe making sure that those APs will not basically start hogging most airtime seems the prudent thing to do. If there are sufficient backstops in place (like rate limiting or automatic down-marking if the traffic is not sparse enough) to avoid the described situation, I am all for it.
>>>> 
>>>> The text probably should also openly discuss that in WiFi/WMM the four available queues by design have different priorities, and by moving NQB out of the default AC_BE while leaving QB flows in there, this effectively runs against  the following text in the draft: "The NQB queue SHOULD be given equal priority compared to queue-building traffic of equivalent importance." (leaving alone the question how an AP or a station is supposed to measure importance)
>>>> 
>>>> 
>>>> Sebastian
>>>> 
>>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 
> 
> 
>