Re: [tsvwg] WGLC: draft-ietf-tsvwg-nqb-14, closes 21 November 2022

[[e-mail]]

Hi Greg,

To, a standards track document should indeed be precise, especially if it is standardizing a forwarding path including an unlimited priority scheduler (AC_VI). This is a clear purpose of draft NQB.

Further responses [RG2] below.

Von: Greg White <[e-mail]>
Gesendet: Dienstag, 29. November 2022 01:22
An: Geib, Rüdiger <[e-mail]>; [e-mail]; [e-mail]
Cc: [e-mail]; David.Black=[e-mail]
Betreff: Re: [tsvwg] WGLC: draft-ietf-tsvwg-nqb-14, closes 21 November 2022

See below [GW2].

From: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Date: Friday, November 25, 2022 at 2:15 AM
To: Greg White <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "David.Black=[e-mail]<mailto:David.Black=[e-mail]>" <David.Black=[e-mail]<mailto:David.Black=[e-mail]>>
Subject: AW: [tsvwg] WGLC: draft-ietf-tsvwg-nqb-14, closes 21 November 2022

Greg,

see [RG].

Von: Greg White <[e-mail]<mailto:[e-mail]>>
Gesendet: Dienstag, 22. November 2022 22:33
An: Bless, Roland (TM) <[e-mail]<mailto:[e-mail]>>; Geib, Rüdiger <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Cc: [e-mail]<mailto:[e-mail]>; David.Black=[e-mail]<mailto:David.Black=[e-mail]>
Betreff: Re: [tsvwg] WGLC: draft-ietf-tsvwg-nqb-14, closes 21 November 2022

See below, marked [GW].


From: tsvwg <[e-mail]<mailto:[e-mail]>> on behalf of "Bless, Roland (TM)" <[e-mail]<mailto:[e-mail]>>
Organization: Institute of Telematics, Karlsruhe Institute of Technology (KIT)
Date: Monday, November 21, 2022 at 11:57 AM
To: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "David.Black=[e-mail]<mailto:David.Black=[e-mail]>" <David.Black=[e-mail]<mailto:David.Black=[e-mail]>>
Subject: Re: [tsvwg] WGLC: draft-ietf-tsvwg-nqb-14, closes 21 November 2022

Hi Rüdiger,

sorry for the delayed reply, I'm currently on a business trip. Please see inline.

On 17.11.22 at 12:48 [e-mail]<mailto:[e-mail]> wrote:

you aren’t editor or author of the NQB draft. I still like to address you, as you are working or at least have been working with WiFi scheduling.
Actually, I haven't been working with WiFi scheduling, so I'm not the desired expert here.

To me, the idea of NQB reads similar to the EF PHB.  As NQB as of yet picks it’s DSCP to deliberately enable prioritized forwarding by WiFi, I think, traffic management as specified by EF applies. I’m referring to the deprecated text of EF here, which doesn’t carry quantitative requirements, but nevertheless seems helpful.

[GW] To be clear, the NQB DSCP was deliberately chosen to enable separate queuing from default traffic in legacy WiFi networks.  As the draft indicates (and as has been discussed extensively on this mailing list), the fact that (by default) that also gives priority in those networks is an unfortunate side effect which results in some challenges.

https://datatracker.ietf.org/doc/html/rfc2598#section-2

The following are my thoughts ( I don’t specify NQB or L4S, I hope not be wrong on L4S ):


1.       What I think applies to NQB too: the NQB scheduler departure rate MUST at all time intervals be above NQB arrival rate – and to me, a policer MUST ensure that.

[RB]So, in general you are right that for the NQB queue to be empty in average and provide low loss,
low delay, low jitter service for the flows, the departure rate must be strictly larger than the aggregate's
arrival rate. I'm actually missing a hint in the draft stating what happens if too many NQB flows enter
the queue causing overload. So this can only be prevented or enforced by policing (and admission
control). However, the way I read the draft, it is probably assumed that the traffic load and traffic usage pattern
is probably known by the provider at this part of the access network, so that there is the assumption that
this will not happen.

[GW] NQB is not a guaranteed low latency service, so I don’t think that the MUST statements above are appropriate. The draft currently recommends implementing a “traffic protection” method rather than a traditional rate policer, and the section where it is discussed (5.2) only talks about issues caused by individual flows.  I agree that we can (and probably should) say more around aggregate traffic management in certain cases.

[RG] A settable bandwidth limitation is obligatory if a scheduler allows to preempt resources. As you confirm, “the NQB DSCP was deliberately chosen to enable separate queuing from default traffic in legacy WiFi networks”. This isn’t just separate scheduling, it’s prioritized scheduling (see https://datatracker.ietf.org/doc/html/rfc8325#section-6.2.3 and 6.2.4). As consumers can’t be expected to do that, and I’m not sure whether WiFi APs offer such a capability in general, to me this must be part of the PHB specification, if the NQB remains on standards track.

[GW2] I may be misunderstanding your comments here, but to be clear, defining the PHB as a priority scheduler is not something that I’m interested in doing. I hope you are not suggesting it. That is fundamentally not what we are trying to achieve here.  As has been discussed previously, the draft covers “interoperability” with network gear that doesn’t support the PHB but instead implements a set of priority queues (legacy W-Fi is an important example).   As has become apparent to me in this exchange, we’ve not specified the management of that interoperability as completely as you would like.  That can be addressed, and I’ve suggested some of the necessary text in my original response below.

[RG2] I may be misunderstanding your draft text here, which to me reads as defining the PHB including a priority scheduler as something that is a fundamental part of the NQB draft which you are interested in doing. I note you are suggesting that:

   Applications that align with the description of NQB behavior in the

   preceding paragraphs SHOULD identify themselves to the network using

   a Diffserv Code Point (DSCP) of 45 (decimal) so that their packets

   can be queued separately from QB flows.  The choice of the value 45

   is motivated in part by the desire to achieve separate queuing in

   existing WiFi networks (see Section 8.3<https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-nqb-14#section-8.3>)
…

   In order to increase the likelihood that NQB traffic is provided a

   separate queue from QB traffic in existing WiFi equipment that uses

   the default mapping, the 45 codepoint is recommended for NQB.  This

   maps NQB to UP_5, which is in the "Video" Access Category.

[RG2] Any DSCP different from DSCP default allows separate queuing by a DiffServ compliant access- or home-network scheduler. Draft NQB requires a DSCP with the intention to access a priority scheduler in WiFi networks with technology prevalent today and within the years to come.


2.       As long as NQB isn’t clearly delineated from L4S, to me it shouldn’t cause L4S ECT 11 marks – i.e., NQB MUST ensure random collisions only with L4S by appropriate sending behaviour and policing too.

3.       Then my take is, that the NQB rate-limit should be between 30% and 50% of the L4S scheduler bandwidth, if I apply EF dimension suggestions for NQB and look at the L4S queue as what’s termed “line rate” by the EF draft.

4.       Traffic above that rate to me should be marked for default forwarding (and I suggest to set ECT 11 in addition, without prior investigation of the ECT value – to deal with the yet unspecified co-scheduling of NQB with L4S).

[GW] IIRC it would not be compliant with RFC3168, RFC8311 or ECN-L4S-ID to CE mark Not-ECT packets.

[RG] Yes, likely. In that case, please provide text on scheduling requirements if NQB traffic shares the same queue with L4S and is marked ECT 1 in addition to a set NQB DSCP, if the NQB draft remains on standards track. I’m also fine with text stating if DSCP NQB is set, a packet MUST NOT be marked ECT 1. Please also add guidelines what to do technically, if traffic is marked with an unexpected combination of DSCP and ECT. Again, I expect requirements and guidance on safe operation from a standards track doc.

[GW2] Yes, this has already been noted, and it is on my list of items to address in the post-WGLC update.


Assuming L4S to receive 50% of the bandwidth available for all default traffic, then NQB would be policed to  [0,3 … 0,5] * 0,5 = 15% to 25% of the overall default traffic.

My questions to you:

5.       would a traffic share of 15% (to 25%) of AC_VI downlink traffic as compared to traffic scheduled for default enable somewhat fair and useful WiFi operation at least under many or most operational conditions? Meaning single up to half a dozen terminals on a single WLAN which operates with almost full spectrum.

6.       Looking at https://datatracker.ietf.org/doc/html/rfc8325#section-6.2.3 , AC_BE seems to have around 10% chance of getting access to WiFi scheduling as compared to AC_VI, that’s why I’d prefer 15% as an NQB/Default traffic share guidance. Is that making sense?

[GW] FYI, all else being equal, the AC_VI:AC_BE scheduling ratio is more like 4.5:1. That said, I would think the concern is mainly around the case where the ISP could police NQB traffic somewhere in their network (e.g. a CMTS or BRAS) and doesn’t have visibility into the WiFi network bandwidth. So, we’d be mainly worried about situations where the WiFi bandwidth is significantly less than the access network rate.   As a result, 15% seems sort of high to me. Maybe 5% would be better?
[RG] Greg, also 5% is ok with me. You are the author of the draft, please provide text informing an operator which configuration enable safe deployment of the PHB, at least under prevailing operational cases (and please add information, which operational conditions of a WiFi network would not recommend activation of NQB).

[GW2] Ok.

7.       Are there operational conditions of WiFi, when NQB deployment isn’t recommended (as a non expert, I thought about multi-terminal WiFi, like a hot-spot, or locations, where the spectrum of many unmanaged WLANs overlap).

8.       As I’m not an expert on WiFi, please add important aspects that I’m not aware of.

I’m asking you these questions since this is a standards track RFC and I feel uncomfortable with the level of guidance offered by it so far for secure and reliable operation. I don’t expect things to be watertight, I’m interested in an operation which wouldn’t cause the network provider hotlines to experience a massive increase in calls, once NQB is enabled.

I agree that the draft could give more operational guidance in this respect.

[GW] For the case that you are concerned about here (downlink traffic from an ISP entering a network containing WiFi gear) I am open to providing more guidance than we currently have.  The current guidance is the following:
1)      The WiFi gear SHOULD carry NQB traffic in a separate queue in AC_BE
2)      If the above isn’t possible in certain equipment, then the WiFi network SHOULD be configured such that AC_VI is equal in priority to AC_BE.
3)      If the above aren’t possible or the conditions are unknown, then:
a.       ISP SHOULD take precautions to prevent issues caused by prioritization of excessive and/or mismarked NQB traffic
b.       ISP equipment SHOULD bleach NQB by default (so that any pass through of NQB is intentional)
c.       As an alternative to bleaching, ISP could deploy traffic protection or policing, for example policing NQB traffic to a set fraction of the interconnection data rate, with excess traffic either being dropped or sent as default.

This part 3c it seems is where more specific guidance is wanted.

[RG] Please clarify to the reader, which of scenarios 1) – 3) is the one likely prevailing in home network for the next decade. It should also be clear to the reader, what is a safe deployment, if WiFi gear is not under control of the network operator. I also wonder, whether informative on an opt-out for customers on NQB would make sense (causing operator DSCP 0 marking of NQB as a result; such advice stays informative of course, but that may help operators designing an NQB offer).

[GW2]  Hmm. I am not sure that I want to try to predict the evolution of technology over the next 10 years (much less document that prediction in an RFC), and I frankly don’t see the need to. AFAIK some current Wi-Fi 6 AP silicon can support scenario 1 (with appropriate software).  AFAIK all current Wi-Fi 5 & 6 AP silicon can support scenario 2 (with appropriate software). For WiFi 6, WFA currently recommends (requires?) support for RFC8325, which NQB updates to match scenario 1. Several companies recently proposed that the IEEE 802.11 committee adopt support for L4S and NQB in WiFi 8. How will this play out over the next 10 years? I don’t know. Is there likely to still be some gear 10 years from now that falls into scenario 3?  Probably. Will it be prevalent? Hard to say.

[RG2] If this is no concern to you, please reword the draft to require a DSCP which will most likely not cause an unlimited priority scheduler to be on an end-to-end path in the Internet in general.

Here is a proposal for consideration:
As an alternative to re-marking, the operator could deploy a traffic protection (see Section 5.2<https://www.ietf.org/archive/id/draft-ietf-tsvwg-nqb-14.html#traffic_protection>) or a shaping/policing function on NQB-marked traffic that minimizes the potential for negative impacts on Default traffic.  In the case that a traffic policing function or a rate shaping function is applied to the aggregate of NQB traffic destined to such a downstream domain, the policer/shaper rate SHOULD be set to either 5% of the interconnection data rate, or 5% of the typical rate for such interconnections, whichever is greater, with excess traffic being either dropped or re-marked and classified for Default forwarding.  A traffic policing function SHOULD allow approximately 100 ms of burst tolerance (e.g. a token bucket depth equal to 100 ms multiplied by the policer rate). A traffic shaping function SHOULD allow approximately 10 ms of burst tolerance, and approximately 50 ms of buffering.

[RG] An operator supporting NQB still needs to ensure safe operation. If this should only be done, if the network operator als operates the WiFi AP, the NQB draft needs to be clear on that. If an operator not controling customer premises WiFi AP config is foreseen to operate NQB too, guidance with requirements on likely safe operation should be part of the draft. To then the operator SHOULD deploy. The remaining text is ok.
[GW2] Ok.

[RG] I note that the “queue protection” is designed to protect the BRAS/CMTS NQB or L4S scheduler. Its task is not to limit traffic in order to prevent unfair advantage for the NQB traffic when getting prioritized access to WiFi spectrum. Please clarify that.
[GW2] You are correct. Nonetheless, in cases where the access network is the bottleneck, protecting the access network queue provides some protection against unfairness in the legacy WiFi network.

[RG2] Please be clear on that in your draft and provide text clearly stating the NQB queue protection is only working on the access node’s NQB scheduler, but not on any other scheduler apart from that. I kindly ask you to add a figure explaining that to the reader. A sketch:

[RG2] ---->----Access-Node NQB scheduler & its own Queue protection----->-----WiFi-AP AC_VI scheduler (strict priority, unlimited)-------

[RG2] ---->----Access-Node QB scheduler -------------------------------------------->-----WiFi-AP AC_BE scheduler (low priority)-------

[RG2] It must be clear to the reader, that draft NQB contains requirements putting the AC_VI priority scheduler on path (no wording obfuscating this by talk limited to “interoperability” or “separate queuing”).


[RG] If the above text is linked to an expectation of 50% access capacity NQB (or L4S), 50% of the access capacity BE, the text should say so. Different configurations may apply, if additional PHBs are deployed  on an access link. Roughly, the burst passing the policer should’n t be bigger, that the NQB schedulers Queue. I’m waiting for a response of the L4S authors on whether it’s ok if non responsive NQB traffic is causing congestion in the NQB queue (if it e.g. completely fills the L4S queue by a single burst). There’s no explicit text to that in your draft and I’d like to understand, whether L4S drops caused by and L4S queue overflowing due to an NQB burst is regarded as “non queue building”.

[GW2] I was not basing this on 50% NQB (or L4S) and 50% Default on the Access Network. See what I wrote below.  I could include some text that describes the derivation of the 5% number.

[RG2]: Yes please, engineering guidance which allows for deduction of parameters under other circumstances than those assumed by a draft is a requirement for a standards track doc to me. Further, I miss an NQB draft text section specifying the typical access bandwidth for which deployment of NQB is recommended. From your response I take the access bandwidth for which you recommend NQB deployment to as a minimum to be “typically on the order of 100 Mbps”.

[GW] In addition to this text, I think that the text around appropriate sender behavior should indicate that the 1 Mbps limit is given in the context of today’s networks “where access network data rates are typically on the order of 100 Mbps” i.e. the upper limit is approximately 1% of “typical” access network speeds.  Thus, the shaper/policer rates above work out to ~5 simultaneous maximum-rate NQB streams on access network segments that are less than the “typical” rate, and more such streams on higher rate access network segments.

[RG] this latter text addresses burstiness of NQB traffic. This discussion is within another thread. I so far failed to have understood your ideas on burstiness. As I think, this topic is very important, if an application  traffic flow must be designed to avoid queue build up, we need to take the time to agree on text helping implentors to understand what’s expected (and operators to understand how they are supposed to configure NQB).

[GW2] In terms of an individual application, the draft says the data rate should not exceed “a small fraction of the expected path capacity on an instantaneous (e.g., inter-packet, or a suitably short time interval) basis”.  I had previously argued that this guidance was reasonable, even if it was imprecise (https://mailarchive.ietf.org/arch/msg/tsvwg/46k1oWi7xt_7OslKt8PIlAYjn8w/). But, I understand that you are not comfortable with the imprecision.  While the term “inter-packet” is precise and well understood, it can be too strict. The terms “small fraction of the expected path capacity” and “suitably short time interval” are not precise. The L4S dual-queue-coupled-AQM implementations in DOCSIS gear set the minimum CE-marking threshold at 4000 bytes, whereas the pseudocode in the dualQ draft defaults this to 1 MTU instead.  Perhaps we could limit the burstiness of NQB senders to 1500 bytes.

[RG2] All of these latter statements are irrelevant for readers of draft NQB, as long as the draft text doesn’t contain specifications and requirements expressing these recommendation. How should operators or app implementers learn that? Please add suitable text to the draft. I thought we had agreed on the definition of “non-queue-building” as provided by EF / Van Jacobsen version: the rate of the NQB access network scheduler MUST be above the rate of the incoming NQB traffic at any timescale. Please confirm that or indicate disagreement (or discuss, if required).





Thoughts?

Greg





Regards,

 Roland





Von: tsvwg <[e-mail]><mailto:[e-mail]> Im Auftrag von Black, David
Gesendet: Mittwoch, 2. November 2022 04:47
An: tsvwg IETF list <[e-mail]><mailto:[e-mail]>
Betreff: [tsvwg] WGLC: draft-ietf-tsvwg-nqb-14, closes 21 November 2022


This email announces a TSVWG Working Group Last Call (WGLC) on:



A Non-Queue-Building Per-Hop Behavior (NQB PHB) for Differentiated Services

draft-ietf-tsvwg-nqb-14

https://datatracker.ietf.org/doc/draft-ietf-tsvwg-nqb/



This draft is intended to become a Proposed Standard RFC.



This WGLC will run through the end of the day on Monday, November 21.

The WG will meet in London on Monday, November 7, while this WGLC

is in progress.



Comments should be sent to the [e-mail]<mailto:[e-mail]> list, although purely

editorial comments may be sent directly to the authors. Please cc: the

WG chairs at [e-mail]<mailto:[e-mail]> if you would like the chairs to

track such editorial comments as part of the WGLC process.



No IPR disclosures have been submitted directly on this draft.



Thanks,

David, Gorry and Marten

(TSVWG Co-Chairs)