Re: [tsvwg] draft-ietf-tsvwg-nqb-15.txt - 5.1 Primary Requirements, Forwarding - Departure rate

[[e-mail]]

Hi Sebastian,

here we discuss NQB native, and afaik, a 50/50 WRR scheduling, if deployed by WRR (although Greg's latest response wasn't clear about that, it had a "whatever" clause). So L4S isn't the measure.

Traffic protection, at least the one that has been published, re-schedules packets, rather than flows. Also Greg's NQB draft argues by re-ordering as an incentive to mark correctly. 
https://datatracker.ietf.org/doc/html/draft-briscoe-docsis-q-protection-06#name-rationale-for-reclassificat

To me, this traffic protection sounds a bit like some kind of FQ mechanism, with the option to re-schedule packets rather than drop them (as compared to FQ-Codel). To me, traffic protection + NQB would sum up to a hierarchical scheduler then (FQ followed by an NQB/QB scheduler - I didn't worry about detailed specs of the NQB-FQ/QB traffic part, but if you operate FQ by x ms <= target <= 5 ms for the NQB portion, using the NQB scheduler rate or a slightly smaller one as the resource to share for these flows, and move away offending packets into the QB scheduler, you seem to have largely made it - as I see in the text referenced above, QB "has a latency target of 10ms", if you want, that's 2 (schedulers) * target 5 ms each). The NQB scheduler following the NQB FQ traffic protection wouldn't see an additional queue then, as the arrivals are shaped. I'm not an FQ expert however and may have misperceptions.

Regards, Ruediger

PS: I agree, single flows are unlikely to congest single access lines and RTT is important, while bandwidth isn't any longer. In many urban regions.

-----Ursprüngliche Nachricht-----
Von: Sebastian Moeller <[e-mail]> 
Gesendet: Donnerstag, 6. April 2023 10:15
An: Geib, Rüdiger <[e-mail]>
Cc: Greg White <[e-mail]>; [e-mail]
Betreff: Re: [tsvwg] draft-ietf-tsvwg-nqb-15.txt - 5.1 Primary Requirements, Forwarding - Departure rate

Hi Ruediger,

> On Apr 6, 2023, at 09:30, <[e-mail]> <[e-mail]> wrote:
> 
> Hi Sebastian,
> 
> I haven't been involved in the measurements characterizing L4S. I'd hope (but I'm not sure) that the L4S queue coupling results in roughly equal congestion in both queues, once one is starting to be congested.

	[SM] This only holds for responsive traffic in the LL-queue. The queue is designed to not drop packet, only mark or, if queue protection is enabled, to move a full "flow" into the non-LL queue. For non-responsive traffic (the type NQB is intended for) there os no push back, as far as I understand. Pete's measurements already demonstrated a number of cases where all that conserves some non-LL traffic throughput is the ~10% default weight for the non-LL queue. So no, the queue coupling is not robust and reliable (a fact the L4S designers implicitly conceded by putting in the weighted scheduler back-stop I would argue). Now, it is possible that alternative L$S-AQMs could avoid this problem, but neither DUalQ nor the related AQM set-up described in the DOCSIS specifications are such alternative implementations and both seem to be easily exploitable.


> Also, if unresponsive traffic is present in the L queue (I'd wonder how a single first C flow would be able to kick in, while standard conformant L4S flows already saturate a link).

	[SM] Well, DualQ specifies a weighted scheduler LL/non-LL 90/10 as back-stop. I assume that would trigger in such a situation. Also if the non-responsive flow's rate exceeds 90% of the capacity the optional queue-protection might kick in, but with faster and faster access links, it becomes IMHO more likely that a traffic source might end up being application limited ad hence might well stay below the critical 90% mark (note that these 90% are for the aggregate LL queue traffic, so if there are multiple flows in the LL-queue they need to share that capacity).


> Otherwise, the C queue will be pretty much starved. That said - I'm not an L4S expert and rely upon the measurements executed during evaluation.

	[SM] These measurements where IMHO performed (see https://github.com/heistp/l4s-tests), and confirmed that under reasonable (but non-optimal) conditions the L4S DualQ falls back to the configured weights of the back-stop scheduler. What was controversial is whether that condition would technically qualify as "starvation" and whether it was sufficiently likely to matter, however none of the L4S proponents was willing to offer a stringent definition of what "starvation" actually means let alone "sufficiently likely". At least that is my recollection*. 
	In L4S' defense, it does offer methods/heuristics that with responsive traffic of similar RTTs will end up sharing roughly equitably between the two queues, albeit not robustly or reliably...

Regards
	Sebastian

*) I was and am on record that L4S should never have been granted RFC status, so I am not unbiased on this topic.


> 
> Regards,
> 
> Ruediger
> 
> -----Ursprüngliche Nachricht-----
> Von: Sebastian Moeller <[e-mail]>
> Gesendet: Donnerstag, 6. April 2023 08:41
> An: Geib, Rüdiger <[e-mail]>
> Cc: Greg White <[e-mail]>; [e-mail]
> Betreff: Re: [tsvwg] draft-ietf-tsvwg-nqb-15.txt - 5.1 Primary 
> Requirements, Forwarding - Departure rate
> 
> Hi Ruediger,
> 
>> On Apr 6, 2023, at 08:24, <[e-mail]> <[e-mail]> wrote:
>> 
>> Hi Greg,
>> 
>> draft NQB is on standards track. Please specify the weights to be set for NQB and QB scheduler by requirements:
>> - some generic text, not based on any implementation (which I think is roughly there). I'd appreciate text stating that QB/NQB share the same overall resources, are configured by the same priority, weight (or minimum departure rate) and also are equipped by the same priority and weight to access spare capacity. There's some text in the draft, but it is not very precise.
>> 
>> - to that a clarification related to your response: you write [GW] It 
>> can be set to whatever ratio the network operator chooses (e.g. 50/50) in the case that L4S support is disabled.
>> Please clarify "whatever" in the sense of standard track NQB: which range of NQB/QB WRR weight configurations is compliant with this draft standard? My perception was exactly 50/50, but "whatever" seems to allow for arbitrary configurations.
>> - I'd strongly suggest that you provide an example traceable for a fair share of readers, which from my perception is good practice of other RFC authors. You reference the DOCSIS L4S implementation, a WRR scheduler with 50/50 weights (and the same for access to spare capacity) seem good to me.
> 
> 	[SM] The DOCSIS scheduler defaults to 90% LL-queue (used for NQB) and 10% classic; this is supposed to be OK as L4S-flows will scale down quickly on cross-pressure from the classic queue. However unresponsive NQB traffic is not expected to scale back and hence will be able to gain ~90% of capacity if sufficiently well-paced to avoid the optional queue-protection functionality (that does not asster maximal per-flow rates, but simply looks at the queueing caused by a flow, which can be minimized by high-precision pacing).
> 	I think the NQB draft should mention this in the "3.3. Relationship to L4S" section explicitly*. The point here is that just moving NQB traffic onto an L4s AQM relies fully on the sender of NQB traffic to voluntarily follow the recommended rate limits (which as we discussed separately are hard to get right, how should n endpoint using rate-limited traffic ever know what the "true" link capacity actually is to scale its permissible rate to, but that is a different kettle of fish). 
> 
> 
> Regards
> 	Sebastian
> 
> 
> *) With that I mean that a standard compliant L4S AQM will not assert that NQB and QB traffic share a link equitably and that the imbalance can be as high as the scheduler weights for a dualQ AQM imply. Sure there are other potential implementations for an L4S AQM, but e.g. for the one described in the DOCSIS specifications with its default 100*230/256 = 89.84375% LL weight it seems worth mentioning that NQB traffic can gain up to 90% of capacity, OR if I happen to be incorrect in my interpretation, WHY the NQB class will not be able to gain an unfair advantage.
> 
> 
>> 
>> Regards,
>> 
>> Ruediger
>> 
>> 
>> -----Ursprüngliche Nachricht-----
>> Von: Greg White <[e-mail]>
>> Gesendet: Donnerstag, 6. April 2023 01:57
>> An: Geib, Rüdiger <[e-mail]>
>> Cc: [e-mail]
>> Betreff: Re: [tsvwg] draft-ietf-tsvwg-nqb-15.txt - 5.1 Primary 
>> Requirements, Forwarding - Departure rate
>> 
>> Hi Ruediger,
>> See my responses below [GW].
>> -Greg
>> 
>> On 4/5/23, 6:12 AM, "[e-mail] <mailto:[e-mail]>" <[e-mail] <mailto:[e-mail]>> wrote:
>> Hi Greg,
>> 
>> Section "7.7.3.2 Inter-SF Scheduler" of CM-SP-MULPIv3.1-I24-221019 contains the following statement:
>> 
>> coupling .... the Classic Service Flow to the Low Latency Service Flow, it relies on the Inter-SF Scheduler to balance this. Weighted Round Robin (WRR) is a simple scheduler that achieves the desired results, and is recommended in [draft-ietf-tsvwg-aqm-dualq-coupled].
>> 
>> The above text covers L4S, not straight NQB.
>> - Please explain how this WRR scheduler is configured to support straight NQB/QB without L4S being configured.
>> [GW] The WRR in DOCSIS has a configurable weight.  It can be set to whatever ratio the network operator chooses (e.g. 50/50) in the case that L4S support is disabled.
>> 
>> - If there's no WRR scheduler, then please explain how an implementer ensures that NQB and QB fairly share the same resource, while each operate with separate queues. I'm especially interested in the part "no configurable service rate/weight etc." for the NQB queue. An example is sufficient, maybe one including a WRR scheduler, if applicable.
>> [GW] Aside from WRR mentioned above, perhaps a TS-FIFO could be used?  I have to admit that I've not thought about other scheduler options extensively.
>> 
>> - if WRR can't be used to realise separate NQB/QB queues for an implementation, please let me know, why this isn't possible.
>> 
>> Regards,
>> Ruediger
>> 
>> 
>> -----Ursprüngliche Nachricht-----
>> Von: Greg White <[e-mail] 
>> <mailto:[e-mail]>>
>> Gesendet: Freitag, 24. März 2023 21:24
>> An: Geib, Rüdiger <[e-mail] 
>> <mailto:[e-mail]>>
>> Cc: [e-mail] <mailto:[e-mail]>
>> Betreff: Re: [tsvwg] draft-ietf-tsvwg-nqb-15.txt - 5.1 Primary 
>> Requirements, Forwarding
>> 
>> 
>> Hi Ruediger,
>> 
>> 
>> FYI I've added an issue in the GitHub tracker to ensure this gets resolved.
>> https://github.com/gwhiteCL/NQBdraft/issues/32
>> <https://github.com/gwhiteCL/NQBdraft/issues/32>
>> 
>> 
>> 
>> 
>> I'll try to answer your question.
>> 
>> 
>> [RFC2598]: The EF PHB is defined as a forwarding treatment for a 
>> particular diffserv aggregate where the departure rate of the 
>> aggregate's packets from any diffserv node must equal or exceed a 
>> configurable rate. The EF traffic SHOULD receive this rate 
>> independent of the intensity of any other traffic attempting to transit the node.
>> It SHOULD average at least the configured rate when measured over any 
>> time interval equal to or longer than the time it takes to send an 
>> output link MTU sized packet at the configured rate. (Behavior at 
>> time scales shorter than a packet time at the configured rate is 
>> deliberately not specified.) The configured minimum rate MUST be 
>> settable by a network administrator (using whatever mechanism the 
>> node supports for non-volatile configuration).
>> 
>> 
>> [NQB]: ... the NQB PHB provides a shallow-buffered, best-effort service as a complement to a Default deep-buffered best-effort service. ... A node supporting the NQB PHB makes no guarantees on latency or data rate for NQB-marked flows, but instead aims to provide an upper-bound to queuing delay for as many such marked flows as it can and shed load when needed.
>> 
>> 
>> So, NQB PHB does not have a configurable departure rate, nor does it guarantee that NQB traffic will receive any particular departure rate, regardless of the presence of other traffic of any intensity.
>> 
>> 
>> <snip>
>