[tsvwg] Re: WGLC for A NQB PHB for Differentiated Services (draft-ietf-tsvwg-nqb) to end 27th May 2024.

[[e-mail]]

Folks,

NQB/L4S-Queue: I think 125 packets of 200 Byte per second which are forwarded when received cause an additional buffer delay of 0.008 ms for default traffic which is waiting to be forwarded on a 200 Mbit/s link (each single packet adds this amount). If there's a collision at all.

If a 200 kbit/s flow and a Cubic flow share the same queue, the process is harder to be described. TCP creates bursty traffic. A single 1500 Byte packet creates 0.06 ms queuing delay. Assuming the backbone-link and DC interfaces to be of Gbit/s bandwidth, any single 200 Byte packet may face up to one ms of additional delay, if queued behind a burst of Cubic traffic. In the absence of congestion, there's likely no serious impact of the gaming flow on the Cubic flow, if the gaming flow is priority queued. If both are queued in the default queue, the gaming traffic may start to experience additional queuing delay, if the Cubic traffic starts to build a standing queue.

The 200 kbit/s flow will not be reduced to a fair share of 199,8 kbit/s, if priority queued and competing with default traffic. 

Regards,

Ruediger

-----Ursprüngliche Nachricht-----
Von: Sebastian Moeller <moeller0=[e-mail]> 
Gesendet: Mittwoch, 22. Mai 2024 08:17
An: Greg White <g.white=[e-mail]>
Cc: Sebastian Moeller <moeller0=[e-mail]>; Livingood, Jason <jason_livingood=[e-mail]>; [e-mail]
Betreff: [tsvwg] Re: WGLC for A NQB PHB for Differentiated Services (draft-ietf-tsvwg-nqb) to end 27th May 2024.

Hi Greg,

see [SM2] below.


> On 21. May 2024, at 23:35, Greg White <g.white=[e-mail]> wrote:
> 
> Sebastian,
> See below [GW].
> 
> On 5/21/24, 10:01 AM, "Sebastian Moeller" <moeller0=[e-mail] <mailto:[e-mail]>> wrote:
> Hi Jason,
>> On 21. May 2024, at 15:57, Livingood, Jason <jason_livingood=[e-mail] <mailto:[e-mail]>> wrote:
>> 
>> On 5/21/24, 02:34, "Sebastian Moeller" wrote:
>> 
>>> [SM] Could you be a bit more explicit, what exactly did you test, what was posiztive and did yu also look for side effects?
>> 
>> While you directed this question to Michael, we have tested this as well in our field trials. We have for example, tested single queue with AQM and looked at p99 latency under load and jitter as well as median & max bitrate for different application flows - then turned on NQB LL and observed the same for the classic and LL queues. What we have found generally is that the bitrates, working latency, and jitter for classic traffic with single queue AQM and dual queue LL is the same; classic queue applications do not experience degradation.
> 
> 
> [SM] Puzzled, how can the bitrate in the C-queue not change when there is also quantitative traffic in the L-queue?
> 
> [GW] This shouldn't be puzzling. See below.  

[SM2]	Well, it is not puzzling, but simply not true as you state below, for a link of capacity X with (for simpler argument) with the C-quueue capacity at Y=X if we add a flow of capacity Z to the higher priority L-queue the capacity available for the C-queue will be X-Z... and so if you can not measure an effect of L-traffic n the C-queue bitrate, respectfully you need to reconsider your measurement approach.
You can IMHO rightfully argue that for low capacity L-traffic that is not significant change in C-traffic capacity, but claiming there is NO change is simply incorrect. You could also try to argue that the same would happen if that low capacity flow would be added to the C-queue, but that is also not quite correct, because the same traffic added to the C-queue would be subject to the different congestion signalling (e.g. a non-zero probability of being dropped in a single queue bottleneck, but I would guess this effect to be really small, but still really small != 0).
In short, please be precise in what you claim.


> 
> 
>> At the same time, the NQB flow experiences working latency that approaches idle latency, with consistently low latency.
> 
> [SM] Well, we know that priority scheduling does work and will give a sufficiently sparse traffic in the priority class shorter delays (and more potential throughput)... so that is not all that surprising.
> Surprising is that according to your argument that improvement in service did not come at a cost for other traffic. And I question that, priority scheduling does not magically create more capacity or transmit opportunities, for every packet you treat to less delay, some other packet(s) will be treated to more delay. We can argue that the amount of this additional delay is insignificant, but that is a different claim than 'it does not exist' as you seem to be making above.
> 
> [GW] That isn't true.

[SM2] A work conserving scheduler between two queues essentially plays a zero-sum game in transmit-slot acquisition, any time an L-queue packet is sent the C-queue is transiently stalled and hence the proto-sojourn time of the next C-quuee packet gets increased. Unless the head packet of the C-queue is dropped with 100% probability that stall will result in an increase in real-sojourn time and hence delay in the C-queue*. And as far as I understand the mechanism by which the C-queue increases the drop probability to adjust the C-queue bitrate (to make room for the L-traffic) is driven by a good part by increases in sojourn time...
I do think your claim "That isn't true" in itself is incorrect. Again, you can argue that this effect is insignificant (as I say above) or small enough to be hard to measure, but claiming as you apparently do this effect does not exist requires more than just generalised argument over long term averages.

*) And for e.g. TCP such an immediate drop can have drastic consequences on the achievable throughput... but I digress.

> A simple thought experiment

[SM2]	Not sure a simple thought experiment is helpful, as you simplified away the mechanistic issues that make your claim incorrect...


> case is a 100 kbps gaming flow (200B @ 60Hz) sharing a 200 Mbps bottleneck with a Cubic TCP flow. Whether these two flows share a single FIFO or are separately queued in an NQB instance, the gaming flow consumes 100 kbps and the Cubic flow can consume the rest (199.9 Mbps).

[SM2] Yes, but here is the kicker, in case both are in the C-queue, the C-queue bitrate is 200 Mbps, in case they are in different queues, the C-queue bitrate is 199.9 and since 200 > 199.9 the claim the C-queue bitrate stays the same is false. (And the amount of significance depends on how much traffic we assume flows in the L-queue).


> There is no difference in capacity available to the Cubic flow.

[SM2]	Mmmh, instead of admitting that C-queue bitrate gets reduced by the L-queue bitrate you now look at an individual competing flow depending on whether the competing flow is in the L- or C-queue. But that is not what Jason claimed, and also not what I found puzzling.


>  In the single FIFO case, with DOCSIS-PIE AQM, the Cubic flow might induce a packet loss every (say) 1.4 million packets, which will cause a single packet drop in the game flow once every 48 hours, so I suppose the single FIFO does reduce the gaming flow rate by 800 bpd (bits per day), and the Cubic flow could take advantage of that. The delay of the Cubic flow is controlled by the AQM, which has a target of 10ms in both cases, so there is no difference in Cubic delay caused by using NQB.    

[SM2]	Well, now you are arguing by significance, which IMHO is a valid argument, but not the one Jason made or I reacted too.

> 
> [GW] Even if the NQB marked flow was non-compliant and was running at 180 Mbps, the Cubic TCP flow would have essentially 20 Mbps either way.  In this case the NQB flow would see 0.007% packet loss in the single FIFO option so that does give the Cubic flow an additional 13 kbps, so I guess it is 20.013 Mbps vs 20 Mbps. So, yes the bitrate is slightly different.  As above, the delay of the Cubic flow is controlled by AQM so remains the same whether single queue or dual queue.  

[SM2]	See above, sure over longer periods this will average mostly out, but mechanistically it is still true that in scheduling there is no free lunch and for every packet treated to lower delay other packets will need to be treated to longer delay (or the ultimate delay: a drop) unless we allow for potentially infinite sized queues...

I am still puzzled why any of this is apparently worth discussing, Jason's claim, as made, is demonstrably incorrect, so why are you arguing the claim is correct?

> 
> [GW] If the NQB marked flow was even more non-compliant, and was 
> running at greater than 180 Mbps, the NQB implementation could potentially provide some protection to the Cubic flow, depending on whether and what kind of Traffic Protection function is provided.

[SM] Well, potentially we could do a lot of things... but in the experiments that Jason performed, I still do not believe that there was no effect of L-traffic on C-capacity so either:
a) the measurement system was not precise enough to measure this*
b) the argument was misstated and did not actually means C-queue bitrate.

*) Which also tells me that likely nobody tried measuring what happens if non-compliant traffic enters the L-queue, which in turn makes the whole report of "works as designed" a lot weaker, there was little question in my mind whether a priority scheduler would work * but a big question how this L4S+NQB experiment would fare with realistic internet traffic, which at times can be adversarial.

Regards
	Sebastian 

*) After all this is the time tested work-horse of more traditional QoS systems


> 
> 
> Regards
> Sebastian
> 
> 
>> 
>> JL