Re: [tsvwg] Question regarding slide 3 of https://datatracker.ietf.org/meeting/106/materials/slides-106-tsvwg-sessb-31-tcp-prague-status-of-implementation-and-evaluation-00#page=3

["De Schepper, Koen (Nokia - BE/Antwerp)" <[e-mail]>]

Hi Sebastian,

We typically load the system measuring these plots with 1 flow L4S, one flow Classic (up to now labeled 1-1 in the paper), then we add 2 load levels of dynamic traffic (labeled 1h-1h for high load and 1l-1l for low load in the paper). The high load means 100 requests per second per 40Mbps throughput per traffic type (Classic & L4S) and low load would be 10 times less (10 requests per second per 40Mbps per type).

On the iccrg presentation, you see we mention these details:
● fixed Ethernet
So smoothly shaped physical layer
● long-running TCPs: 1 ECN 1 non-ECN 
1 flow of each traffic type
● web-like flows @ 300/s ECN, 300/s non-ECN 
So this is for the 1h-1h case, where there are 3 times the 100 requests per second, which means for a link of 3 times 40Mbps = 120Mbps (see further) and not for the 1l-1l which would be for 30 times the 40Mbps
● exponential arrival process
For reproducibility and fair comparison, we use a prepared scenario file that is generated as a exponential distribution with a average arrival rate of 100 requests per second in this (1h) case, played our 3 times as fast
● file sizes Pareto distr. α=0.9 1KB min 1MB max
Again for reproducibility and fair comparison, we use a prepared request size file with the parameters specified
● 120Mb/s 10ms base RTT
For this network throughput and base RTT

In this slide we grouped 3 different experiments:
- DualPI2 using for the L4S/ECN flows DCTCP (a good version like the original DCTCP in kernel 3.19 or Prague now for the recent kernels (we didn't maintain DCTCP to be aligned with Prague)) and classic flows Cubic without ECN
- FQ_Codel using for the ECN flows Cubic with ECN enabled and classic flows Cubic without ECN
- PIE using for the ECN flows Cubic with ECN enabled and classic flows Cubic without ECN
So here PIE and FQ_Codel don't use L4S.

The paper plots are using the same network and traffic conditions (for the first row, while the second row is the 1l-1l case), but with slightly different AQM mechanisms and traffic type combinations.
The difference in the paper are for the first DualPI2 DCTCP/CUBIC experiment, in the presentation we had the time-shifted FIFO (you can see that the latency of Classic is never later than that of L4S + about 30ms and v.v.), while if the WRR priority scheduler is used, the latencies are completely decoupled (so L4S even lower latencies and Classic not limited, only by their own congestion control).

For the PIE there shouldn't be much difference besides the typical variance we see below 99.9 percentiles for the classic traffic type (L4S is more consistent up to 99.999).

For FQ_Codel the experiment is completely different, as the ECN-Cubic is replaced by the DCTCP traffic and for ECT(1) we used the immediate 1ms threshold (you are correct we forgot to mention in Table 1, thanks for noticing) instead of the Codel-ECN in the presentation for managing ECN-Cubic.

Hope this clarifies up to here.

Probably you wonder why the latency of the L4S is much bigger in case of FQ, compared to DualQ, which is simple to explain:
- In FQ every flow has its own queue AND AQM with a threshold of 1ms. So all flows are controlled around 1ms, while in L4S they just find usually an empty queue without the 1ms threshold being hit at all, because there is Classic traffic that keeps the L4S flows below the link capacity via the AQM coupling.
- When dynamic flows kick in, there will be many Q's that need to be scheduled, plus new flows or "empty-Q" flows will get priority over the 1ms L4S flows. This means that also L4S flows have to wait much more in FQ than in DualQ. First in their own queue of around 1ms and second for their queue getting scheduled in the RR scheduler. Additionally their bandwidth is aggressively varying at a micro scale when flow queues build up and disappear and when flows start and their packets get priority. This causes additional jitter for the FQ-L4S flows.

So from this comes my insight: use FQ if you think fair rate is most important, use DualQ if you think smooth throughput (low rate jitter)) and low latency (jitter) is most important.

Hope this clarifies the difference between FQ and DualQ (and their results) too, and allow you to reproduce these results. Note that these delays are measured per packet and is only the sojourn time between enqueue and dequeue (so exact queue delay CDF per packet), not anu averaged/smoothed delay that is typically used in some test frameworks which make it impossible to spot these 99.999 percentile delays.

See also inline for some of your specific questions.

Koen.

-----Original Message-----
From: tsvwg <[e-mail]> On Behalf Of Sebastian Moeller
Sent: Wednesday, June 2, 2021 12:13 PM
To: TSVWG <[e-mail]>
Subject: [tsvwg] Question regarding slide 3 of https://datatracker.ietf.org/meeting/106/materials/slides-106-tsvwg-sessb-31-tcp-prague-status-of-implementation-and-evaluation-00#page=3

Hi Bob, Koen,

I recently had a look at https://datatracker.ietf.org/meeting/106/materials/slides-106-tsvwg-sessb-31-tcp-prague-status-of-implementation-and-evaluation-00#page=3 again, and I wonder whether you could share more information about the data/experiments underlaying that figure.

It looks similar to Figure 7 of https://www.bobbriscoe.net/projects/latency/dctth_journal_draft20190726.pdf, except the individual CDFs look slightly different. Figure7 does not have much of a legend and is also not referenced at all in the manuscript. Could you share the exact test conditions, as well as the modifications to fq_codel you mention in the manuscript as well as the ce_threshold value you used (assuming you used that at all), please?

[K] You are right, seems we forgot to mention we used a 1ms threshold for FQ_Codel in the manuscript in Table 1. But besides that:
"(we) used a modified version
of FQ-CoDel, where L4S support was added by using a shallow
ECN marking threshold for any ECT(1) packet, with an
additional check to ensure that the threshold is only applied
if there is more than 1 packet in the queue. The queue length
check was added to prevent 100% marking at lower link rates,
considering that packet serialisation in such cases takes longer"

Also, in that manuscript you seemed to have used fq_codel with target 5ms and interval 100ms and compared it against DualPI2 with a 1 ms ref_delay target" for the L-queue. How does your modified fq_codel stack up in comparison, how are target and interval interpreted for ECT(1) traffic?

[K] We really used a 1ms threshold for ECT[1], so there is a direct head to head comparison possible in the paper (otherwise the 80% percentile would be rather around 5ms instead of 1ms).

Best Regards
	Sebastian