Re: [tcpPrague] TCP Prague under-utilising capacity when RTT scaling turned on

["Tilmans, Olivier (Nokia - BE/Antwerp)" <[e-mail]>]

Hi Ashutosh,

 > This is about an issue I found with the recently added RTT scaling feature
 > of TCP Prague linux kernel implementation. This was first observed with
 > commit number 7f267bd
 > <https://github.com/L4STeam/linux/commit/7f267bd591c249697f63f81b21a1755202
 > e86cc2#diff-38ce93325583f02d790276f5cafd1c42>  ( Mar 16, 2020 ) where RTT
 > scaling was turned on by default and also with future commits when we
 > explicitly enable RTT scaling.
 > 
 > The plot below shows the throughput of a single TCP Prague flow. Although
 > the available capacity was 1Gbps, TCP Prague is not able to reach that
 > point anytime during the experiment.

Thanks again for this report. The most inexplicable behavior here are the
apparent cwnd reductions that are happening while the connection is
under-utilizing the link (so should not be receiving CE marks).

I am unfortunately unable to reproduce this behavior on a phy or emulated
testbed, so would need additional information to work through this issue.

Could you confirm that this is also happening with the tip of the current
Branch? I.e., 3b63cc0 is the "future commits" you are referring to above?

In addition to the ss plots, could you:
- Confirm that this happens with both the internal TCP pacing and the fq
qdisc as pacer on the data sender
- Confirm that this happens both with and without gro/gso/tso/lro on the
endhosts (data sender and receiver, and also the aqm node as fq does not do
gro splitting prior to enqueue)?
- Confirm that the problem persists if you increase the base RTT--a simple
netem qdisc on the reverse path to add 2-5ms should be sufficient, do not
forget to disable gro/gso as that poorly interacts with netem.
- Log the reported CE marks by the AQM, as well as the
delivered_ce/received_ce counters throughout the experiment on the data
sender/receiver?
- Log eventual drops (dropped counter on AQM, and retransmission
counters on the data sender).


Thanks!


Best,
Olivier


 > 
 > The experiment settings were as follows
 > 
 > *	This experiment was done on the Cloudlab <https://www.cloudlab.us/>
 > testbed with a 3-node topology ( source, router, receiver).
 > 
 > *	The bottleneck between the router and receiver was a 1 Gbps wired
 > link ( 10 Gig interfaces , capacity restricted to 1Gbps using linux traffic
 > shaping tools (tc) ).
 > 
 > *	The flow was sent using iperf3.
 > 
 > *	The AQM at the router was a FQ qdisc with a single bucket and was
 > marking packets with ECN at a marking threshold of 5 ms. You can use the
 > following parameters with the tc-fq qdisc to replicate this setting :  fq
 > limit 5000p flow_limit 5000p orphan_mask 0 ce_threshold 5ms
 > *	The "ECN fallback on detection of classic ECN AQM" feature of TCP
 > Prague were disabled for this set of experiments.
 > *	The propagation / base delay of the setup was very low ( around 0.4
 > ms ) and no delay was added on top.
 > 
 > If interested, you can look at the ss data plots for these experiments at :
 > https://drive.google.com/drive/folders/1O6uEngxrDX5ipY71sjqr36lXCQBoZV--
 > ?usp=sharing
 > <https://drive.google..com/drive/folders/1O6uEngxrDX5ipY71sjqr36lXCQBoZV--
 > ?usp=sharing>
 > 
 > Looking forward to your comments and questions regarding these results.
 > 
 > Thank you,
 > 
 > Ashutosh Srivastava
 > First year PhD student,
 > Department of Electrical and Computer Engineering
 > NYU Tandon School of Engineering