Re: [L4s-discuss] Configuring a L4S test plant

[Sebastian Moeller <[e-mail]>]

Hi Matteo.

> On Oct 3, 2023, at 15:42, Matteo Guarna S303434 <[e-mail]> wrote:
> 
> Greetings everyone,
> 
> I hope the question isn't too off-topic, please forgive me in advance if it is so.
> 
> I am still trying to perform some fairness measurements with both L4S and classic flow, although now on a physical test plant instead of a virtualized one. I'm relying on the L4STeam Github project for the deployment of the L4S architecture and I am looking for someone who's familiar with the project and might be willing to help me: in fact I seem not to be able to achieve the correct configuration.
> 
> My setup is very simple: I have four servers (two senders and two receivers) exchanging two traffic flows through one server acting as a router. One client-server pair uses Prague as CC, while the other uses Cubic. All servers have the patched kernel provided in the https://github.com/L4STeam/linux/ repository branch.
> 
> If I trigger a congestion on the router by generating both the Prague and the Cubic flows (let's say the flows measure 100 Mbit/s each, and they come though a L2 switch both on the same router's input interface on a 1Gb Ethernet link; only a 100M link though is in place on the output interface towards the receivers) I see the L4S flow having higher delay, higher jitter and a smaller (and more variable) bandwidth share. The Prague share is 1/4 of the Cubic share. I am sending an attachment with a graphical representation of the scenario here described.
> 
> I configured my L4S endpoints as follows:
> 
> - I set the CC as tcp Prague (sysctl -w net.ipv4.tcp_congestion_control=prague)
> - I set the AccEcn, even if it's not necessary apparently (sysctl -w net.ipv4.tcp_ecn=3)
> - I disabled the required offloading capabilities on the endpoints (sudo ethtool -K $NETIF tso off gso off gro off lro off)

	[SM] I think you need to do the same on the router... or with your topology with running prague and cubic over separate end-points especially on the router itself. Side-node, sch_cake grew a split-gso mode to automatically handle this issue because it can be a bit of a whack-a-mole problem to make these configs stick (and in the case of cake the idea was to make deployment easy even for non-experts).


> - I configured the fair queue on the endpoints (sudo tc qdisc replace dev $NETIF root fq)
> 
> I configured my router as follows:
> 
> - I enabled forwarding through these interfaces to obtain the routing capabilities (sudo sysctl -w net.ipv4.ip_forward=1)
> - I set the dualpi2 on both interfaces (sudo tc qdisc replace dev $NETIF root dualpi2)
> 
> I then applied the fair queue and disabled the offloading capabilities on both my classic endpoints to ensure that the classic and l4s flows act as fairly as possible, but to no avail (even without these precautions the results remain roughly the same).

	[SM] Again, I think with your topology offloads at the endpoints should not have much influence, but at the router the well might. If that turns out to help this might be explained by Prague's (and/or DualQ's L-queue) considerably higher sensitivity to bursty traffic compared to classic traffic and queue.

> 
> I am sure I am missing some important details in the setup, and I would really appreciate some help.

	[SM] To me this looks rather straight forward, and I probably would try something similar, but I did not actually try in practice.

Regards & good luck
	Sebastian


> 
> Regards,
> 
> Matteo
> 
> P.s.
> I just want to point out that by looking at the packet traces everything seems fine: Prague carries the ECN=1, the dualpi2 marks packets with ECN=3, the AccEcn control signals on the ACE fields are coherent, and no losses occur in the Prague flow, while they do happen with the Cubic flow. It looks like Prague is underperforming for whatever reason. Furthermore, if I switch back to two Cubic flows I measure perfect share, equal delay and equal jitter, so it looks to me like there are no physical impairments on the testbed.<testplant_issue.pdf>-- 
> L4s-discuss mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/l4s-discuss