Re: [tsvwg] new tests of L4S RTT fairness and intra-flow latency: defaults ready for testing

[Sebastian Moeller <[e-mail]>]

Hi Jonathan,

to come back to this.


> On Nov 18, 2020, at 01:28, Jonathan Morton <[e-mail]> wrote:
> 
>> On 17 Nov, 2020, at 3:32 pm, De Schepper, Koen (Nokia - BE/Antwerp) <[e-mail]> wrote:
>> 
>> The RTT-independence was implemented, available and demonstrated several meetings ago already and as presented working very well according to our tests. The following parameters are now set as default, so can be tested out of the box:
>> 
>> All Prague flows with an RTT below 25ms will now converge to the same rate, independent of their real base RTT. This means that flows with a bigger RTT than 25 ms will never have to compete against smaller than 25ms RTT flows. 
>> 
>> Now the defaults are set, I'm looking forward to independent evaluations.
> 
> Since our tests are quite well automated, we were able to run a subset of them (all at 50Mbps) against the new defaults this evening.
> 
> I'll give you credit: there is some improvement in some of the tests.  However, we could still draw most of the same conclusions from the new data as we did from last week's data; the big-picture problems are still present and in some cases have actually deteriorated.
> 
> I'll focus on two major concerns in particular:
> 
> 1: Prague outcompetes CUBIC in DualPI2, at a common baseline RTT.  This only stops being true when the BDP is large enough for Prague to have difficulty growing to steady state in a reasonable amount of time.
> 
> With the new code, the Jain's index improves from .823 to .987 at 10ms (the advantage in both cases being to Prague), but actually worsens from .880 to .838 at 20ms, and from .936 to .890 at 80ms.  All of these are sampled after allowing two minutes for the flows to converge to steady-state.
> 
> 2: Prague vs Prague competition on differing RTTs.
> 
> Here is Figure 3 from the test report we recently posted, followed by an equivalent chart generated from the new data this evening.  Let's play spot the difference:
> 
> 
> 
> 
> I can say that the throughput ratio for Prague vs Prague via DualPI2 is, in fact, slightly improved in the new data, but it is still significantly worse even than the 16:1 ratio expected from the baseline RTTs at identical average cwnd.  In a similar test with 80ms versus 20ms RTTs, the two Prague flows also have more than the expected 4:1 throughput ratio.  I don't have an immediate explanation for that.
> 
> Notice that with both the old and new code, CodelAF gets very close to parity in throughput with the same traffic load, and that even through DualPI2, a pair of CUBIC flows is closer to parity than a pair of Prague flows.  That is not, overall, an improvement in RTT independence from switching to TCP Prague and/or DualPI2.
> 
> However, we did find an improvement in fairness, compared to the older code, when comparing 20ms vs 10ms Prague flows.  That's what you were going for, wasn't it?  A shame that, in achieving that singular success, so many other things are left unresolved.

	[SM] Would be interesting to see 20/10, as well as 26/13 with the newly changed code... My hunch is that 26/13 should look more different from 20/10 than the additional 6ms (30%) RTT offset would lead on to believe.


Best Regards
	Sebastian


> 
> I'm sure we will have the opportunity to run more tests on your future efforts.  For the moment, with limited time on our hands, this will have to do.
> 
>  - Jonathan Morton