Re: [tsvwg] new tests of L4S RTT fairness and intra-flow latency: defaults ready for testing

[Sebastian Moeller <[e-mail]>]

Hi Jonathan,

The changes that Oliver submitted to TCP Prague not only increase the fudge RTT to 25ms, but they also, as Koen mentioned, increase the time it takes for Prague to switch to the new fairness mode to 500 RTTs, or 5 seconds at 10ms RTT.
I really wonder how that affects fairness if the shallow queue is used by a bunch of short flows with 10ms RTT. As far as I can tell the consequences of this delayed engagement have not been described properly. 
@Koen, @Oliver, in the kernel code you hint at a paper about your RTT independence method. It would be great if you could post links to your analysis how this transition period affects sharing between flows inside the shallow queue and across queues. Ideally that data would show data for the old 100 RTT transition delay, as well for the new value of 500 RTTs.

My fear is that this will now give transient TCP Prague flows an undeserved advantage (for longer, that transition was part of the RTT independence code before), and just because that condition has not been tested by external parties yet, does not mean that it is no matter of concern. In fact that lack of external testing is rather cause for concern, as so far almost all external testing found problem spots in L4S almost immediately.
In fact the non-chalance in which these RTT independence parameters where changed apparently with an email to this list, and without any note that the functionality of these new parameters had actually been empirically verified.

Best Regards
        Sebastian



On 18 November 2020 01:28:05 CET, Jonathan Morton <[e-mail]> wrote:
>> On 17 Nov, 2020, at 3:32 pm, De Schepper, Koen (Nokia - BE/Antwerp)
><[e-mail]> wrote:
>> 
>> The RTT-independence was implemented, available and demonstrated
>several meetings ago already and as presented working very well
>according to our tests. The following parameters are now set as
>default, so can be tested out of the box:
>> 
>> All Prague flows with an RTT below 25ms will now converge to the same
>rate, independent of their real base RTT. This means that flows with a
>bigger RTT than 25 ms will never have to compete against smaller than
>25ms RTT flows. 
>> 
>> Now the defaults are set, I'm looking forward to independent
>evaluations.
>
>Since our tests are quite well automated, we were able to run a subset
>of them (all at 50Mbps) against the new defaults this evening.
>
>I'll give you credit: there is some improvement in some of the tests. 
>However, we could still draw most of the same conclusions from the new
>data as we did from last week's data; the big-picture problems are
>still present and in some cases have actually deteriorated.
>
>I'll focus on two major concerns in particular:
>
>1: Prague outcompetes CUBIC in DualPI2, at a common baseline RTT.  This
>only stops being true when the BDP is large enough for Prague to have
>difficulty growing to steady state in a reasonable amount of time.
>
>With the new code, the Jain's index improves from .823 to .987 at 10ms
>(the advantage in both cases being to Prague), but actually worsens
>from .880 to .838 at 20ms, and from .936 to .890 at 80ms.  All of these
>are sampled after allowing two minutes for the flows to converge to
>steady-state.
>
>2: Prague vs Prague competition on differing RTTs.
>
>Here is Figure 3 from the test report we recently posted, followed by
>an equivalent chart generated from the new data this evening.  Let's
>play spot the difference:
>
>
>
>
>I can say that the throughput ratio for Prague vs Prague via DualPI2
>is, in fact, slightly improved in the new data, but it is still
>significantly worse even than the 16:1 ratio expected from the baseline
>RTTs at identical average cwnd.  In a similar test with 80ms versus
>20ms RTTs, the two Prague flows also have more than the expected 4:1
>throughput ratio.  I don't have an immediate explanation for that.
>
>Notice that with both the old and new code, CodelAF gets very close to
>parity in throughput with the same traffic load, and that even through
>DualPI2, a pair of CUBIC flows is closer to parity than a pair of
>Prague flows.  That is not, overall, an improvement in RTT independence
>from switching to TCP Prague and/or DualPI2.
>
>However, we did find an improvement in fairness, compared to the older
>code, when comparing 20ms vs 10ms Prague flows.  That's what you were
>going for, wasn't it?  A shame that, in achieving that singular
>success, so many other things are left unresolved.
>
>I'm sure we will have the opportunity to run more tests on your future
>efforts.  For the moment, with limited time on our hands, this will
>have to do.
>
> - Jonathan Morton

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.