[tsvwg] question regarding L4S claims

[Sebastian Moeller <[e-mail]>]

Dear list,

in a recent presentation, the L4S team attempted to show why L4S is a major improvement over the status quo.

Here I want to focus on the implicit claims in slide 4 of http://bobbriscoe.net/presents/2004ietf/202005L4S-brief.pdf

The claim here is that within a budget of 50ms* L4S will allow a distance between endpoints of 3200 Km, while pie or fq_codel would only allow a distance of 500Km.

I believe this analysis to be simplistic to a point of being deliberately misleading, the data presented here are:

 							
AQM:							fq_codel	L4S
Delay budget for 'responsive fee l[ms]:	50		50
min non-network delay [ms]:			-13		-13
99.9th %ile queuing delay [ms]:		-32		-4.5

Left for propagation round trip [ms];	5		32 		[Note: 50-13-4.5 actually equals 32.5, which traditionally rounds to 33ms...]
Equivalent reach in fibre [km]:			500		3200
Equivalent reach in fibre [miles]:		310		2000


This has multiple issues that I want to enumerate.

1) The budget value itself, is debatable, but since it will affect all AQMs equally I am willing to ignore that (besides raising questions about where the 50ms were derived from initially).

2) min non-network delay of 13 ms is a projected value in https://tools.ietf.org/html/draft-han-iccrg-arvr-transport-problem-01, which has not been demonstrated to be actually achievable, the reported current values were 130 to 1118 ms. This immediately makes it obvious that the comparison seems extremely optimistic.

3) 99.9th %ile queuing delay: the assumption here seems to be that only the single AQM hop actually controls all variable RTT jitter. And that 99.9 is a relevant number here, which obviously is debatable.

4) Left for propagation round trip & equivalent reaches: the 1 ms per 100Km RTT rule of thumb really only works for complete fiber runs, any conversion between photons and electrons on the way, any router and other network equipment will introduce additional variable delay, as will retransmits due to data corruption.


I would really like to see real world tests, in which RTT and RTT variance are measured and shown for a 3200Km path across the continental U.S. with an L4S AQM at the bottleneck, and I would like to see the full path RTT and jitter over time as well as as CDF, as for the supposed application it does not really matter where along the path the jitter is acquired.
More hard data, less overly optimistic back of the envelope estimates, please.


Best Regards
	Sebastian



*) I like the latency budget approach, we can haggle about the exact number, but the idea generally seems sane and sound, and both numbers used often 100 and here 50ms are too round to be realistic results of empiric experiments, but rather indicate some aggregation of empiric results with a modest amount of rounding ot result in a nice and clean number.
I also note that the cited https://tools.ietf.org/html/draft-han-iccrg-arvr-transport-problem-01#page-12 actually claims:
"AR/VR developers generally agree that MTP latency becomes imperceptible below about 20 ms [Carmack13].  However, some research
   has concluded that MTP latency MUST be less than 17ms for sensitive users [MTP-Latency-NASA]." The original Carmack13 website is gone and even the wayback machine comes up empty, but given the URL:
[Carmack13]
              Carmack, J., "Latency Mitigation Strategies", February
              2013, <https://www.twentymilliseconds.com/post/latency-mitigation-strategies/>

www.twentymilliseconds.com, I really wonder where the 50ms budget was derived from.