Re: [tsvwg] question regarding L4S claims

[Sebastian Moeller <[e-mail]>]

Addendum:

http://www.altdevblogaday.com/2013/02/22/latency-mitigation-strategies/

Has a copy of Carmack's assessment:
"“A total system latency of 50 milliseconds will feel responsive, but still subtly lagging. One of the easiest ways to see the effects of latency in a head mounted display is to roll your head side to side along the view vector while looking at a clear vertical edge. Latency will show up as an apparent tilting of the vertical line with the head motion; the view feels “dragged along” with the head motion. When the latency is low enough, the virtual world convincingly feels like you are simply rotating your view of a stable world."

So according to Carmack the goal for immersive VR/AR should be 20ms:
“It’s twenty milliseconds, or go home to play with your space rockets, which are probably an easier task than a genuine virtual reality headmounted display."

"Human sensory systems can detect very small relative delays in parts of the visual or, especially, audio fields, but when absolute delays are below approximately 20 milliseconds they are generally imperceptible."

So I respectfully argue that ig Carmack is dragged into this discussion, then please use his preferred reference value of 20ms, or refer to actual psychophysical research.


Best Regards
	Sebastian




> On Jun 4, 2020, at 11:27, Sebastian Moeller <[e-mail]> wrote:
> 
> Dear list,
> 
> in a recent presentation, the L4S team attempted to show why L4S is a major improvement over the status quo.
> 
> Here I want to focus on the implicit claims in slide 4 of http://bobbriscoe.net/presents/2004ietf/202005L4S-brief.pdf
> 
> The claim here is that within a budget of 50ms* L4S will allow a distance between endpoints of 3200 Km, while pie or fq_codel would only allow a distance of 500Km.
> 
> I believe this analysis to be simplistic to a point of being deliberately misleading, the data presented here are:
> 
>  							
> AQM:							fq_codel	L4S
> Delay budget for 'responsive fee l[ms]:	50		50
> min non-network delay [ms]:			-13		-13
> 99.9th %ile queuing delay [ms]:		-32		-4.5
> 
> Left for propagation round trip [ms];	5		32 		[Note: 50-13-4.5 actually equals 32.5, which traditionally rounds to 33ms...]
> Equivalent reach in fibre [km]:			500		3200
> Equivalent reach in fibre [miles]:		310		2000
> 
> 
> This has multiple issues that I want to enumerate.
> 
> 1) The budget value itself, is debatable, but since it will affect all AQMs equally I am willing to ignore that (besides raising questions about where the 50ms were derived from initially).
> 
> 2) min non-network delay of 13 ms is a projected value in https://tools.ietf.org/html/draft-han-iccrg-arvr-transport-problem-01, which has not been demonstrated to be actually achievable, the reported current values were 130 to 1118 ms. This immediately makes it obvious that the comparison seems extremely optimistic.
> 
> 3) 99.9th %ile queuing delay: the assumption here seems to be that only the single AQM hop actually controls all variable RTT jitter. And that 99.9 is a relevant number here, which obviously is debatable.
> 
> 4) Left for propagation round trip & equivalent reaches: the 1 ms per 100Km RTT rule of thumb really only works for complete fiber runs, any conversion between photons and electrons on the way, any router and other network equipment will introduce additional variable delay, as will retransmits due to data corruption.
> 
> 
> I would really like to see real world tests, in which RTT and RTT variance are measured and shown for a 3200Km path across the continental U.S. with an L4S AQM at the bottleneck, and I would like to see the full path RTT and jitter over time as well as as CDF, as for the supposed application it does not really matter where along the path the jitter is acquired.
> More hard data, less overly optimistic back of the envelope estimates, please.
> 
> 
> Best Regards
> 	Sebastian
> 
> 
> 
> *) I like the latency budget approach, we can haggle about the exact number, but the idea generally seems sane and sound, and both numbers used often 100 and here 50ms are too round to be realistic results of empiric experiments, but rather indicate some aggregation of empiric results with a modest amount of rounding ot result in a nice and clean number.
> I also note that the cited https://tools.ietf.org/html/draft-han-iccrg-arvr-transport-problem-01#page-12 actually claims:
> "AR/VR developers generally agree that MTP latency becomes imperceptible below about 20 ms [Carmack13].  However, some research
>    has concluded that MTP latency MUST be less than 17ms for sensitive users [MTP-Latency-NASA]." The original Carmack13 website is gone and even the wayback machine comes up empty, but given the URL:
> [Carmack13]
>               Carmack, J., "Latency Mitigation Strategies", February
>               2013, <https://www.twentymilliseconds.com/post/latency-mitigation-strategies/>
> 
> www.twentymilliseconds.com, I really wonder where the 50ms budget was derived from.
>