Re: [tsvwg] Real time low latency video (SCReAM) and L4S

[Ingemar Johansson S <[e-mail]>]

Hi

Yes, I can imagine that the rate should change as you say below. Currently though I am out of CPU cycles so I cannot do more experimentation 😊

 

/I

 

From: Greg White <[e-mail]> 
Sent: den 10 december 2020 21:32
To: Ingemar Johansson S <[e-mail]>; Sebastian Moeller <[e-mail]>
Cc: [e-mail]
Subject: Re: [tsvwg] Real time low latency video (SCReAM) and L4S

 

Ingemar,

 

Thanks for sharing the results of these experiments.

 

In the fq_* case, the available bandwidth for the SCReAM flow would go in steps more like 40Mbps, 20Mbps, 13Mbps, 10Mbps as the flow count changes.  Not that I’m asking you to run new experiments, but it would be interesting to see the behavior in that scenario.  

 

-Greg

 

 

From: tsvwg <[e-mail] <mailto:[e-mail]> > on behalf of Ingemar Johansson S <ingemar.s.johansson=[e-mail] <mailto:ingemar.s.johansson=[e-mail]> >
Date: Thursday, December 10, 2020 at 3:51 AM
To: Sebastian Moeller <[e-mail] <mailto:[e-mail]> >
Cc: "[e-mail] <mailto:[e-mail]> " <[e-mail] <mailto:[e-mail]> >
Subject: Re: [tsvwg] Real time low latency video (SCReAM) and L4S

 

Hi

Please see inline

 

/I

 

From: Sebastian Moeller <[e-mail] <mailto:[e-mail]> > 
Sent: den 10 december 2020 10:56
To: Ingemar Johansson S <[e-mail] <mailto:[e-mail]> >
Cc: [e-mail] <mailto:[e-mail]> 
Subject: Re: [tsvwg] Real time low latency video (SCReAM) and L4S

 

Hi Ingemar,

 

one clarification, pure CoDel is not the state of the art AQM that L4S competes against, at the very least it is fq_codel (or cake), which for multiple concurrent flows is a big difference (in your single flow tests mentioned below that does not matter, but for real links it makes a ton of difference, also doing these tests without cross-traffic is not too realistic, ist it?). I thought that was obvious, but I should be more explicit in the future. See my anecdotal delay data from cake inn the other thread, with 3 concurrent multi-flow loads.

[IJ] Sure, multiple flows have their merit but within each queue you still have CoDel properties show below.






On Dec 10, 2020, at 10:18, Ingemar Johansson S <ingemar.s.johansson=[e-mail] <mailto:ingemar.s.johansson=[e-mail]> > wrote:

Hi

I write up one post to address a few of the arguments for/against L4S seen from a SCReAM perspective and from the general point of streaming of low latency video.
I don't expect that the opponents of L4S will become convinced but I hope that it will help others to get some perspective around the matters brought up below. There is a certain risk that this can end up in yet another lengthy thread. I will personally eject from this quite quickly to spare the rest of the audience. 

1) "SCReAM, last I checked, has no code to implement the detection heuristic." (for RFC3168 bottlenecks) : 
SCReAM as well as other rate control algorithms need to have a fall back to delay based congestion control, to at least get a decent behavior for the cases where L4S is not implemented. In addition  a video source has a variable frame size output, not just  because of occasional I frames, P frames also vary in size. This means that realtime low latency applications that strive for a low e2e delay will leave some air for other flows. It is not a given that they will behave like SCReAM but regardless of implementation they all have a rate limited source with a rate that varies a lot over short time scales.
With SCReAM I currently see the problem that it is a bit too reactive to congestion and that it needs some improvement and an RFC3168 detection algorithm is perhaps needed then. But currently, SCReAM is a pretty weak competitor against flows with infinite source bitrates. 
But.. instead of just repeating this argument over and over again. Get the SCReAM BW test application from https://github.com/EricssonResearch/scream <https://protect2.fireeye.com/v1/url?k=ff9920f4-a0021995-ff99606f-86d8a30ca42b-c64bc8a87233ed38&q=1&e=2c5a2521-8d71-4608-80c6-4d726601a40b&u=https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream>  and run it with the following additional arguments 
-ect 1 -rand 20 
The code is public, it is just to fire away in test beds.

2) L4S gives very limited gain over CoDel (5ms is only a little more than 1ms). 
I will address this argument as well as the argument that the argument that the congestion control algos chartered in the RMCAT WG already themselves can keep delay low. The simulated test case is SCReAM running with a test trace from NVENC, the video frames are scaled to match the target rate, in the first experiment I assume that the video encoder is very responsive to changes in target bitrate, in the second experiment a bit sluggish behavior is mimicked that makes the video coder react 200ms late, this is a common case in many video encoders (e.g NVIVIA Jetson Nano and Xavier NX and RPI 3). The bottleneck is a 20ms RTT and a bandwidth that varies between 40 and 15Mbps in steps. The max video bitrate is set to 30Mbps.

*SCReAM_no_ECN_v_0.0.png : Shows the performance with no ECN support at all. The video frame delay illustrates the delay the end user will experience, the larger the variations, the larger delay is needed in a dejitterbuffer is needed to avoid a choppy play out.
*SCReAM_ECT0_CoDel_5ms_100ms_v_0.0.png : Shows the performance with  CoDel ECN marking.  Slightly better than no ECN but still up to 150ms frame delay spikes
*SCReAM_ECT1_L4S_v_0.0.png : Now with L4S. The delay is greatly reduced. What is left is the frame delay due to the serialization delay of the larger video frames. The nominal bitrate is slightly lower than the other two cases above in the congested area (when the bottleneck BW is <= 30Mbps), this is a natural tradeoff between latency and throughput.

Now we complicate things a bit for the experiment when the video coder is a bit more sluggish in its behavior. Now it takes 200ms for the video coder to respond to a changed target bitrate.

 

              Well, now try again with your period of rate changes similar to the delay instead of multiple seconds...

 





* SCReAM_no_ECN_v_0.2.png : As expected one can see that the video frame delays increase yet some more.
* SCReAM_ECT0_CoDel_5ms_100ms_v_0.2.png : Roughly the same performance with some tendency towards a higher frame delay.

 

              [SM] Well, over a 20ms RTT link CoDel's default values are not perfect, here 2ms_40ms might be a better match, but you still have the non high frequency ECH response to deal with. Before you interpret that as an indication of L4S' inherent superiority, please show the respective behavior over a link with 100ms base RTT. This cuts both ways.... 

[IJ] 100ms base RTT is beginning to get into the border of the useful for an online cloud gaming session. But fair enough, tried this as well. CoDel does still give up to 100ms extra video frame delay while in the L4S case it is still is limited to the serialization delay. 

One observations that as you contemplate that CoDel AQMs can be configured, my interpretation is that it is possible to also update these for L4S

 





* SCReAM_ECT1_L4S_v_0.2.png : Video frame delay is still very low, despite this added extra video coding artifact.

 

              [SM] If you look at the data you realize that the latency spikes are missing for one reason only, the headroom that SCReAM over L4S leaves is accidentally? large enough that the rate decrease steps actually still leave enough room for the SCReAM flows ot continue at their initial rate. Looking at that graph troubles will start, if rate steps will be closer in time than 200ms or of the rate steps are large enough to actually eat up all the rate under-shoot that SCReAM over an L4S bottleneck generates... "Blaming" that on L4S is a bit of  a stretch, no?

              To illustrate my point, at around second 50 the undershoot is not sufficient (plot of video rate in blue and link capacity in red touch) and lo and behold we see a prominent spike in the delay plot. Are you sure that your hypotheses of why L4S leads to superior performance in this test is actually HFCC and not simply the fact that it under estimates the capacity and hence simply rarely fills the bottleneck buffers?

              If we look at the temporal overshoot, all plots show a similar temporal response profile, it is just due to the L4S cases lower utilization that overshoot does not translate into latency spikes. 

[IJ] Sure, is that a problem?. It is a side effect of the dense L4S marking that SCReAM leaves extra headroom. It manages to sense the bottleneck better.

              In short I do not think the presented data is sufficiently broad to allow extrapolation to the generic case. It is bit like the fas-track-to-close-CDN examples that have been presented numerous times in the past. I do not doubt that under the tested condition L4S gives the best low latency performance, but I do not think your argument why L4S performs better actually is driven by the actual data.

[IJ] Still, this case punches a hole in your argumentation , right ? 

And I don’t see that the CDN examples are so controversial. Despite the fact that I live up in the cold north I have 12ms to most of the content that I try to access.  Cloud rendered video for online gaming is for natural reasons quite unlikely to run across transatlantic links. Sure, players can be in different parts of the world but the cloud rendering (== high bitrate video) is likely to be close to the end user. 






To conclude. 
+ L4S improves performance a lot.
+ Congestion controls that only rely on delay and packet loss measurements does not give a very low delay.

 

              [SM] These are decent arguments for HFCC in general and not for L4S specifically, no? Assuming that your tests do not simply demonstrate that paced video streaming over variable rate links should always leave enough headroom for the to be expected rate steps....

 






I hope that this should cast some light on the question whether L4S is useful or not.

 

              [SM] It does not answer the question though whether L4S is the best or even a reasonable implementation of HFCC. 

 

Again, I am not sure that your conclusion is the only or even the simplest reason, why L4S-enables SCReAM seems to perform better in these tests, with the added question of what end users would prefer, higher quality bitrates with the occasional glitch or smoother but lower quality video.

[IJ] The rationale is that it is better with low latency and smooth streaming, choppy video is quite noticeable in the long run. Surely one can argue against this. 

 

Best Regards

              Sebastian

 

 






/Ingemar
================================
Ingemar Johansson  M.Sc. 
Master Researcher

Ericsson Research
RESEARCHER
GFTL ER NAP NCM Netw Proto & E2E Perf
Labratoriegränd 11
977 53, Luleå, Sweden
Phone [phone]
SMS/MMS [phone]
[e-mail] <mailto:[e-mail]> 
www.ericsson.com <http://www.ericsson.com> 

Talk about a dream, try to make it real
                 Bruce Springsteen
=================================