Re: [Covidimpacts-workshop] Forward Error Correction Qs

[Cullen Jennings <[e-mail]>]

> On Nov 13, 2020, at 4:25 AM, Mirja Kuehlewind <[e-mail]> wrote:
> 
> So you say the reaction to too few bandwidth was sending more data...?
> 
> Is the FEC fixed or is that somehow adopted dynamically?
> 

No, it’s not the case where as the packet loss goes up, the endpoints start sending more data. (that gets proposed on a regular basis but someone eventually points out why that is not such a hot idea)

At a high level the conceptual works along theses lines: 

1. The congestion control algorithm, or something else, provides an estimate of the bandwidth 

2. A rate controller decides what percentage of the bandwidth to allocate to each category of data that need to be sent: (audio, video, audio FEC, video FEC, feedback metrics, control). 

3. Given the bandwidth available for video, the rate controller will pick the resolution / frame rate to use for video 

4. Both opus and H.264 can dramatically adjust the bitrate they are sending at to match what the rate controller gives them as a target. 

That is a very gross over simplification of what any of the major systems do. What they actually do is a much more complicated but fits into that overall model. This area has a ton of IPR as well as trade secrets so it is not something that going to get discussed much on an open list. If you want to dig into details, I’m happy to do that outside an IETF context. The open source wbeRTC media engine is also a great example of what this looks like for people that want to dig deeper. 

One of the things that complicates the above is you have control loops running at very different times lines. The control loop changing the bitrate from the video decoder can not react to changing bandwidth as fast as the congestion control typically reacts. There are some great publications on how to deal with this at the USPTO. 

I also want to be a bit more nuanced about the quality of bandwidth. People tend to focus on how many bits per second users can get. Sure that matters but it is only one factor. If I had a 1.5 mbps connection that once a minute dropped all my packets for 1 second, it would be nearly impossible to have a good video call across that short of AI that predicts what I will say next. If I had the same connection and it lost twice as many packets as the first, but the losses were randomly distributed, that would deliver an awesome video conference without doing anything clever. Obviously jitter and latency are also important. The fact that we buy internet connections largely differentiated only by bandwidth in the advertising might be part of the focus on this.  I look forward 

The general quality of the internet that webex calls were running over was worse in March 2020 than it was in January 2020. VoIP apps have always had to deal with come users were on connections that were fast and some where slow - that was not new with covid-19. However the changes in other quality aspects of the internet were part of what caused significant shifts in how that bandwidth was allocated in step 2.