Re: [rmcat] NADA Implementation in Mozilla

[Sergio Mena <[e-mail]>]

Hi Sergio,

Thanks a lot for your interest.

It’s a great idea to be able to compare common statistics across 
different bandwidth estimation algorithms.  In fact, in our code we had 
to modify the logging process in the default algorithm to support that, 
but the plotting part is less sophisticated than what you've been 
sharing :-)

Would you be willing to hook (and modify as needed) the logging 
mechanism in our Firefox branch to your repo? We can explain where all 
logs are located.

In one of our algorithm's modes -- NADA-OWD -- we are indeed still using 
transport-wide cc feedback. We had also implemented the new CCFB format 
(in past hackathons) in another branch, but we never came round to 
merging both branches.

About how we carried out the testing scenarios, they were mostly manual, 
i.e., having actual calls between two of us, then collecting and 
processing the logs from the sender. For more details on this, please 
check our update at IETF-106 
(https://www.ietf.org/proceedings/106/slides/slides-106-rmcat-nada-update-01). 
If someone could write scripts to automate this, that would be of course 
awesome!

Thanks,

Xiaoqing & Sergio


On 08/12/2020 17:03, Sergio Garcia Murillo wrote:
> Hi Xiaoqing and Sergio,
>
> Great to see you are still working on it.
>
> I proposed a while back the use of a common logging format for being 
> able to create tools for measuring the performance and comparing the 
> behavior of the different bwe algorithms:
>
> https://github.com/medooze/bwe-stats-viewer
>
> Would you consider adding support to it on your source code? If you 
> are still using the transport wide cc feedback messages the 
> information dumped would be quite straight forward.
>
> Also, it would be interesting knowing how did you perform the testing 
> scenarios and if we could create a set of scripts to replicate them 
> easily, what do you think?
>
> Best regards
> Sergio
>
> On 08/12/2020 16:24, Xiaoqing Zhu (xiaoqzhu) wrote:
>>
>> Hi all,
>>
>> A while back we presented some initial evaluation results of the NADA 
>> implementation in the open-source Mozilla browser under various 
>> real-world settings at IETF-106 
>> <https://www.ietf.org/proceedings/106/slides/slides-106-rmcat-nada-update-01>. 
>> After further tinkering, we are happy to share a cleaned-up version 
>> of our implementation rebased to a recent version of the Mozilla code:
>>
>> https://github.com/sergio-mena/gecko-dev/tree/nada
>>
>> It supports the use of either one-way-delay (OWD) or round-trip-time 
>> (RTT) as the congestion signal for NADA, and includes all algorithmic 
>> features for congestion control as described in the corresponding RFC 
>> <https://tools.ietf.org/html/rfc8698>. Some illustrative results from 
>> running the modified Firefox Nightly browser using the default 
>> algorithm, or OWD/RTT-based NADA can be found here 
>> <https://www.dropbox.com/s/g3idujywj573gax/2020-12-07-nada-eval-in-mozilla.pdf?dl=0>. 
>>
>>
>> For your convenience, this branch also contains a folder. It contains 
>> brief instructions and a utility python script for post-processing 
>> the logsand plotting the graphs (as shown in the link above):
>>
>> https://github.com/sergio-mena/gecko-dev/tree/nada/nada_eval
>>
>> Since this effort has spanned over multiple years, we’ve rebased our 
>> code twice along the way to catch up with major changes in the 
>> Mozilla base code.Those rebases required us to squash our commit 
>> history. So we are also sharing two branches below for archiving the 
>> original commit history:
>>
>> https://github.com/sergio-mena/gecko-dev/tree/nada_rebase_0
>>
>> https://github.com/sergio-mena/gecko-dev/tree/nada_rebase_1
>>
>> Please feel free to check out the above. Let us know if you are 
>> interested inintegrating it in your WebRTC-based app ortesting it out 
>> in the real-world – always happy to jump on a call for that😊
>>
>> Thanks,
>>
>> Sergio and Xiaoqing
>>
>
> -->