Re: [tsvwg] draft minutes from interim

[Jonathan Morton <[e-mail]>]

> On 6 Mar, 2020, at 7:27 pm, Bob Briscoe <[e-mail]> wrote:
> 
> During Jonathan's slides, there was a conversation that seems to be completely missing where Greg asked for SCE scenarios to not just show experiments as selected individual time series that could be "cherry picked", but to usually give results over a range of realistic link rates and RTTs and a range of traffic scenarios. Also there was a request to present summary "figures of merit" e.g. queue delay, utilization, rate ratio, packet drop, ECN marking, normalized flow completion time, etc. Also to show "whiskers", e.g. 99-%ile as well as mean, across a number of experiment runs.

I don't think the conversation was *completely* missed in the minutes:

> - Greg White asked if data is anecdotal cases and time series, rather than wider range, CDFs.

At the time, I remember pointing out that for a half-hour talk, we of course had to be selective, but that we could discuss specific tests and presentation formats offline.  Our code is also available for you to perform your own tests and reproduce ours.

The reason we prefer to present time-series charts is because they show the evolution behaviour of the congestion control in a way that makes behavioural anomalies obvious.  In particular, this was the best way to show the robustness of SCE to a shift from an SCE-aware bottleneck to an otherwise identical RFC-3168 bottleneck and back, a test which had been specifically requested by one of the Chairs; we have not yet seen a demonstration of an equivalent scenario in L4S.  We do this to project confidence that SCE works in a wide variety of circumstances, not just the ones actually presented.

Our choices of scenario to present were in part directed towards cases which are considered "difficult", either traditionally with conventional congestion, or through experience with high-fidelity congestion in particular.  We wanted to show that we could handle these "difficult" cases cleanly.  At least one of those cases was one that we have previously had difficulty with, and had been asked to find ways to improve on - so we did, and showed the improvement.

We are now in a process of generating more comprehensive results and with additional forms of data presentation.  These should be ready to show at Vancouver, though it looks increasingly likely that we will have to participate remotely this time.  And of course we will specifically address the question over ECT(1) proposed by the Chairs.

 - Jonathan Morton