[CCWG] [draft-ietf-ccwg-rfc5033bis-03] Feedback on draft

[Adithya Abraham Philip <[e-mail]>]

Hi!

Overall I found the proposal extremely detailed, and it covered pretty
much all the things I would consider important for CCA evaluation.
Setting guidelines down is very important given the variety of new
CCAs being proposed and adopted, and I think this proposal will help a
lot in improving the current state of CCA evaluation.

Some thoughts I had about potential improvements to the proposal:
explicitly state which workloads it was tested with.

1. Section 3.2.1 (CCA's to be evaluated against): Should we explicitly
list BBRv1 as one of the CCAs to be evaluated against, given that it
is the second most popular CCA [4] on the Internet today? Also I'm not
aware of how widely deployed DCCP is, and NewReno is also apparently
used by < 1% of Alexa top 20k websites [4]. If DCCP is also not very
widely used, maybe NewReno and DCCP should both be MAY instead of
SHOULD evaluate against?

2. Serction 4: Adding an Internet core/inter-domain link congestion
setting as one of the settings to run evaluations in, where there are
thousands of simultaneous flows and 10s of Gbps bandwidths. This is
based off of past work which has shown that congestion exists on the
core of the Internet [1], and at least two other pieces of work which
show that CCA behavior can change at scale, such as buffer sizing [2],
and BBRv1 fairness/NewReno Mathis model [3] (full disclosure - the
latter work is by my co-authors and me). I would think this fits in
Section 4, but the difficulty in running experiments at such high
bandwidths might force it into Section 5.

3. Section 5: It might be good to specify workload types as one of the
evaluation dimensions. CCAs are typically evaluated with a bulk
transfer workload, but in our experiments we've found that CCA
fairness can be drastically affected by the service using it. For
example, a video streaming service will probably have its network
traffic behavior be a function of both its ABR and the CCA. If the CCA
is specially designed for certain workloads like video streaming, it
MUST be evaluated with that workload, and if it's a general purpose
CCA, it should

4. Flow counts and buffer sizes: It might be good to explicitly add an
evaluation condition where the number of flows and buffer size is
varied. 1 Cubic flow vs 1 BBR flow is pretty much fair in Cubic's
favor, but 10 Cubic flows versus 10 BBR flows result in unfairness to
Cubic or BBR depending on the buffer size [5].

5. Would it be possible to define terms like starvation more
quantitatively? But I understand if these are deliberately left
ambiguous since there might not be consensus on what is considered
starvation.

Overall I think the current proposal checks most of the boxes for good
CCA evaluation! Some things I think the proposal does very well are:
1. Covering a wide variety of evaluation settings, but dividing it
into must-haves (section 4) and good-to-have's (section 5). It helps
prevent the requirements from seeming so laborious to implement that
it's considered impractical.

2. The inclusion of testing on a variety of paths such as wired and
wireless, the latter of which is often neglected, and the inclusion of
RTP evaluation, which is timely given how popular real time
communication services are today.

I hope these comments were helpful!

Regards,
Philip

[1] Inferring persistent interdomain congestion
https://dl.acm.org/doi/10.1145/3230543.3230549
[2] Buffer sizing for congested Internet links
https://ieeexplore.ieee.org/abstract/document/1498335
[3]
Revisiting TCP congestion control throughput models & fairness
properties at scale
https://dl.acm.org/doi/abs/10.1145/3487552.3487834
[4] The Great Internet TCP Congestion Control Census
https://rajkiranjoshi.github.io/files/papers/sigmetrics20-gordon.pdf
[5]
Are we heading towards a BBR-dominant internet?
https://dl.acm.org/doi/10.1145/3517745.3561429