Re: [iccrg] Musings on the future of Internet Congestion Control

[Michael Welzl <[e-mail]>]

[ Disclaimer about this and follow-up emails: that’s just me talking here, I didn’t check with my co-authors (and I’m happy for them to chime in if they disagree with something I say!). ]


Dear Bob,

Many thanks for taking the time to write such a long answer!

While I’m glad to see that this paper is causing some controversy (ICCRG has become too boring!), I also don’t think it should agitate you as much as it seems to have done  :)
Let’s be clear about what this paper is saying. First, this is much more about the problem than about a particular solution - we intend to give some hints about possible directions, and yes, these hints contain a certain bias towards on-path support, but the main point of the paper is really that there is a growing problem of underutilization. And, we do believe that this problem isn’t likely to be solvable for all time with the *purely* end-to-end approach that we have to day.   (“purely” is indeed a missing word in the text that you quote below, which argues about a "shift away from the current end-to-end approach” - though this is really how the “current e2e approach” is).

I don’t think our paper should (or easily could?) be read as trying to “forbid” an e2e control loop altogether.  After all, we call “changing the e2e control loop” one of the solutions, and dedicate a whole subsection to it!  We argue primarily against “blind” end-to-end operation, and in this subsection, we say (4 A):  "This points at a need to make an informed choice instead.”   (paced chirping, which you mention below, is indeed a way to obtain more information).

The paper’s final statement is:
“We hope this paper will persuade our peers that it is worth considering these issues, to debate and experiment with alternative designs, and help prepare the Internet to shift towards a future in which unnecessary latency is substantially reduced or removed, and congestion control is no longer routinely implemented end-to-end."

“Routinely” is a key word here. We are currently facing a reality that is as far from “forbidding” and end-to-end loop as it could possibly be - what is “forbidden” is instead the use of proxies, as they have been known to ossify protocols (in case of TCP), and are now prevented from interfering (in case of QUIC).  As a result, people now show that TCP (with proxies) can sometimes work better than QUIC [  https://doi.org/10.1002/sat.1432 <https://doi.org/10.1002/sat.1432> ], which, to me, is a bit of a dilemma. Shouldn't we aim for a future where the end-to-end loop can be better informed, by *knowingly* interfacing with non-transparent proxies that can work with local knowledge?

So, now, to your points below:

> On Jun 16, 2022, at 1:04 PM, Bob Briscoe <[e-mail]> wrote:
> 
> Michael,
> 
> This seems to be a case where a group of researchers has written a paper about wanting to shift its focus to congestion control over path segments (PEPs, proxies, etc), but it's written as if it's an argument for /everyone/ to shift away from the e2e problem (without actually providing strong enough arguments to justify that). I've tried to boil this down to 3 generic points (the last of which is a superset of Christian's point):
> 
> 
> 1) Segmented paths inherently involve a potential rate-mismatch between one segment (seg1) and the next (seg2). If flow over seg1 is /faster/ i.e. ahead of seg2, then yes, a middlebox can be worthwhile, 'cos the middlebox M between them can buffer the difference. But if seg1 is /slower/ i.e. behind seg2, M cannot invent data that hasn't arrived yet. 
> 
> The mmWave case in your paper is an example of the latter (impossible) case: the typical mmWave case would be a v fast but intermittent link close to B (the 'client'). It would be no use getting up to speed fast over the short mmWave segment, unless A (the 'origin server') had premonition so that it had started getting up to speed earlier over the long RTT part of the path, before the client even knew it needed the data. {Note 1}
> 
> Obviously, a cache at M is a form of premonition. But my point is that for true communication of /new/ information between A & B, one cannot get round the fact that B cannot receive /new/ information faster than A sends it, no matter how much magic there is in box M between them. In such cases, information flow has to be e2e. So, if some researchers choose to focus on the simpler cases where paths can be segmented at middleboxes, someone is still going to have to solve the e2e problem, for all the cases when /new/ information generated at A needs to get to B fast. {Note 2} 

This appears to me to be a way of saying: putting a proxy next to an mmWave link doesn’t help when the bottleneck is earlier in the path.  That, of course, is absolutely right - and I agree that we should not *remove* the end-to-end loop for this reason!

I’m pulling up your “note 1” to answer it in context:
> {Note 1} With mmWave, it makes sense for M to be able to store up incoming data if the mmWave blocks for a while, but that's a not the problem of getting up to speed or diminishing feedback, that's the problem of intermittency, and Delay Tolerant Networking as a solution."

DTN is an interesting case to think about in this context; would it work well for the short time scales that one may want to consider here? It’s called “Delay-Tolerant” after all, which probably won’t be a good characterization of "traffic across mmWave” in general.  Also note that fluctuations on mmWave links don’t always entirely eliminate communication. Perhaps this paper is of interest:
DOI: 10.1109/INFOCOMWKSHPS50562.2020.9162881. It’s also available from: http://witestlab.poly.edu/~ffund/pubs/tcp-mmwave.pdf


> 2) This paper seemed to confuse what the comms industry has done (the low hanging fruit) with what needs to be researched (so industry can eat the high hanging fruit as well). Yes, CDNs have deployed solutions in cases where middleboxes /can/ improve performance (e.g. because some data or some programme logic is cachable), or where the last mile is the slow segment (e.g. satellite). And yes, there is research to be done to ensure middleboxes can help in the presence of e2e crypto provided by QUIC, etc. ...
> 
> ...However, if a paper is going to argue for a move away from /research/ on the e2e problem, it needs to prove that middleboxes would be able to solve the /whole/ e2e problem (which requires premonition - see point #1). It is not enough to show that /some/ cases (the low hanging fruit) can be solved using middleboxes. 

Entirely move away from it is not what our paper says. Indeed, you seem to get quite hung up on a missing “purely” or “exclusively” in one sentence (which, to me, was implicit in the word “current”, but I see that this sentence would have benefitted from adding “purely”).


> I would have expected to see such arguments in Pt IV-B on "Network-assisted congestion control", but these were the only sentences I could find that supported this argument:
> "explicit feedback schemes may work well on shorter path segments"
> "This could be achieved by putting the control close to the bottleneck." [having put it there, how does one communicate between A and B, when A or B is not close to the bottleneck?]
> If you want to shift your research focus to segmented paths and middleboxes, please go ahead. But if you're going to tell everyone else to shift their focus as well, you're going to have to come up with a lot more convincing arguments than these. Pt IV-B was far too vague to support the following conclusion:
>     "we argue
>     for improved congestion feedback with a shift away from the
>     current end-to-end approach; this will improve performance"

…. that’s that sentence.


> 3) There are still promising directions to address the e2e diminishing feedback problem of getting up to speed. E.g.
> * Using cached path knowledge, as Christian pointed out;

… and as our paper says, also in section 4A:
"One possibility is to try to learn from past success or failure, and to assume some correlation between a newly starting data transfer and previous transfers."


> * Diminished feedback can be addressed e2e by sending some packets faster than others (eg. by sending chirps) to find the available capacity without increasing the overall average rate {Note 3}. That research direction is extremely relevant to a paper on diminished feedback. 

It’s a potentially noisy, and delayed, signal; sure it can work, but why not get explicit help from on-path devices instead (or in addition)?  E.g. from a box directly adjacent to a fluctuating-capacity mmWave bottleneck.  So I’m not saying that e2e chirping is not worth pursuing, but I do speak in favour of explicit on-path support.


> * Ensuring control signal feedback scales with rate
> 
> Whether or not you had enough space to cite promising research directions that address the e2e problem, their very existence ought to have diluted the message that everyone should shift away from the e2e problem. 
> 
> 
> Bob
> 
> 
> PS. Nits re. the assessment of the Neubot results around Fig 5: 
> 1/ The appropriate metric for growth of the range of rates should be the ratio between P75 & P25, not the difference. If the average grows, it would be unusual for the /difference/ between P25 and P75 not to grow. 

I don’t understand this. You can have an average with a small stddev - just because its absolute value gets larger, the stddev wouldn’t have to. In other words, the boxes in fig. 5 could stay tiny and just move upwards.


> 2/ All the Neubot measurements before 2019 seem highly questionable, given when it started to use CUBIC in 2019 the high end jumped about an order of magnitude.

Their upper end, yes; not their lower end. The main point here is that the lower end hardly increases.


> {Note 2} It is easy to fool oneself into thinking that A can just be moved nearer to M, which is placed near to B. But, ultimately, A and B map to physical points in the real world where information is generated and consumed, and they expect telecommunications to do the job of transmitting information between them. Even if they are virtualized processes that can migrate, the physical location of A is constrained by where (physically) prerequisite info flows are coming from and B is constrained by where (physically) onward information flows are going to.

I agree with this.


> {Note 3} Paced Chirping works well for continuous link types like Ethernet. But, since Joakim & I presented the work on Paced Chirping to the ICCRG, we've found it's really tough to probe many types of bottleneck link for their instantaneous rate, because lots of link types are highly discontinuous on short timescales (e.g. WiFi, DOCSIS, LTE). However, that doesn't mean that this whole direction is a non-starter - it needs more research attention, not less.

I agree with this.

Cheers,
Michael


> 
> 
> Bob
> 
> On 15/06/2022 09:02, Michael Welzl wrote:
>> Dear ICCRGers,
>> 
>> We just got a paper accepted that I wanted to share:
>> Michael Welzl, Peyman Teymoori, Safiqul Islam, David Hutchison, Stein Gjessing: "Future Internet Congestion Control: The Diminishing Feedback Problem", accepted for publication in IEEE Communications Magazine, 2022.
>> 
>> The preprint is available at:
>> https://arxiv.org/abs/2206.06642 <https://arxiv.org/abs/2206.06642>
>> I thought that it could provoke an interesting discussion in this group.
>> 
>> Figures 4 and 5 in this paper show that, across the world, network links do not just become "faster”: the range between the low end and the high end grows too.
>> This, I think, is problematic for a global end-to-end standard - e.g., it means that we cannot simply keep scaling IW along forever (or, if we do, utilization will decline more and more).
>> 
>> So, we ask: what is the way ahead?  Should congestion control really stay end-to-end?
>> 
>> Cheers,
>> Michael
>> 
>> 
>> 
>> _______________________________________________
>> iccrg mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.irtf.org/mailman/listinfo/iccrg <https://www.irtf.org/mailman/listinfo/iccrg>
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/ <http://bobbriscoe.net/>