Re: [iccrg] Musings on the future of Internet Congestion Control

[David Hayes <[e-mail]>]

Hi Michael,

I think it is great that you and the other authors make these 
points and put them out there for discussion. And I'm really happy 
to see them tested and disputed on the list. With the Internet 
increasingly more heterogeneous, I think it is important our 
solutions also adapt accordingly. They of course have been, but I 
think you are pointing to a better way forward.

Perhaps putting it more generally is helpful. I think of it this 
way:

Once upon a time end-to-end communication across the Internet 
could have been considered as communication over links and routers 
that were roughly statistically similar in terms of their 
delay/loss/consistency/signalling/etc. However, end-to-end 
communication across today's Internet may involve communication 
over various "domains", each "domain" with possibly vastly 
different characteristics. Tuning an end-to-end congestion control 
to be efficient for such communication is hard (maybe 
unachievable). Tuning an end-to-end congestion control for 
efficient communication within a particular "domain" is 
achievable.

So to me, if each domain could be considered as block that has its 
own optimised intra-domain congestion control, then end-to-end 
control could be considered as managing the combination of domain 
blocks along the end-to-end path.

Research on intra-domain congestion control then is similar to our 
current congestion control research; with further work needed to 
design and optimise it for the vastly different existing and 
future domains. There is then also a need for end-to-end control 
research, where the object is to manage the combination of domain 
blocks along the path (as opposed to managing congestion of links 
and routers along the path)--of course each end-to-end connection 
may result in a different combination of blocks, compounding the 
problem.

Regards,

David


PS Of course, in the scenario where the end-to-end path only 
transits a single "domain", there is only intra-domain 
communication, and end-to-end control could look like a nicely 
tuned version of one of our current mechanisms.



On Fri, Jun 17 2022 at 09:38, Michael Welzl <[e-mail]> 
wrote:

> [ Disclaimer about this and follow-up emails: that’s just me 
> talking here, I didn’t check with my co-authors (and I’m happy 
> for them to chime in if they disagree
> with something I say!). ]
>
> Dear Bob,
>
> Many thanks for taking the time to write such a long answer!
>
> While I’m glad to see that this paper is causing some 
> controversy (ICCRG has become too boring!), I also don’t think 
> it should agitate you as much as it seems to
> have done  :)
> Let’s be clear about what this paper is saying. First, this is 
> much more about the problem than about a particular solution - 
> we intend to give some hints about
> possible directions, and yes, these hints contain a certain bias 
> towards on-path support, but the main point of the paper is 
> really that there is a growing
> problem of underutilization. And, we do believe that this 
> problem isn’t likely to be solvable for all time with the 
> *purely* end-to-end approach that we have to
> day.   (“purely” is indeed a missing word in the text that you 
> quote below, which argues about a "shift away from the current 
> end-to-end approach” - though
> this is really how the “current e2e approach” is).
>
> I don’t think our paper should (or easily could?) be read as 
> trying to “forbid” an e2e control loop altogether.  After all, 
> we call “changing the e2e control loop”
> one of the solutions, and dedicate a whole subsection to it!  We 
> argue primarily against “blind” end-to-end operation, and in 
> this subsection, we say (4 A):  "This
> points at a need to make an informed choice instead.”   (paced 
> chirping, which you mention below, is indeed a way to obtain 
> more information).
>
> The paper’s final statement is:
> “We hope this paper will persuade our peers that it is worth 
> considering these issues, to debate and experiment with 
> alternative designs, and help prepare the
> Internet to shift towards a future in which unnecessary latency 
> is substantially reduced or removed, and congestion control is 
> no longer routinely implemented
> end-to-end."
>
> “Routinely” is a key word here. We are currently facing a 
> reality that is as far from “forbidding” and end-to-end loop as 
> it could possibly be - what is “forbidden”
> is instead the use of proxies, as they have been known to ossify 
> protocols (in case of TCP), and are now prevented from 
> interfering (in case of QUIC).  As a
> result, people now show that TCP (with proxies) can sometimes 
> work better than QUIC [  https://doi.org/10.1002/sat.1432 ], 
> which, to me, is a bit of a dilemma.
> Shouldn't we aim for a future where the end-to-end loop can be 
> better informed, by *knowingly* interfacing with non-transparent 
> proxies that can work with
> local knowledge?
>
> So, now, to your points below:
>
>  On Jun 16, 2022, at 1:04 PM, Bob Briscoe 
>  <[e-mail]> wrote:
>
>  Michael,
>
>  This seems to be a case where a group of researchers has 
>  written a paper about wanting to shift its focus to congestion 
>  control over path segments
>  (PEPs, proxies, etc), but it's written as if it's an argument 
>  for /everyone/ to shift away from the e2e problem (without 
>  actually providing strong enough
>  arguments to justify that). I've tried to boil this down to 3 
>  generic points (the last of which is a superset of Christian's 
>  point):
>
>  1) Segmented paths inherently involve a potential rate-mismatch 
>  between one segment (seg1) and the next (seg2). If flow over 
>  seg1 is /faster/ i.e. ahead of
>  seg2, then yes, a middlebox can be worthwhile, 'cos the 
>  middlebox M between them can buffer the difference. But if seg1 
>  is /slower/ i.e. behind seg2, M
>  cannot invent data that hasn't arrived yet. 
>
>  The mmWave case in your paper is an example of the latter 
>  (impossible) case: the typical mmWave case would be a v fast 
>  but intermittent link close to B
>  (the 'client'). It would be no use getting up to speed fast 
>  over the short mmWave segment, unless A (the 'origin server') 
>  had premonition so that it had
>  started getting up to speed earlier over the long RTT part of 
>  the path, before the client even knew it needed the data. {Note 
>  1}
>
>  Obviously, a cache at M is a form of premonition. But my point 
>  is that for true communication of /new/ information between A & 
>  B, one cannot get round
>  the fact that B cannot receive /new/ information faster than A 
>  sends it, no matter how much magic there is in box M between 
>  them. In such cases,
>  information flow has to be e2e. So, if some researchers choose 
>  to focus on the simpler cases where paths can be segmented at 
>  middleboxes, someone is
>  still going to have to solve the e2e problem, for all the cases 
>  when /new/ information generated at A needs to get to B 
>  fast. {Note 2} 
>
> This appears to me to be a way of saying: putting a proxy next 
> to an mmWave link doesn’t help when the bottleneck is earlier in 
> the path.  That, of course, is
> absolutely right - and I agree that we should not *remove* the 
> end-to-end loop for this reason!
>
> I’m pulling up your “note 1” to answer it in context:
>> {Note 1} With mmWave, it makes sense for M to be able to store 
>> up incoming data if the mmWave blocks for a while, but that's a 
>> not the problem of getting
> up to speed or diminishing feedback, that's the problem of 
> intermittency, and Delay Tolerant Networking as a solution."
>
> DTN is an interesting case to think about in this context; would 
> it work well for the short time scales that one may want to 
> consider here? It’s called
> “Delay-Tolerant” after all, which probably won’t be a good 
> characterization of "traffic across mmWave” in general.  Also 
> note that fluctuations on mmWave links
> don’t always entirely eliminate communication. Perhaps this 
> paper is of interest:
> DOI: 10.1109/INFOCOMWKSHPS50562.2020.9162881. It’s also 
> available from: 
> http://witestlab.poly.edu/~ffund/pubs/tcp-mmwave.pdf
>
>  2) This paper seemed to confuse what the comms industry has 
>  done (the low hanging fruit) with what needs to be researched 
>  (so industry can eat the high
>  hanging fruit as well). Yes, CDNs have deployed solutions in 
>  cases where middleboxes /can/ improve performance (e.g. because 
>  some data or some
>  programme logic is cachable), or where the last mile is the 
>  slow segment (e.g. satellite). And yes, there is research to be 
>  done to ensure middleboxes can
>  help in the presence of e2e crypto provided by QUIC, etc. ...
>
>  ...However, if a paper is going to argue for a move away from 
>  /research/ on the e2e problem, it needs to prove that 
>  middleboxes would be able to solve
>  the /whole/ e2e problem (which requires premonition - see point 
>  #1). It is not enough to show that /some/ cases (the low 
>  hanging fruit) can be solved
>  using middleboxes. 
>
> Entirely move away from it is not what our paper says. Indeed, 
> you seem to get quite hung up on a missing “purely” or 
> “exclusively” in one sentence (which, to
> me, was implicit in the word “current”, but I see that this 
> sentence would have benefitted from adding “purely”).
>
>  I would have expected to see such arguments in Pt IV-B on 
>  "Network-assisted congestion control", but these were the only 
>  sentences I could find that
>  supported this argument:
>
>  * "explicit feedback schemes may work well on shorter path 
>  segments"
>  * "This could be achieved by putting the control close to the 
>  bottleneck." [having put it there, how does one communicate 
>  between A and B, when A or B
>  is not close to the bottleneck?]
>
>  If you want to shift your research focus to segmented paths and 
>  middleboxes, please go ahead. But if you're going to tell 
>  everyone else to shift their
>  focus as well, you're going to have to come up with a lot more 
>  convincing arguments than these. Pt IV-B was far too vague to 
>  support the following
>  conclusion:
>      "we argue
>      for improved congestion feedback with a shift away from the
>      current end-to-end approach; this will improve performance"
>
> …. that’s that sentence.
>
>  3) There are still promising directions to address the e2e 
>  diminishing feedback problem of getting up to speed. E.g.
>  * Using cached path knowledge, as Christian pointed out;
>
> … and as our paper says, also in section 4A:
> "One possibility is to try to learn from past success or 
> failure, and to assume some correlation between a newly starting 
> data transfer and previous transfers."
>
>  * Diminished feedback can be addressed e2e by sending some 
>  packets faster than others (eg. by sending chirps) to find the 
>  available capacity without
>  increasing the overall average rate {Note 3}. That research 
>  direction is extremely relevant to a paper on diminished 
>  feedback. 
>
> It’s a potentially noisy, and delayed, signal; sure it can work, 
> but why not get explicit help from on-path devices instead (or 
> in addition)?  E.g. from a box directly
> adjacent to a fluctuating-capacity mmWave bottleneck.  So I’m 
> not saying that e2e chirping is not worth pursuing, but I do 
> speak in favour of explicit on-path
> support.
>
>  * Ensuring control signal feedback scales with rate
>
>  Whether or not you had enough space to cite promising research 
>  directions that address the e2e problem, their very existence 
>  ought to have diluted the
>  message that everyone should shift away from the e2e problem. 
>
>  Bob
>
>  PS. Nits re. the assessment of the Neubot results around Fig 5: 
>  1/ The appropriate metric for growth of the range of rates 
>  should be the ratio between P75 & P25, not the difference. If 
>  the average grows, it would be
>  unusual for the /difference/ between P25 and P75 not to grow. 
>
> I don’t understand this. You can have an average with a small 
> stddev - just because its absolute value gets larger, the stddev 
> wouldn’t have to. In other words,
> the boxes in fig. 5 could stay tiny and just move upwards.
>
>  2/ All the Neubot measurements before 2019 seem highly 
>  questionable, given when it started to use CUBIC in 2019 the 
>  high end jumped about an order
>  of magnitude.
>
> Their upper end, yes; not their lower end. The main point here 
> is that the lower end hardly increases.
>
>  {Note 2} It is easy to fool oneself into thinking that A can 
>  just be moved nearer to M, which is placed near to B. But, 
>  ultimately, A and B map to physical
>  points in the real world where information is generated and 
>  consumed, and they expect telecommunications to do the job of 
>  transmitting information
>  between them. Even if they are virtualized processes that can 
>  migrate, the physical location of A is constrained by where 
>  (physically) prerequisite info
>  flows are coming from and B is constrained by where 
>  (physically) onward information flows are going to.
>
> I agree with this.
>
>  {Note 3} Paced Chirping works well for continuous link types 
>  like Ethernet. But, since Joakim & I presented the work on 
>  Paced Chirping to the ICCRG, we've
>  found it's really tough to probe many types of bottleneck link 
>  for their instantaneous rate, because lots of link types are 
>  highly discontinuous on short
>  timescales (e.g. WiFi, DOCSIS, LTE). However, that doesn't mean 
>  that this whole direction is a non-starter - it needs more 
>  research attention, not less.
>
> I agree with this.
>
> Cheers,
> Michael
>
>  Bob
>
>  On 15/06/2022 09:02, Michael Welzl wrote:
>
>  Dear ICCRGers,
>
>  We just got a paper accepted that I wanted to share:
>  Michael Welzl, Peyman Teymoori, Safiqul Islam, David Hutchison, 
>  Stein Gjessing: "Future Internet Congestion Control: The 
>  Diminishing Feedback
>  Problem", accepted for publication in IEEE Communications 
>  Magazine, 2022.
>
>  The preprint is available at:
>  https://arxiv.org/abs/2206.06642
>  I thought that it could provoke an interesting discussion in 
>  this group.
>
>  Figures 4 and 5 in this paper show that, across the world, 
>  network links do not just become "faster”: the range between 
>  the low end and the high end
>  grows too.
>  This, I think, is problematic for a global end-to-end standard 
>  - e.g., it means that we cannot simply keep scaling IW along 
>  forever (or, if we do,
>  utilization will decline more and more).
>
>  So, we ask: what is the way ahead?  Should congestion control 
>  really stay end-to-end?
>
>  Cheers,
>  Michael
>
> _______________________________________________
> iccrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/iccrg


-- 
David Hayes
SimulaMet
https://www.simula.no/people/davidh