Re: [Rdma-cc-interest] Congestion Control for Large Scale Data Center Networks

["Roni Even (A)" <[e-mail]>]

Hi David,
I agree that "fairness" is important, at least in DCs we can see ROCEv2 over UDP and TCP traffic. We are also not looking at new congestion control algorithm but more about which ones we can use in the environment and how to implement them. Looking also at HPCC  https://liyuliang001.github.io/publications/hpcc.pdf and RCP http://yuba.stanford.edu/~nanditad/thesis-NanditaD.pdf.  The DC network is not the general Internet, it is a managed network and may also be a single network domain.

As for the work in TSVWG , of course I looked at it and in my view dualq-coupled is orthogonal to the congestion information I mention here.
As for SCE , it does not describe what is the congestion behavior from the sender side and  I also looked at https://tools.ietf.org/id/draft-heist-tsvwg-sce-one-and-two-flow-tests-00.txt and it is clear that for high throughput SCE flows back off in term of competition due to the earlier notification comparing to non SCE flows that will wait for drop or CE marks.

BTW: this is part of the draft I intend to submit

Roni Even


From: Black, David [mailto:[e-mail]]
Sent: Tuesday, September 24, 2019 4:50 PM
To: Roni Even (A); [e-mail]
Subject: RE: Congestion Control for Large Scale Data Center Networks

Hi Roni,

> it will be good if the IETF will be able to recommend an e2e congestion protocol that will allow interoperability between vendors and that will leverage the information from the network .

... and I'd like $10 million in small unmarked bills, please :-).

Seriously, IMHO, the goal for the network ought to be robustness to the variety of current congestion control protocols, not all of which coexist well in the same switch queue.   In other words, I think the better goal to pursue is "to provide fairness between different applications that may use different congestion algorithms" although I would suggest avoiding the word "fairness" in this context as it's typically used to mean "TCP fairness" in this context, whereas something more general is wanted.   In contrast, pursuit of a single e2e congestion protocol that will be universally adopted appears unreasonable and unachievable at this juncture.

There is important work going on in the TSVWG WG that will improve that congestion control protocol coexistence and help achieve low latency (which is not possible in all cases for traffic mix reasons that I won't explain here for brevity).

I suggest reviewing the TSVWG drafts on L4S and SCE, as well as the presentation materials from the Montreal TSVWG meetings.  Here are some pointers:

  *   L4S drafts (@ https://datatracker.ietf.org/wg/tsvwg/documents/): draft-ietf-tsvwg-{l4s-arch,aqm-dualq-coupled,ecn-l4s-id}
  *   SCE drafts: Start with draft-morton-tsvwg-sce - the other two draft-morton-* drafts are somewhat related, but the sce draft is the core draft.
  *   TSVWG presentation material from Montreal: https://datatracker.ietf.org/meeting/105/session/tsvwg
We (TSVWG WG) would greatly appreciate some more data center and RDMA "eyes" on this material, as it is highly relevant to data centers.

Thanks, --David (TSVWG WG co-chair)

From: Rdma-cc-interest <[e-mail]<mailto:[e-mail]>> On Behalf Of Roni Even (A)
Sent: Tuesday, September 24, 2019 3:08 AM
To: [e-mail]<mailto:[e-mail]>
Subject: [Rdma-cc-interest] Congestion Control for Large Scale Data Center Networks


[EXTERNAL EMAIL]

Hi,
We had side meetings at the last two IETF meeting about better congestion control for data centers with good number of participants.

The high throughput of the Data center networks require a good congestion control for data centers should provide low latency, fast convergence and high link utilization.  Since multiple  applications with different requirements may run on the DC network it  is important to provide fairness between different applications that may use different congestion algorithms.  An important issue from the  user perspective is to achieve short Flow Completion Time (FCT).

It is clear that we are not going to make changes to ROCE but we still would like to look at good e2e congestion control. Currently there are multiple published work presented in different papers (examples are DCQCN, HPCC, RPC) but it will be good if the IETF will be able to recommend an e2e congestion protocol that will allow interoperability between vendors and that will leverage the information from the network .

We would like to have a side meeting in Singapore and intend to submit a draft that will discussed the different options hoping to be able to collaborate between the interested people to work on a common direction.

Regards
Roni Even