[Idr] Re: I-D Action: draft-wang-idr-dpf-00.txt

[Kevin Wang <[e-mail]>]

Hi Robert,

> So do you agree that your solution will not work in a single plane with more than 3 stage CLOS topology ?

I used multi-plane 5 Clos as an example because it is dominantly popular in today’s hyperscaler design. Coloring should also work for a single plane 5 Clos. Note a single super spine could participate in multiple planes. The figure below shows one example of how to color a single plane 5 Clos. Again, please just take this as yet another example. DPF provides the machinery to color the network. We have seen customers coloring their networks using innovative approaches we have not imagined.


[image.png]

Thanks,
Kevin




Juniper Business Use Only

From: Robert Raszuk <[e-mail]>
Date: Thursday, March 26, 2026 at 1:49 PM
To: Wang, Kevin <[e-mail]>
Cc: idr@ietf. org <[e-mail]>
Subject: [Idr] Re: I-D Action: draft-wang-idr-dpf-00.txt

[External Email. Be cautious of content]

Hi,

> Due to the lack of entropy and granularity of the AI/ML flows

AI/ML flows use different network technologies in the AI clusters ...


> [Kevin] Many of today’s 5 stage fabric uses the multi-plane design

So do you agree that your solution will not work in a single plane with more than 3 stage CLOS topology ?

Thx,
Robert



On Thu, Mar 26, 2026 at 6:42 PM Wang, Kevin <[e-mail]<mailto:[e-mail]>> wrote:
Hi Robert,

Thanks for the comments. Please see my response inline...

From: Robert Raszuk <[e-mail]<mailto:[e-mail]>>
Date: Tuesday, March 17, 2026 at 2:53 AM
To: Wang, Kevin <[e-mail]<mailto:[e-mail]>>
Cc: idr@ietf. org <[e-mail]<mailto:[e-mail]>>
Subject: [Idr] Re: I-D Action: draft-wang-idr-dpf-00.txt

Hi Kevin,

Observation and question after yr presentation:

Observation:

Let's consider this slide:
[chrome_fECgMX5SKO.png]

Net result of this extension is reduced to half fabric capacity to reach each of the prefix. Sure you can fix it when adding more spines to the slide, but when you are constructing DC fabric you do need to account for it ahead of time.

So please reiterate what are we gaining here if there is no oversubscription between leaf to spine for vanilla CLOS fabric ?

Of course I understand that you can map a prefix to all colors (just like today), but then we are back to the colorless control plane.

Based on the fundamental property of IP traffic and the benefit of statistical multiplexing any partitioning of the forwarding plane is IMO not a good thing - even if it looks very colorful on the slides.

[Kevin] “no oversubscription” can only guarantee that there is no congestion if traffic is evenly distributed to all spines. Due to the lack of entropy and granularity of the AI/ML flows, random hashing usually can’t guarantee even traffic distribution across all spines in a 3-Clos topology. This has been observed in both the real networks and labs.


Question:

Your picture in 3 stage CLOS looks like this:

[image.png]

How will it look when there are 5 stages of DC fabric ? Do you still foresee this still being manageable in any form of shape when there are more than 3 stages ?

[Kevin] Many of today’s 5 stage fabric uses the multi-plane design at the super-spine level. A typical design could have super-spine1 talks to the 1st spine in each pod, super-spine2 talks to the 2nd spine in each pod, …. Figure 3 of RFC-7938 shows an example. This kind of multi-plane design was intended to allow the 5 stage Clos to scale up to 100k+ GPUs. It also naturally aligns with the multi-color design above (e.g. Spine1 is colored as red in each pod. Super-spine1 would also be colored as red).

Thx,
Robert