[Idr] Re: New Version Notification for draft-xu-idr-fare-01.txt

[Jeff Tantsura <[e-mail]>]

Hi,

I’m quite surprised not to see any references to draft-ietf-bess-ebgp-dmz, that has enjoyed wide deployment in hyperscale networks and AI clusters, and is available in at least 5 implementations.
I believe, it is solving all the shortcomings of the initial link-bandwidth draft. Transitivity of the community is left to the implementation and SHOULD be configurable, since the community has to be regenerated at each hop (to provide the cumulative behavior) either would work, the only requirement is that the receiver should not make assumptions about transitivity and accept either - by now the behavior of all major BGP implementations.

We have been discussing normatively making the community transitive, either in the original draft or in draft-ietf-bess-ebgp-dmz (and changing the intended status), this should not affect existing behavior and can be changed as per implementation if needed.

Editorial:
Adaptive routing is a misleading and incorrect term to use - it is called W-ECMP for a reason, there’s no changes to the routing, only forwarding weight rebalancing.
Introduction section is inaccurate and makes assumptions wrt rail domain size, traffic patters and some other artifacts of an AI cluster.

Distributing congestion information and/or BW available (total BW - used) in BGP is an extremely bad (and given the characteristics of the feedback loop) rather harmful idea.

At this point in time I’m not going to go into specifics of the proposal.

I’m against the progress of this document, and don't see a need for yet another solution that also introduces non backwards compatible changes to a widely deployed behavior.

Thanks,
Jeff

> On Jul 23, 2024, at 05:21, Tiger Xu <[e-mail]> wrote:
> 
> 
> Hi all,
> 
> [I-D.ietf-idr-link-bandwidth] has specified a way to perform weighted ECMP based on link bandwidths conveyed in the non-transitive link bandwith extended community.  However, it is impractical to enable adaptive routing in a 5-stage CLOS network where eBGP is used as the underlay routing protocol by directly using the non-transitive link bandwidth extended community due to the following constraints as mentioned in [I-D.ietf-idr-link-bandwidth].
> 
>    "No more than one link bandwidth extended community SHALL be attached
>    to a route.  Additionally, if a route is received with link bandwidth
>    extended community and the BGP speaker sets itself as next-hop while
>    announcing that route to other peers, the link bandwidth extended
>    community should be removed.  The extended community is optional non-
>    transitive."
> 
> Hence, this document defines a new extended community referred to as Path Bandwidth Extended Community and describes how to use this newly defined path bandwidth extended community to achieve adaptive routing in a 5-stage CLOS network.
> 
> Any comments or suggestions are welcome.
> 
> Best regards,
> Xiaohu
> 
> 发件人: [e-mail] <mailto:[e-mail]> <[e-mail] <mailto:[e-mail]>>
> 日期: 星期二, 2024年7月23日 20:10
> 收件人: Hang Wu <[e-mail] <mailto:[e-mail]>>, Hongyi Huang <[e-mail] <mailto:[e-mail]>>, Junjie Wang <[e-mail] <mailto:[e-mail]>>, Peilong Wang <[e-mail] <mailto:[e-mail]>>, Qingliang Zhang <[e-mail] <mailto:[e-mail]>>, Shraddha Hegde <[e-mail] <mailto:[e-mail]>>, Tiezheng <[e-mail] <mailto:[e-mail]>>, Xiaohu Xu <[e-mail] <mailto:[e-mail]>>, Yadong Liu <[e-mail] <mailto:[e-mail]>>, Yinben Xia <[e-mail] <mailto:[e-mail]>>, Zongying He <[e-mail] <mailto:[e-mail]>>
> 主题: New Version Notification for draft-xu-idr-fare-01.txt
> 
> A new version of Internet-Draft draft-xu-idr-fare-01.txt has been successfully
> submitted by Xiaohu Xu and posted to the
> IETF repository.
> 
> Name:     draft-xu-idr-fare
> Revision: 01
> Title:    Fully Adaptive Routing Ethernet using BGP
> Date:     2024-07-21
> Group:    Individual Submission
> Pages:    11
> URL:      https://www.ietf.org/archive/id/draft-xu-idr-fare-01.txt
> Status:   https://datatracker.ietf.org/doc/draft-xu-idr-fare/
> HTMLized: https://datatracker.ietf.org/doc/html/draft-xu-idr-fare
> Diff:     https://author-tools.ietf.org/iddiff?url2=draft-xu-idr-fare-01
> 
> Abstract:
> 
>    Large language models (LLMs) like ChatGPT have become increasingly
>    popular in recent years due to their impressive performance in
>    various natural language processing tasks.  These models are built by
>    training deep neural networks on massive amounts of text data, often
>    consisting of billions or even trillions of parameters.  However, the
>    training process for these models can be extremely resource-
>    intensive, requiring the deployment of thousands or even tens of
>    thousands of GPUs in a single AI training cluster.  Therefore, three-
>    stage or even five-stage CLOS networks are commonly adopted for AI
>    networks.  The non-blocking nature of the network become increasingly
>    critical for large-scale AI models.  Therefore, adaptive routing is
>    necessary to dynamically load balance traffic to the same destination
>    over multiple ECMP paths, based on network capacity and even
>    congestion information along those paths.
> 
> 
> 
> The IETF Secretariat
> 
> 
> _______________________________________________
> Idr mailing list -- [e-mail] <mailto:[e-mail]>
> To unsubscribe send an email to [e-mail] <mailto:[e-mail]>