Re: [Idr] Shortest Path Routing Extensions for BGP Protocol

["Acee Lindem (acee)" <[e-mail]>]

Hi Robert,
See inline.

From: <[e-mail]<mailto:[e-mail]>> on behalf of Robert Raszuk <[e-mail]<mailto:[e-mail]>>
Date: Saturday, July 9, 2016 at 3:49 PM
To: Acee Lindem <[e-mail]<mailto:[e-mail]>>
Cc: "Keyur Patel (keyupate)" <[e-mail]<mailto:[e-mail]>>, "Abhay Roy (akr)" <[e-mail]<mailto:[e-mail]>>, "Derek Man-Kit Yeung (myeung)" <[e-mail]<mailto:[e-mail]>>, "Venu Venugopal (venuv)" <[e-mail]<mailto:[e-mail]>>, IDR List <[e-mail]<mailto:[e-mail]>>
Subject: Re: Shortest Path Routing Extensions for BGP Protocol

Hi AC,

Many thx for responding.

As far as your question 100K+ was about number of compute nodes, virtual compute nodes in flat routing (enabled via SR-IOV) & switches together as in your proposal each of them is a node in the SPT. Unless you are planning to introduce analogy to ospf areas here :)

No areas here… If you are running BGP on the 100K+ compute nodes as I believe you are suggesting with the reference to SR-IOV, you would probably not want flat routing. In many of today DCs, the compute nodes are merely hosts on ToR subnets and these prefixes can be easily computed using SPF optimizations.


In any case the most powerful property of BGP is hierarchy and (multi-level) indirection via next hop. In normal BGP deployments next hops which are carried in BGP are resolved in IGP. In DC cases where eBGP is used between all fabric stages next hops resolves to connected routes.

In your case in flat routing you no longer have such recursion.

However I assume your proposal can coexist with other SAFIs and say external to the DC desitnations will be still carried in 1/1 or 2/1 and their next hops will be resolved by routes inserted into RIB by running SPF correct?

As Keyur responded, SPF routes could be used for resolution of other SAFIs.


Last .. LFA is a great technology however I am not sure if enabling it in topologies where you have 10s of ECMP active paths and where you are ready by design for a failure handling is a right thing to do. What are other real use cases for giving up on pure BGP best path selection and normal BGP multipath ?

I agree that the value of LFA is limited if you have prevalent ECMP and a data plane that does fast ECMP rehashing in the event of failure. The other inherent advantage of SPF-based computation is that a single failure only results in the directly impacted NLRI being advertised as opposed to all the impacted prefixes having to propagate hop-by-hop. This should improve convergence and even scaling.

Thanks,
Acee



Best regards,
R.


On Fri, Jul 8, 2016 at 6:22 PM, Acee Lindem (acee) <[e-mail]<mailto:[e-mail]>> wrote:
Hi Robert,

Thanks for engaging…

From: <[e-mail]<mailto:[e-mail]>> on behalf of Robert Raszuk <[e-mail]<mailto:[e-mail]>>
Date: Friday, July 8, 2016 at 11:51 AM
To: "Keyur Patel (keyupate)" <[e-mail]<mailto:[e-mail]>>, Acee Lindem <[e-mail]<mailto:[e-mail]>>, "Abhay Roy (akr)" <[e-mail]<mailto:[e-mail]>>, "Derek Man-Kit Yeung (myeung)" <[e-mail]<mailto:[e-mail]>>, "Venu Venugopal (venuv)" <[e-mail]<mailto:[e-mail]>>
Cc: IDR List <[e-mail]<mailto:[e-mail]>>
Subject: Shortest Path Routing Extensions for BGP Protocol

Hi,

I have reviewed your proposal.

Turning path vector or distance vector protocol into link state carrier is no doubt a bold idea :).

“Fortune befriends the bold.” -Emily Dickinson


Effectively what you are proposing is to use BGP TCP sessions to propagate link state database creating first "link vector protocol" !

For start I have few questions:

Q1:

RFC7752 has gone via lot of efforts (especially sections 3.2 and up) to include number of OSPF or ISIS specific encodings. In your proposal you mentioned OSPF twice and not even once ISIS. Does it mean that you are not going to use all encoding for specific IGPs as defined in RFC7752 ?

We are using the BGP Protocol-ID defined for BGP-EPE. The IANA request will be generalized from “BGP-EPE” to “BGP” in support of Segment Routing and other enhancements. The BGP-LS NLRI are specific BGP and not either IS-IS or OSPF.


Q2:

Who creates and maintains LSDB in each BGP speaker ? Are you planning to run OSPF and or ISIS except disable it to establish any adjaciencies ?

I’ve seen designs like this in my time but have never been of a fan of them ;^). BGP will do the SPF directly and maintain the SPT. You’ll note that a simplified SPF is already done for ORR.


Q3:

Currently there are already to models to build DCs with BGP ... one uses BGP to create only lean underlay the other is to use BGP for both underlay and tenants (example project Calico for the latter). With that scale wise I think your proposal will work great for the former. However I do have concerns about using your model for the latter where say 10,000 or 100,000 /32s or /128s from each VMs are injected and you need to construct SPT with all of those.

Similar to those designs, the SPT could be limited to the underlay. However, if there is no requirement for the benefits of L2 or L3 VPNs, I see no reason why these 100Ks of leafs VM prefixes in DC center couldn’t be supported.


Q4:

Related to Q3 in your model and say flat DC routing each compute node other then just injecting 10s of /32s and being "done" now becomes an IGP node. Since your document explicitely targets Massively Scaled Data Centers (MSDCs) I am concerned that having 100,000+ IGP nodes and in many case much more is not the best idea.

100K+ switches in a single DC fabric (i.e., BGP routing domain)? I have some experience in link-state protocols and I can tell you that OSPF is I/O bound mainly due to the flooding. If done right, the SPF calculation can be done with minimal computation. While BGP-LS isn’t the world's most compact encoding, it is completely incremental.


Q5:

Have you considered just proposing an OSPF route reflector instead without stuffing BGP into the mix ? As some of you perhaps remember the work on this started around year 2000 to optimize PE-CE CSC deployments :) It seems to me very reusable for this goal.

We looked at lots of alternatives and this one seemed like the best one. Please pass me a pointer to the work you mention.

Thanks,
Acee



Best regards,
Robert.