[Idr] 答复: I-D Action: draft-xu-idr-bgp-route-broker-02.txt

["[e-mail]" <[e-mail]>]

Hi,

发件人: Robert Raszuk <[e-mail]>
日期: 星期五, 2023年8月11日 16:41
收件人: [e-mail] <[e-mail]>
抄送: idr@ietf. org <[e-mail]>
主题: Re: I-D Action: draft-xu-idr-bgp-route-broker-02.txt
Hi,

For the benefit of IDR WG participants could you clearly state by email or in the draft what is the problem with addressing your deployment needs by using hierarchical VPNv4/v6 route reflection (as is - no single line of code change needed) + RTC along with it ?

【Xiaohu3】Assume there are 100K vPEs/servers, 100k VRFs/tenants, 100M VPN routes (most of them are host routes) in a large-scale data center

So you have 1 VRF per vPE ? And each such VRF is advertising 1000 routes ? Not a very realistic case but let's continue ...

[Xiaohu4] As I mentioned earlier, it assumes that one vPE is attached with 20 VRFs on average. In fact, for some VRFs which contains a lot of containers, 1000 routes per VRF may not be enough.

SDN environment, you deploy a two-layer hierarchical RR architecture where there are 10 top-level RRs and 10 bottom-level RRs. As mentioned in the draft, there is no scaling pressure on top-level RRs since each just needs to store about 10 percent of the 100M VPN routes and peers with 10 bottom-level RRs.

Please observe that in practice static allocation of RT to top level RRs never work well. And now in the other mail you suggested to make it even worse with unique export  RT per VRF even in the same VPNs.

[Xiaohu4] Can you give concrete reasons why it doesn’t work well?

However, each bottom-level RR needs to peer with about 10K vPEs, in the worst case where the collection of VRFs attached to the 10K vPEs cover all VPNs within the data center, therefore each RR would have to store the 100M VPN routes.

Then increase the number of lower level RRs. If you have 100K PEs, deploy 1000 of lower level RRs.

[Xiaohu4] It may be acceptable provided the cost associated with 1K RRs was not a problem.


Furthermore, it’s almost impossible to use the peer-group attribute for the 10K vPE peers since the VRFs attached to one vPE are different from that of another vPE.

Usage of statically configured peer-groups is an archaism. Modern BGP stacks use the concept of dynamic update groups and peer templates with inheritance. Besides this is VPN env so update generation is optimized by attribute based walk.

【Xiaohu】I doubt the effect of dynamic update group in the data center SDN environment where it’s unacceptable to attach the same VRFs to a group of vPEs.

As a result, each bottom-level RR would have to hold a local-RIB containing 100M VPN route entries and 10K RIB-out tables with each consists of about 20K VPN routes (assume each vPE is attached with about 20 VRFs). Besides, the route update frequency would be very high due to the widely adoption of containers.

Again you have made a suboptimal choice of the number of RRs and now you are drawing conclusions based on it.

BTW, there is no need to introduce the headache issue associated with RTC in the hierarchical RR architecture as mentioned in https://www.ietf.org/archive/id/draft-mohanty-idr-rtc-hierarchical-rr-00.html.

It's not a headache but a solution.

Besides, you can still drop locally. Dropping is cheap.

For an example, there is no need for the top-level RRs to reflect the RT membership advertisement received from the route brokers.

With hierarchy of RRs you do not need to do that either. You can push your static RT ranges configured on the top level RRs and be done just like in your current proposal. But very soon you will see that static RT range assignment on top level RRs is suboptimal.

【Xiaohu4】It seems straightforward for top-level RRs acting as centralized collectors and distributors of VPN routes, to be preconfigured with a block of RTs and then advertise them towards its peers.

[Xiaohu3] It’s true that Virtual RRs are widely used in the data center SDN environment. However, it’s not that easy to scale the well-known BGP-based SDN solution to support more than 5K vPEs.

Then maybe it's time to consider not using the well-known BGP based SDN solution for such deployments ?

[Xiaohu4] Maybe RTC is not a good choice for data center SDN scenarios.  ORF-based route pull modes as described in (https://datatracker.ietf.org/doc/draft-xu-bess-l3vpn-prefix-orf/) may be more efficient since it doesn’t need to distribute any given RT membership from PEs to PEs.

[Xiaohu3] Route brokers don’t need to be capable of handling all VPN routes at any given period of time. There are many ways to achieve that goal. BTW, it’s acceptable to cost a relatively long period of time for VPN route convergence when restarting all vPEs within a data center due to some unexpected reasons (e.g.，power interruption in the scope of a data center）.

Putting aside a debate if this is good or bad idea to space in time BGP processing from any peer till some other peer's routes are accepted and all propagated to the top level RRs - how are you going to to do that ?

Are you going to keep rejecting BGP OPEN session establishment or if the session is already established are you going to artificially stop peers from sending by say TCP zero-window block ?  Just curious how are you going to delay this.

[Xiaohu] Honestly speaking, the above approaches seems to me a little bit clumsy, sorry. In fact, there are many elegant ways to achieve the above goal, for example, you can make the vPEs as passive BGP peers which would not actively initialize BGP session with route brokers. In other words, route brokers could initialize BGP sessions with those passive BGP peers deliberately according to their current resource status.

Best regards,
Xiaohu

Thx,
Robert