Re: [bess] Questions/concerns about BGP Color-Aware Routing (CAR)

["Dhananjaya Rao (dhrao)" <[e-mail]>]

Hello Srihari,

Thank you for your review of the BGP CAR proposal and comments. My apologies for the delayed response.

I’d like to mention that some of your comments are similar to ones made earlier by your colleague(s), and we’d responded to them with the necessary clarifications. However, I will attempt to address them again here.

To reiterate, BGP-CAR is fundamentally an evolution of the BGP-LU paradigm. It extends a regular IP prefix with a color since the goal is to support intent specific routing to given network endpoint. It is operationally and procedurally consistent with BGP-LU.

As such, its way simpler than forcing the IP-VPN paradigm into underlay routing as BGP-CT does which brings with it unnecessary overhead and suboptimal behaviors as we have commented before.

The objections you pose primarily target the extensibility we’ve defined as part of the BGP-CAR NLRI. But those are in some cases incorrect, and in other cases unfounded or even vague claims of complexity IMHO. I will try to clarify once more inline. That said, we’ll be glad to discuss any valid critiques and concrete suggestions for improvement.

Since we are defining a new SAFI to accommodate the color specific NLRI, it is apropos to introduce a modern, extensible NLRI definition. Precisely because it’s a new SAFI that will be turned on in transport networks.

The NLRI design not only addresses specific operational scenarios that we have come across efficiently, but also enables additional extensions to be accommodated in future, flexibly and efficiently.

If the goal was to introduce no changes at all or to intentionally retrofit an existing SAFI, then BGP-LU alone could be deployed with a separate underlay IP assigned for each color or intent. There isn’t a need to do much more IMHO.

Also, we once again respectfully disagree that the CAR NLRI definitions are a fundamental change to BGP. Plenty of existing BGP SAFIs have introduced similar extensions to NLRI – starting with BGP-LU for label/label-stack to EVPN with route-type specific definitions and BGP-LS with TLVs, to name a few.

Rather than make the CAR NLRI extensions hardwired, we are taking the opportunity to provide a structured definition with a route-type and TLVs for per-route data such as label, SID, prefix-SID etc. Not a fundamental change. And there are no rules in IDR that non-key info or TLVs are disallowed in a BGP NLRI. Fears regarding arbitrary misuse are misplaced since any definitions do go through IDR WG review.

Please also see some of the specific clarifications inline (DR>>)


From: Srihari Sangli <[e-mail]>
Date: Tuesday, December 14, 2021 at 10:13 PM
To: "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Re: Questions/concerns about BGP Color-Aware Routing (CAR)
Resent-From: <[e-mail]>
Resent-To: <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>, <[e-mail]>
Resent-Date: Tuesday, December 14, 2021 at 10:13 PM

Hello,

I am not sure if my email reached the authors and hence resending my email. I would like to know the comments from the authors of draft-dskc-bess-bgp-car draft. Thanks.

Thanks & Regards,

srihari…


From: Srihari Sangli <[e-mail]>
Date: Thursday, 18 November 2021 at 1:06 PM
To: "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Questions/concerns about BGP Color-Aware Routing (CAR)


Hi, Folks:



This is regarding draft-dskc-bess-bgp-car-03 . I have reviewed the previous presentations & discussions on IDR wg meetings and mailer. In preparation for the upcoming IDR interim, I am raising concerns about CAR proposal to establish end-to-end intent-aware paths. Can the authors of the draft respond to these.



IMO BGP CAR is an overkill for what we need.  Some of the proposed changes in CAR proposal look to me as pretty fundamental to BGP. While “any” protocol extensions are possible, IMHO we should review the trade-offs. I see the CAR authors are trying to solve the same problem that BGP Classful Transports aka BGP-CT solves in a different way. My objection is – while we want to provide alternative solution to classful transports, we don’t want to take BGP protocol constructs to orthogonal direction which is not good for both operators and vendors.



Personally, when I compare BGP CAR to BGP-CT, the BGP-CT is a natural extension of existing IP-VPN mechanism. I have not seen much discussion on what’s missing in BGP-CT and I’m sure more discussions may lead to improvements (if any) in that proposal.  I propose we review BGP CT proposal and provide suggestions to improve it. It seems better bang for buck than introducing new concepts in BGP which seems suboptimal and complexity from protocol extensions.



DR>> I’ve addressed this in the introductory responses above. The IP-VPN model is not a suitable model nor necessary.



Following is the list comments/objections (not a complete list) to the CAR proposal. Please see below for more details.



1.       Color ambiguity: As per details in section 2.9.3 and 2.8 of the draft, the color represents the intent within the color domain (which can span across IGP/AS domains) and as authors describe it, it is under a common administration. I have following observations and objections related to this.

·         The intent is present in two places : NLRI and attribute. This leads confusion as to which one to believe.

DR>> The draft is very clear on the scope and usage. The color is carried in the NLRI and indicates intent within domains with a consistent color to intent mapping, which is 99% of deployments. The LCM only gets attached and used if the routes traverse into a domain with a completely different color mapping. It thereby provides clarity and autonomy between such disparate administrative domains.


·         Even though we achieve some mapping across domains under common “color” administration,

·         For CAR NLRIs with LCM, implementation can become complicated to deal with color present in NLRI and color present in LCM, the effective color can vary from prefix to prefix and this may end up inefficient data storage and prefix management.

DR>> This is an incorrect assertion. The draft clearly states the usage.


·         It also adds operational complexity for troubleshooting to trace the routes and color across hops, multiple RRs and different domains. The operator has to track the effective color, sometimes it will be the color in the NLRI, and sometimes it may be in LCM, and it has to be tracked on a hop by hop basis along the multi-domain path – thus making it impractical or very difficult to track prefix and its effective color.

DR>> The NLRI of the BGP CAR route does not change across hops/domains and hence is very easy to track. I’ve clarified above where and how LCM is used.


·         One may end up with routes as below. These are two different prefixes as the color is different. But LCM makes them comparable. So can they be considered as multipaths ? Can either of the paths be used for route resolution ?  If one has to be preferred over other route, what is the selection criteria and why. The draft does not provide details:
Route1: Prefix:1.1.1.1, Color 100; attribute local-pref 10
Route2: Prefix:1.1.1.1, Color 200; attribute LCM 100, local-pref 20

DR>> There appears to be a misunderstanding of the BGP CAR solution. Above is a misconstrued example which will not happen in deployment.


·         With ADD-PATH, multiple paths for the prefix {E,C} get advertised, but the identity of the originator of the route will be lost. For troubleshooting, it may be very useful to know who originated the route, especially with color mapping across domains.

DR>> BGP-LU does not have originator identity and has been deployed extensively with ADD-PATH. Further, if needed, an attribute can easily carry originator information, when the originator is not the device (e.g. PE) itself.


2.       CAR NLRI encoding: In section 2.9 of the 03 version, the non-key fields can have “Transitive” bit. This is going towards the attribute definition way. While this may be a good idea to provide key and non-key structure for managing prefixes, I am not sure if the added complexity would be justified.

DR>> As I clarified above, adding structure is not complexity. It actually simplifies design and implementation.


·         This is going towards a mode where NLRI can become very big with many objects can exist within NLRI as non-key fields. This may also lead to more discussions when a new object is introduced as to should it be in attribute-only or as non-key field only in NLRI or both, leading to interoperable issues and misinterpretations. IMHO the use case does not mandate or lead to such a big change in NLRI encoding.

DR>> These are vague fears. The IDR WG exists for this purpose. Besides, the BGP CAR NLRI definition controls how many bytes can be sent in NLRI including non-key information.


·         I understand that moving label from prefix SID to the NLRI may improve packing, but it is not clear if there is a definite gain.  Even if there is some gain, the number of labeled routes in any network would be relatively small thus making gain insignificant. We know that operators attach communities to NLRIs that may bring in uniqueness, so you may be optimizing for a non-practical use case. I feel this is an academic exercise. The Prefix-SID has been standardized at IDR (rfc8669) and has been deployed in many networks already.  I have not seen discussions on NLRI packing as an issue raised by any operator. Also, I’m not convinced if it is worth going through protocol modifications and implementations across networks, as the so called “packing issue” will continue to exist until old implementations are around. We should not have coded prefix-sid as an attribute IMHO, but it is too late now. Associating label/prefix-SID with next hop is a better way to solve that problem. There is already a draft (draft-kaliraj-idr-multinexthop-attribute). Its worthwhile to look into that.

DR>> Not sure if this is an academic comment. Just because there could be scenarios where an optimization may not be applicable, does not mean we should not define an efficient NLRI. The target should be to define an optimal SAFI and where possible take advantage of it.


·         If an operator wishes to attach a community or some identifier to uniquely identify the point of origination of such a color prefix, then packing advantage is lost unless we bring that “attribute” into NLRI as a non-key. This is going to add more complexity.

DR>> If an operator attaches a unique community, they will hopefully be aware of its scale/implication. We are not proposing any such community in non-key field.


3.       Path compression: This happens at aggregation/pinch points. Section 2.7 of the draft describes the path availability by proposing add-path. I agree add-path gives the path diversity, but it also has negative effect.

·         ADD-PATH is a router by router feature. It has to be enabled pervasive across networks. We all know that ADD-PATH can be pretty expensive from memory and processing perspective. Not to mention the troubleshooting overhead by tracking path-ids. ADD-PATH is a good feature to avoid route oscillations when it was proposed, but using it for more use cases will be costly for the network. Operators with BGP-LU are painfully aware of this and are careful to turn on ADD-PATH in their networks for this exact reason.

·         ADD-PATH  has been tried and deployed at RR and primarily iBGP peering. We tried solving it for EBGP with edge_discriminator attribute and we do not have deployment experience of this working at inter-AS boundaries.

DR>> ADD-PATH is well understood and used where necessary. We aren’t proposing anything different.


4.       Forwarding diversity: Section 2.9.2.1 describes how a label TLV can be encoded in the CAR NLRI but the draft does not address the following scenario

·         It is not possible for a router to express different forwarding paths/encapsulations for a CAR NLRI. This can be useful in inter-AS scenarios when you want to expose the diverse paths across domains.
NLRI: 1.1.1.1, color C1, label L1
NLRI: 1.1.1.1, color C1, label L2

DR>> BGP CAR can provide the necessary forwarding diversity. Not sure what you’re thinking of.


5.       Color based resolution: Section 2.5 introduces how CAR route can be resolved, but I’m not too sure if the authors have worked out the details of the scheme.

·         For example if a router has the following routes, cost of computing the best path is not trivial. It is also not intuitive. Implementation wise, this can be inefficient walk all routing information. Either there is a CPU or memory cost to this way of color mapping.
NLRI:1.1.1.1 color 100, attribute local-pref 100
NLRI2: 1.1.1.1 color 200, LCM 100, attribute local-pref 200
NLRI3: 1.1.1.1 color 300, attribute local-pref 300
NLRI4: 1.1.1.1 color 400, attribute LCM 100, local-pref 400

·         Operationally this may be difficult for operator too.

DR>> There appears to be a basic misunderstanding on your part on how the CAR route operates across color domains and is resolved. This is not a valid example.


6.       VPN CAR: This may be an interesting concept about extending the color to CE.

·         But, is this requirement real ?

·         The draft proposes replacement of L3vpn family & ipv4-unicast which is the de facto for PE-CE protocol with CAR NLRI. This will bring in operational nightmare.

·         I’ve raised concerns with LCM above and it adds even more complexity when such intent/color is extended to CE routes IMHO from deployment, troubleshooting perspective.

DR>> CAR is designed to be extensible, and there are valid use-cases where this is applicable and will be used; fears should not constrain progress.


7.       RTC mechanism:

·         In the CAR proposal color is encoded in the NLRI and hence the current RTC mechanism cannot be leveraged. I understand that LCM would not be present for all CAR NLRIs and hence a new mechanism needs to be devised to cover NLRI with and without LCM.

DR>> We will address this in a future version.

Regards,
-Dhananjaya

I appreciate comments from the authors and others on this topic.

Thanks & Regards,

srihari…



Juniper Business Use Only