Re: [Idr] Part 3 of CAR/CT Adoption call (7/14/2022 to 7/27/2022) - Operational Differences

[Kaliraj Vairavakkalai <[e-mail]>]

Dhananjay,

About the churn argument:

Not true. The advertised CT label does not change when local failure events happen and
nexthop changes from one nexthop to another. Because of Per-Prefix/Per-TC-Prefix label
allocation mode.

So Churn is not propagated further. About RD:EP routes propagated further, that is correct,
but it would not churn on local failure events. And the ‘further propagation’ can
be advantageous in some cases, explained towards end of this email.

About the scale argument:

In any transport-network, the number of routes an Ingress PE will see is a function of the following parameters:


a.       (NUM_PFX) Number of Egress-nodes originating the transport-prefix (CT: IP-prefix with Unique-RDs).



1.       Unicast-prefix (advertised by either pe11 or pe12): this number will be 1.

   Ref: our example topology: https://datatracker.ietf.org/doc/html/draft-kaliraj-idr-bgp-classful-transport-planes-17#section-18


2.       Anycast-prefix (advertised by both pe11 and pe12):

                            i.   If unique-RD is used on pe11, pe12: this number value will be 2.

                          ii.   If same-RD is used on pe11, pe12: this number value will be 1.



b.       (NUM_BN) Number of Border-nodes fanout for ingress-PE in its domain, that the route is traversing with “nexthop-self”.



1.       This number value will be 2, for this example topology.

Note: Addpath is enabled with nexthop-unchanged at RRs. Not enabled at EBGP-boundary or at nexthop-self BNs.

So the formula is (NUM_PFX * NUM_BN) for this scenario. And I think that holds good for LU, CT, CAR.

For a unicast prefix, the number of CT routes at ingress-PE pe25 will be 2.
For a anycast prefix, if unique-RD is used, number of CT routes at pe25 will be 4.
                      if same-RD is used, number of CT routes at pe25 will be 2.

For unicast-prefixes, it is same scale for CT and CAR.
(This is assuming other conditions remain equal like ‘addpath-EBGP/addpath-with-nhs is not used’.
For any solution using addpath-send in contiguous domains, this number will be higher. So we want to
 use addpath judiciously and not everywhere).

For anycast-prefixes,


·         When using same-RD, the scale in CT is same as CAR (provisioned with same IP:Color on all anycast-sites).

·         When using unique-RD, the scale in CT is more, but bounded. It’s a function of the variable: “number of anycast-sites”.

>       NV2>        Install a local multipath route for forwarding, and originate a single new route. This leaves the local router in control of multipath.


And yes, this is another possible option, and the RD will identify the Aggregating BN, that is originating the new route.


And please note, the further propagation of RD:AnycastEP routes can be advantageous as-well in following ways:
(These are what are not possible with CAR, because it doesn’t support the a.2.i variation above)



•         It gives the ingress an idea of how many sites are currently advertising the anycast-prefix.

If all but one anycast-site are lost, just looking at the route-list can raise an alarm,

and operator in ingress-domain can take corrective action to avoid an impending loss of connectivity

to the anycast-prefix.



•         When per-prefix-label is used (based on RD:EP) instead of per-tc-prefix-label (based on TRDB EP)

The ingress can get TE control to direct/ecmp/replicate traffic towards the different anycast sites.

In this case the label allocated/carried on these RD:EP CT routes will be unique. And even here,

the local failure churn Will Not be propagated beyond local-BN, because per-prefix-label is in use.

In essence, our emphasis has been on the fact that RD allows all this flexibility, variations. We expect Unique-RD will be used in most
Cases. Same-RD may also be used in some cases based on customer need. We don’t disagree with that, but we do recommend use of unique
RDs, for the above reasons. If you got a feeling from reading the draft that only one of the variations MUST be used, that is not the case.
We will refine the text in the next version.

IMO, a TE solution is all about ability to provide better visibility. CT allows for that, while letting
user control how much visibility is needed.

Thanks for all the discussion.

Some more responses inline, to your original email. KV>

Thanks
Kaliraj

From: Idr <[e-mail]> on behalf of Dhananjaya Rao (dhrao) <dhrao=[e-mail]>
Date: Wednesday, July 27, 2022 at 10:08 PM
To: Natrajan Venkataraman <natv=[e-mail]>, Susan Hares <[e-mail]>, [e-mail] <[e-mail]>
Subject: Re: [Idr] Part 3 of CAR/CT Adoption call (7/14/2022 to 7/27/2022) - Operational Differences
[External Email. Be cautious of content]


Hi Nats,

You have not addressed the specific issues we raised on your proposed revised solutions to the problems with unique RDs. In fact you have conveniently skipped past the entire list of issues and questions.

Interestingly, you say below :

“NV3> As per Section 8, there is nothing that is “strongly” recommended. Customers will be provided the flexibility to configure how they want their endpoint/BN and how they would want their RD under it.”

Literally read, -Section 8- does not include “strongly” recommend. But it describes potential benefits of using unique RDs, without describing any of the limitations/disadvantages.

However, other sections in the latest version (-17) of the CT draft have following statements:

Section 10.9 :
“  Deploying unique RDs is strongly RECOMMENDED because it helps in troubleshooting by uniquely identifying originator of a route and avoids path-hiding. “

Section 10.1 :
“      Unique RD SHOULD be used by the originator of a Classful Transport route to disambiguate the multiple BGP advertisements for a transport end point.
“
Section 10.2:
“Unique RD SHOULD be used by the originator of a Classful Transport route to disambiguate the multiple BGP advertisements for a transport end point.”

I didn’t check further. And of course, we have your many statements on the list and in IDR presentations.

Now, even though you did not address the specific issues below, from the couple of statements you made about operator flexibility to configure, you appear to be leaning towards using the same RD values as a potential solution to the quandary of slow convergence, providing basic BGP multipath/local repair without redundant routes and churn.

I couldn’t find a single mention of configuring the same RD values on originators in the -17 version of the CT draft. Apologies if I missed it. Or perhaps that will be in the next version.

Using the same RDs on originators undermines all benefits you have repeatedly stated for having an RD in NLRI.

But if that’s really the only practical option, then it begs the question, why is an RD, and its subsequent requirement for a VPN-like import at every hop, needed at all in the transport NLRI ?

Regards,
-Dhananjaya

From: Natrajan Venkataraman <natv=[e-mail]>
Date: Monday, July 25, 2022 at 10:33 PM
To: "Dhananjaya Rao (dhrao)" <[e-mail]>, Susan Hares <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Re: [Idr] Part 3 of CAR/CT Adoption call (7/14/2022 to 7/27/2022) - Operational Differences

Hi DJ,

Thanks for raising a separate question as per my request. Please find my answers with NV3> below.

Best,
-Nats-

From: Idr <[e-mail]> on behalf of Dhananjaya Rao (dhrao) <dhrao=[e-mail]>
Date: Monday, July 25, 2022 at 7:17 AM
To: Susan Hares <[e-mail]>, [e-mail] <[e-mail]>
Subject: Re: [Idr] Part 3 of CAR/CT Adoption call (7/14/2022 to 7/27/2022) - Operational Differences
[External Email. Be cautious of content]

Hi CT co-authors,

I’m starting a new question on one of the basic issues that the CT RD based approach is exposed to, as suggested by Nats in the thread : https://mailarchive.ietf.org/arch/msg/idr/Quqc-Z3--lfmyfgzBXRNB-b9730/<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/idr/Quqc-Z3--lfmyfgzBXRNB-b9730/__;!!NEt6yMaO-gk!C94cQ4ZpYSgBMtBceQSKUiHz2a3gG_wxXOMPUHEDx-xob6L7HZM9XzpLhed4SiarUX3GYrPJ0OsZzVq4hxbwOn8RVRtUAA$>

This is not a new issue, though it wasn’t discussed as part of Q3. And a couple of options Nats responded with raise new questions.
This issue impacts all use-cases discussed, and has a greater impact on the popular Anycast scenario.


CT strongly recommends using unique RDs for advertised routes – the claimed benefits being troubleshooting, enabling forwarding diversity etc.

NV3> As per Section 8, there is nothing that is “strongly” recommended. Customers will be provided the flexibility to configure how they want their endpoint/BN and how they would want their RD under it. Please read on for further explanation of label allocation modes.
https://www.ietf.org/archive/id/draft-kaliraj-idr-bgp-classful-transport-planes-17.html#name-use-of-route-distinguisher<https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-kaliraj-idr-bgp-classful-transport-planes-17.html*name-use-of-route-distinguisher__;Iw!!NEt6yMaO-gk!Hgm3XUYkjvxvplXx8gp7-u4mIY_BwOVHdV1vZhsCg9f8f7wB6lld09Hw2cMBYhKW5wKixFBWLo-ESVe-wtfWtc2wTk9gUVVW$>



·         But the use of unique RDs create e2e LSPs to the originating nodes that do not provide local domain convergence and multipath. Withdraws of CT routes need to be propagated up to ingress PEs for traffic re-convergence.


For this, CT defines a “RD stripping”/import procedure.
https://datatracker.ietf.org/doc/html/draft-kaliraj-idr-bgp-classful-transport-planes-17#section-10.4<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-kaliraj-idr-bgp-classful-transport-planes-17*section-10.4__;Iw!!NEt6yMaO-gk!C94cQ4ZpYSgBMtBceQSKUiHz2a3gG_wxXOMPUHEDx-xob6L7HZM9XzpLhed4SiarUX3GYrPJ0OsZzVq4hxbwOn8JFyy-Sw$>


·         However, it still does not provide failure localization/suppression at intermediate transit nodes.





For illustration, we take the same topology from the other thread, filled out slightly:

                                               Domain2
                                                      --------   BN2 -- -- -- E2 (IP1, C1)
                                                     |
                              E1  -- -- -- BN1
                                                     |
                                                      --------   BN3 -- -- -- E3 (IP1, C1)
                                  Domain1                         Domain3


As per CT procedures, there will be two CT routes for Anycast IP IP1 -  RD2:IP1 from E2 and RD3:IP1 from E3

As described in the CT draft referred section, at a transit node, such as BN1 in Domain1, both routes come together via the RD stripping / import procedure to form a multipath in C1 RIB.

The draft claims that this keeps churn about failure of node E2 from propagating beyond BN1.


·         However, this is not accurate. Since these are 2 unique routes (with RD2 and RD3), each has its own bestpath and both will be advertised out from BN1 towards E1 by default.


·         This is despite having same local label allocated by BN1 and same next-hop (BN1) being advertised, hence being redundant advertisements that serve no purpose beyond BN1.


·         So, the churn about the failure of either of them will be unnecessarily advertised to E1.  And note, with Anycast, there could be routes from many such egress nodes across domains.

KV> As described above, there will be no churn. Per-prefix-label/Per-TC-prefix-label label allocation mode is used.
KV> So when either BN2,BN3 fail, no route update is sent to E1.

The draft does not elaborate whether or how the redundant routes are suppressed.

But now, there are two possible options briefly proposed by CT co-authors in the above Q3 thread.


a.       “NV2>      RD is flexible enough that for cases like Anycast, where multiple paths are not desired, RD may be set to the same value across routers.”


·         Configuring same RD is possible. But it contradicts the heavy emphasis CT has placed on advertising unique RDs for troubleshooting and identifying originator. Is that not needed here ?


·         Also, given that this Anycast IP is intended to originated by routers across different providers (color/admin domains), how do the providers coordinate the configuration of the same RD values on their respective Anycast routers ?


b.       NV2>        Install a local multipath route for forwarding, and originate a single new route. This leaves the local router in control of multipath.



·         This is another interesting option. Since there are two or more RD routes that contribute to the multipath for the stripped/imported route multipath, which RD route will be selected to be sent to peers ? And what is the selection criteria ?


·         The statement above uses the term ‘originate’, which could suggest it is a route from the intermediate node, e.g BN1. If so, is that with a local RD ?

KV> yes. It will be local RD of the intermediate node, that is playing the aggregator role.


·         That again contradicts the emphasis on keeping the original RD. Besides, the co-authors have stated earlier that there is no RD rewrite. The above logic would effectively be an RD rewrite, which also has implications.

KV> It doesn’t contradict. The RD is now identifying the BN that originated this aggregated route.
KV> It’s not a RD-rewrite. Since the original received BGP routes are not propagated further.
KV> Only the new originated route is advertised. It is similar to Aggregation. <EOM>

It will be useful if the authors clarify on these options and the operational/protocol considerations we’ve described above.

NV3> This claim is not true. CT allows for multiple fail-safe mechanisms including local repair. Let us explore your use case further based on section 18.4.2
https://www.ietf.org/archive/id/draft-kaliraj-idr-bgp-classful-transport-planes-17.html#name-local-repair-of-primary-pat<https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-kaliraj-idr-bgp-classful-transport-planes-17.html*name-local-repair-of-primary-pat__;Iw!!NEt6yMaO-gk!Hgm3XUYkjvxvplXx8gp7-u4mIY_BwOVHdV1vZhsCg9f8f7wB6lld09Hw2cMBYhKW5wKixFBWLo-ESVe-wtfWtc2wTlMmkDtr$>

NV3> There are two ways in which operators can choose to allocate label for BGP-CT on SNs and BNs. Again, these are conscious operator choices based on the use-cases the customer is trying to solve.

1.       Per-Prefix/Per-Transport-Class (Unique Label for IP1/Transport-Class)

2.       Per-Prefix (Unique Label for RDx:IP1)
NV3> Which the customer can couple with on SNs and BNs


1.       Same RD for a transport-class

2.       Unique RD for a transport-class

NV3> The above building blocks provide the operator the flexibility to achieve a broad set of the use-cases (including the ones mentioned above) without getting affected by path selection pinch points, allowing for local-repair at Transit BNs as well as availability of unique paths up until the Ingress “E1” node.

NV3> If it is not clear in the draft, then we can take an action item to add text to the draft to update this detail on various label allocation and RD usage modes as part of Section 8. I will add this to the presentation slides for BGP-CT as action items requested by Susan for the next 4 months. Once this is clarified in the draft, I hope we can come to a consensus.

NV3> Thanks again for bringing this out.


Regards,
Dhananjaya


From: Idr <[e-mail]> on behalf of Susan Hares <[e-mail]>
Date: Friday, July 15, 2022 at 1:47 AM
To: "[e-mail]" <[e-mail]>
Subject: [Idr] Part 3 of CAR/CT Adoption call (7/14/2022 to 7/27/2022) - Operational Differences

The IDR Adoption call for CAR and CT technologies is extended for an extra week
(7/14/2022 to 7/27/2022).

The IDR adoption call on Q2 has brought many discussions on pro/cons of the CAR
and CT BGP technologies:
https://mailarchive.ietf.org/arch/msg/idr/AP_ClbZgpkX6CNy7TaZiU8SMD5w/<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/idr/AP_ClbZgpkX6CNy7TaZiU8SMD5w/__;!!NEt6yMaO-gk!C94cQ4ZpYSgBMtBceQSKUiHz2a3gG_wxXOMPUHEDx-xob6L7HZM9XzpLhed4SiarUX3GYrPJ0OsZzVq4hxbwOn8viUHGVg$>

Since the IDR chairs agree with the summary of Jeff Haas (IDR Co-chair) posted on March 21, 2022 –
that for route resolution and route origination/propagation, BGP-CAR and BGP-CT
are functionally identical, but operationally different.
    ( https://mailarchive.ietf.org/arch/msg/idr/e69NRd9i2aG0WUxFkShEfQHZsHo/<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/idr/e69NRd9i2aG0WUxFkShEfQHZsHo/__;!!NEt6yMaO-gk!C94cQ4ZpYSgBMtBceQSKUiHz2a3gG_wxXOMPUHEDx-xob6L7HZM9XzpLhed4SiarUX3GYrPJ0OsZzVq4hxbwOn8MbBnGzQ$>

This forum (Part 3) is place to discuss operational differences between the CAR and CT drafts.

These procedures for this mail thread are:


A.       On original discussion on point in a draft

1.       Text in either the CAR or CT draft that is applicable to the operational difference.

2.       Provide a sample topology.

3.       Discuss the pros/cons clearly stating whether your post is a: pro, con, or clarifying question.
Try to split clarifying question posts away from pro/con posts. Remember, you can post to this forum several times.
In your technical discussion, define your terms and avoid value judgement.



B.       On a discussion based on message in Q2 or Q1 mail list:

1.       Quote and link to Q1 or Q2 forum’s message.

2.       Text in either the CAR or CT draft that is applicable to the problem
      (Either provide the text or give specific reference (section, paragraph, and line(s)).


3.       Provide a sample topology.

4.       Summarize your viewpoint as: correction, clarifying question, pro, or con.

5.       Discuss the pro/con or text giving technical discussion.
In your technical discussion avoid define terms, give specific
examples in terms of draft text, sample topology, and facts.
If you state an opinion, back it up with technical facts.

If you are making a correction, assume the other person either misunderstood
or has a difference of opinion.  Respect each other as each IDR member is
entitled to their opinion. As a group we are working toward excellent BGP
in deployed technology.

In your delightful passion on these two technology additions to BGP (CAR and CT), please adhere to these rules.

Thank you, Susan Hares



Juniper Business Use Only


Juniper Business Use Only


Juniper Business Use Only