Re: [Idr] WG adoption call - draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)

[Kaliraj Vairavakkalai <[e-mail]>]

Igor, pls see inline KV3>

Thanks
Kaliraj


Juniper Business Use Only
From: Igor Malyushkin <[e-mail]>
Date: Saturday, November 25, 2023 at 3:39 PM
To: Kaliraj Vairavakkalai <[e-mail]>
Cc: Susan Hares <[e-mail]>, [e-mail] <[e-mail]>
Subject: Re: [Idr] WG adoption call - draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)
[External Email. Be cautious of content]

Hi Kaliraj,

Thank you for the detailed comments, mine are below as usual (IM3).

пт, 24 нояб. 2023 г. в 23:43, Kaliraj Vairavakkalai <[e-mail]<mailto:[e-mail]>>:
Hi Igor, please see my responses inline KV2>

Thanks
Kaliraj


Juniper Business Use Only
From: Igor Malyushkin <[e-mail]<mailto:[e-mail]>>
Date: Friday, November 17, 2023 at 5:24 AM
To: Kaliraj Vairavakkalai <[e-mail]<mailto:[e-mail]>>
Cc: Susan Hares <[e-mail]<mailto:[e-mail]>>, [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Idr] WG adoption call - draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)
[External Email. Be cautious of content]

Hi Kaliraj,

My responses are below ([IM2]).


пт, 17 нояб. 2023 г. в 13:32, Kaliraj Vairavakkalai <[e-mail]<mailto:[e-mail]>>:
Hi Igor,

Please find some responses inline. KV>

Thanks
Kaliraj


Juniper Business Use Only
From: Igor Malyushkin <[e-mail]<mailto:[e-mail]>>
Date: Wednesday, November 15, 2023 at 11:45 PM
To: Kaliraj Vairavakkalai <[e-mail]<mailto:[e-mail]>>
Cc: Susan Hares <[e-mail]<mailto:[e-mail]>>, [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Idr] WG adoption call - draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)
[External Email. Be cautious of content]

Hi Kaliraj,

Thanks for your response! In general, I agree with the list above and personally find your work promising. I just want to clarify several things. Please, see my inline.

чт, 16 нояб. 2023 г. в 05:14, Kaliraj Vairavakkalai <[e-mail]<mailto:[e-mail]>>:
Hi Igor, thanks for your comments.

This MNH draft applies to single path carrying an ordered set of one or more nexthops.
[IM] That's about outgoing messages I believe. I asked about the sources of it. Let's say, we have two paths for the same destination: x.x.x.x/y {1;2;3} and x.x.x.x/y {5} from different peers (and with the different next-hops as well). A node selects the latter due to its shorter AS_PATH, would it be possible to propagate the route further with the MNH attribute and both next-hops?
    KV> Good question. The node may readvertise the best path with a MNH containing the nexthop (EP1) of best path as active leg, and nexthop (EP2) of inactive path as backup leg. E.g.,

        MNH-X:
          +NFI-X <Primary, Num-Nexthops=2>
                +FI-X <Action=Push, RelativePref=1>
                     -FA-U <Type=1, EP=EP1>
                +FI-Y <Action=Push, RelativePref=2>
                     -FA-X <Type=1, EP=EP2>

 [IM2] Ok, thank you, now I understand it better.

Yes Addpath advertises multiple paths along with their attributes, but only if they have unique nexthops. So as I see it, the real use of Addpath is also to advertise multiple nexthops. But Addpath does not specify any relationship/order in the nexthops sent, and uses high RIB-out/in scale, because of using all attributes.

Conceptually, if the sending BGP speaker is able to come up with “what kind of multiple-nexthop forwarding” is needed at the receiving nodes, then that can be conveyed in a single route update in a more expressive manner, instead of carrying all paths to receiver and further consuming CPU in multipath computation at all receiving-nodes. MNH just provides that expressiveness on the wire. How the sending speaker arrives at the forwarding-info to be sent in MNH (ECMP, Ordered fallback, WECMP, etc) - there can be more than one ways to figure that out, based on different usecases - as we are discovering. This just opens up a BGP based standard API to the box’s BGP RIB.
[IM] Thanks for the additional clarification.

And about the point on carrying labels in unlabled families: nodes not supporting MNH will just ignore it. Only those that have configured an AF to use MNH will use it. So I think there is no surprise element to any old BGP speakers receiving the label/MNH. If a receiving node that understands MNH does not understand enough of the contents to safely use it, they don’t use it (‘Attribute Discard’ approach). But I agree, the error handling will evolve as the draft matures.
[IM] I'm worried about the approach when we define some new attributes and significantly change the very nature of an address family. Where does this road lead us? I consider an address family not only as an encoding container but also as some semantics behind it, and it looks like additional attributes can alter it in any possible way.
   KV> I get your concern. I consider an address family as having a core business logic. And these encoding enhancements should not alter that business logic. They should just help in achieving that business logic better.
   KV> IOW, encoding deficiencies should not come in the way of achieving the business logic better. I agree with you that the core business logic of an AF should not be diluted. But I also feel giving any AF a little more expressability
   KV> should not hurt its business logic.
[IM2] Well, we can already send prefixes for one AFI with next-hops from another. A more detailed text about the path resolution of unlabeled families with labeled next-hops would be helpful.

KV2> Since that is already existing functionality, documenting that elsewhere may be good. may be the LU-EPE draft, which makes use of that functionality?

[IM3] Agree. And, as I understand the process, after the adoption you would probably be asked to remove the rest use cases from the body.

KV3> yea, I was thinking of moving the usecases to a separate document.
Additionally, I'm curious about the process of withdrawing a label when a connected with it next-hop is alive it shouldn't be withdrawn.

KV2> I’m presuming this is a general question, not referring to any specific text in the draft?

[IM3] Yes. I can’t express the exact problem at the moment. But it is clear to me, that we don’t only have the possibility to advertise service routes with labels now, but we also have some dynamic situations when a sender has a reason to re-advertise us a bunch of routes because something has happened with its underlying MPLS infrastructure, especially for Inter-AS links.

KV3> are you talking about a new label being advertised because of nexthop-reachability changes? Conceptually, the advertised label should not change if <Received-label, NH> tuple havent changed.
KV3> the reachability change to the NH should be absorbed locally. Though there could be some implementation dependency here.

Even when someone advertises service routes via labeled unicast (please, no), in this situation it leads to the withdrawal of these routes, because they are tight to their labels.
KV3> I didn’t understand this. Withdrawal will be seen if the BGP route’s NH is unreachable (e.g. no tunnel reachability, in a bgp free core). And that would be same for unlabeled or labeled cases.
KV3> So I am missing the scenario here.

But maybe my understanding is wrong and having a label attached to SAFI 1 by MNH makes a route non-decoupling from its label, just like for SAFI 4 and 128. Although, the text says that NHS nodes delete the attribute and only may attach it again, but not a must. That's probably another side of this dynamic: some hops can be labeled, some not.
KV3> That’s right. MNH is typically expected to be used intra-AS between and ingress and egress PE, and not advertised outside an AS.
KV2> In my view, the node’s liveness and label-advertisement are decoupled. A label could signify a service-endpoint at a node or the node itself.
KV2> So a ‘label’s association with a BGP FEC’ can be changed/withdrawn without the node actually going down.

[IM3] I agree, but it is all good only when the sender is LER. An LSR depends also on some MPLS-transport which have a less stable nature than service endpoints and even nodes themselves.
KV3> I think both LER and LSR are affected by NH-unreachability (causing withdrawals) in the mpls-transport, if it is a BGP-free core that uses mpls tunneling.

KV2> Also, to set some baseline, stating my understanding: in BGP the (downstream allocated) label itself is not withdrawn by itself. It is not part of the key in NLRI. Only the FEC in NLRI (e.g. SAFI 128, SAFI 4) are withdrawn, and as a result
KV2> the association with the previously associated label is removed. IOW, the (downstream allocated) label advertised in the 8277 NLRI is just used in the nexthop at receiving node.

[IM3] Agree. SAFI 4/128 gives us the ability to send an IP route and signal LSP toward its next hop at the same time. A label inside an MNH attribute can be attached to a route dynamically and anywhere.
KV3> Correct. E.g., in 6PE an egress-ASBR can attach MNH with explicit-null to IPv6 service-route, which will be consumed by the ingress-PE. If re-advertising the route, the PE does not need to attach MNH.
KV3> No change in service family AFI/SAFI..
KV2> MNH carries the (downstream allocated) label in an attribute, which actually expresses this non-key aspect better.

[IM3] I'd say, it expresses it in a broader way, yes. But nothing prevents us to attach an MNH for SAFI 4 instead and express whatever we want.
KV3> True, that’s not prohibited.
KV2> Only a (upstream allocated) label advertised in AFI 16399 NLRI (MPLS Namespaces) can be explicitly withdrawn because in that case, it is part of the FEC/key in the NLRI.
I feel the flexibility (carrying labels in unlabled families) itself is beneficial. Even today we have usecases (LU-EPE, 6PE, 4PE) where we want to impose labels on unlabeled service routes by resolving over labeled families. And some of these usecases (6PE, 4PE) require redistribution between Internet-families and Labeled-AFs which is risky. Using MNH allows safe walled gardens with consistent service, transport AF-layers, avoiding such redistribtions. And, I agree that interaction/precedence of the label carried in MNH with the label carried in other places on the route need to be specified.
[IM] At this very moment, I can't see where this flexibility is justified. It more looks like the approach to get rid of the LU.
KV> I feel having transport layer families (like LU) and resolving service families over them has the advantage of indirection and BGP PIC.
KV> So I think we cannot get rid of LU as a transport-family, unless the indirection is provided by some other means.
     KV> Its core business logic is to carry transport end-points.
     KV> Having flexibility to carry label on service-routes does not mean we need to eliminate a transport layer family. it just allows additional expressability for AFs at each layer.
[IM2] I agree with everything about the LU and for exactly this reason I want to better understand when the LU doesn't suit and we should allocate extra labels for service routes.

KV2> Firstly, one thing I wanted to clarify: just because of carrying a label, the business logic of an AF is not altered. E.g. both SAFI 128 and SAFI 4 carry labels. But still they have distinct business logic,
KV2> the former is a service-family and latter is a transport-family.

[IM3] Let’s take a closer look at your example with SAFI 1 vs. 4. One can, intentionally oversimplifying, consider SAFI 4 as SAFI 128 for GRT (Global Routing Table). And, to my bitter disappointment, lots of people actually use SAFI 4 for service prefixes (for both families). We can see that the industry decided to use v6 labeled unicast solely for spreading reachability (notorious RFC4798) while at the same time, its brother is used in both ways.
KV3> I understand your concerns. We want to avoid such overloading of SAFI and redistribution. But I wonder if inability to carry label in SAFI 1 may be one reason why people tend to use SAFI 4 for service prefixes.
KV3> I think we need to raise awareness about this, and also create alternatives that allow people to keep the SAFI separation.
KV3> I also feel resolving over SAFI-4 transport to impose label is not enough, we may need an alternative to carry label in SAFI 1 directly.
Some vendors still consider v6 LU not the same way as v4 LU. Thus, we see some flexibility in business logic here depending on AFI rather than SAFI. So, I cannot say that the SAFI 4 is a transport family (I’d like to, but I just can’t). I think we will see the same for SAFI 1 and labels inside MNH, unfortunately. People will use it instead of LU because modern marketing teaches them to use fewer protocols and technologies.
KV3> I realize your fear. I think raising awareness is the way to deal with such marketing. We see outages now and then because of putting all eggs in one basket, or such SAFI redistribution.
KV3> Keeping transport-routes separate in SAFI-4 is advantageous for quicker convergence and better security.  Since its scale is low. Different layers have different propagation scopes and speed.
KV3> If we mix trasport routes in one SAFI-1, we loose such advantages. We can add text regarding this in this draft if that helps.
KV3> One important thing I feel is: ‘not doing anything’ is not going to help in countering such marketing. We need to create alternatives, as well as raise awareness about best practices.

LU hasn’t ceased to exist, because SAFI-128 can carry a label. It is used in conjunction with SAFI-4. I expect the same thing to happen with SAFI 1 and SAFI 4.

[IM3] The first part of this statement is truth. For the second part (specifically, for SAFI 1, I have no questions for SAFI 4), I've already expressed my view. Let's see.
KV3> Ack.

KV2> I think the right question to ask is: in a certain usecase (e.g. 6PE, LU-EPE) whether we are carrying a service-label or a transport-label. It is possible that we have been carrying a service-label in a transport-family,
KV2> just because there was no other way to carry it in service family so far. The explcit-null in 6PE is actually a service-label, it binds to the service-fec(IPv6). We just impose it by virtue of recursive resolution over a transport-family.
KV2> That is the reason it forces us to use two loopbacks (a workaround) when we want to do nexthop-self and EPE-style 6PE simultaneously.

[IM3] To me, both cases are about the transport labels, not the service ones.
KV3> Taking example of explicit-null, it instructs to lookup in GRT inet-table. Similar to a VPN-label, that instructs to lookup in VRF inet-table. That’s the reason I consider it as service-label.

Having any reachability inside transport families is a compromise, not a solution. So, the movement toward any labels attached to service families is very concerning to me.

KV2> Further, taking the e.g. of EPE, there are circumstances, where the A/A or A/B relationship of two EPE peers is not expressable using a single LU-label, without making an assumption that the two peers give us exactly the same
KV2> service-family routes at all times. We know this may not be true unless the two bgp-peer-session are on parallel-links between two nodes without any bgp policies, and even then momentarily they can get out of sync.
KV2> I think such assumptions arise because the EPE label is actually a service-label. But we are imposing it by virtue of recursive resolution over a transport family.

[IM3] That’s interesting because from my side the whole idea behind EPE is to have distinct labels for all possible peers. We attach a label to a next-hop and we can’t have the same next-hop for different peers, thus we have different labels.
KV3> There is also the FRR backup options in LU-EPE, I was referring to backup-peer-link described in epe-draft sec 6.1.1<https://www.ietf.org/archive/id/draft-gredler-idr-bgplu-epe-15.html#section-6.1.1>.
KV3> In that case, the EPE LU label doesn’t point to a single peer, it points to a FRR nexthop which has a primary peer-link and backup-peer-link.
KV3> but all service routes advertised with that EPE peer/32 (e.g. ASBR{1,3}.link1 in Fig2 sample topology<https://www.ietf.org/archive/id/draft-gredler-idr-bgplu-epe-15.html#figure-2>) as nexthop may not share the same backup-peer (ASBR{1,4}).
KV3> I think this problem happens because the EPE label is gathered by the resolution over a transport-layer. If it is brought closer to the service-route, we get better more-specific granularity.
That makes these labels transport ones, not service. Any synchronization of receiving routes from different peers comes into play if we don't want to advertise all routes of these peers because of MNH. Classical EPE doesn't require it. Is my understanding correct?
KV3> So I was talking about classic epe-draft sec 6.1.1<https://www.ietf.org/archive/id/draft-gredler-idr-bgplu-epe-15.html#section-6.1.1> only.  It is possible that you are not using this FRR-backup option today, the ip-forward backup<https://www.ietf.org/archive/id/draft-gredler-idr-bgplu-epe-15.html#section-6.1.3> option is more widely used, AFAIK.

KV3> And one thing to clarify about the ‘suggested’ model of doing “EPE with MNH” – In this model, add-path will still be used from ASBR to RR to convey all EBGP SAFI 1 routes.
KV3> MNH is used just to carry the EPE-label in SAFI-1 routes (addpath enabled), instead of having to resolve over LU path to get the EPE-label.  That provides better granularity
KV3> of specifying FRR EPE label per service-route, Instead of per transport nexthop. When the epe enabled ebgp-peer goes down, the traffic-repair will happen
KV3> locally at the egress ASBR, instead of at ingress-ASBR based on SAFI-4 withdrawal.

KV2> IF we were able to specify EPE-label in SAFI 1, then different SAFI 1 routes can carry different EPE-labels, based on ‘per-nexthop’ label allocation mode, allocating an EPE label per ECMP/FRR peer nexthop-set
KV2> that the service-route is pointing to. This model allows for a ‘Low-fib ASBR’, just another usecase.
KV2> So I think it is beneficial to be able to express service-labels for service-families (SAFI 1 also) without depending on a transport family.

[IM3] Sorry, but I definitely miss something here, because if we have a label in a per-next-hop manner attached to services routes of this next-hop, I cannot see the difference with having a label attached to the next-hop itself by LU.
Actually, I can’t see how MHN complies with some EPE cases, if we choose a single route from many peers at node X and send only it, we lean on the node decision process, having path attributes of the winner in the network. Path attributes of none-best routes are lost. Some of them (let’s say, some communities) may be used as input to the decision process of other routers. We can use local policies at ingress to select appropriate exit points, for example. But maybe the answer is “Don’t use MNH in this case”.
KV3> as clarified above, it is ‘use MNH with addpath’ from ASBR to IBGP-network/RR.
“Low-FIB ASBR” sounds strange, probably the idea is not to install service routes in FIB, populating LFIB only,
KV3> Yes. LU-EPE kind of provides basics of it, but doesn’t handle the ECMP/FRR scenarios well.
but still advertise them. Isn’t it possible with LU? I think it is.
KV3> Unfortunately not. Anecdotely I’ve learnt from people interested in this kind of functionality, that they havent been able to achieve it using existing mechanisms like LU transport.
KV3> (worth mentioning that they were not willing to overload SAFI-4 with service-routes)
KV3> Thanks. <EOM>


I also don't think that 4PE/6PE require any redistribution between AFIs, no one requires us to disseminate service routes as labeled ones, it is enough only for their next-hops. In other words, all the described cases are already solved by the LU.
KV> I suppose you are mentioning the LU-EPE (https://datatracker.ietf.org/doc/html/draft-gredler-idr-bgplu-epe-15<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-gredler-idr-bgplu-epe-15__;!!NEt6yMaO-gk!AqPF6zorBb_vXUUBej0ZWFFpJYB9l8Je5Us7dC0rR9HvOMKNL6c4pudWp9ePqJ0bIlZokbBy7Tueagxo7P__$>) way (nexthop-unchanged on SAFI 1 routes) of doing 6PE? I agree in that way, redistribution between AFs is not needed.
KV> If the same approach needs to be used with nexthop-self on SAFI 1 routes, it may need multiple loopbacks, one to advertise explicit-null, and another to advertise implicit-null
 [IM2] Yes, the idea behind my previous comment can be expressed in this draft (although, I don't like the way it's written).
KV2> I’d appreciate any input to make the LU-EPE draft better. We need to revive it and get it adopted as-well. Since it is deployed technology but the draft is still an individual draft.
[IM3] Got it, thank you.

Yes, to preserve a NH in this case we may need an additional loopback. I'm not sure that to overcome this "issue" we need to invent something really new. Maybe there should be more convincing use cases. :)
KV> But in mechanisms described in rfc4798 or in https://www.ietf.org/archive/id/draft-mishra-idr-v4-islands-v6-core-4pe-06.html#section-7.4<https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-mishra-idr-v4-islands-v6-core-4pe-06.html*section-7.4__;Iw!!NEt6yMaO-gk!AqPF6zorBb_vXUUBej0ZWFFpJYB9l8Je5Us7dC0rR9HvOMKNL6c4pudWp9ePqJ0bIlZokbBy7TueakmVBZIe$>, AFI redistribution is needed. SAFI 1  routes are redistributed into SAFI 4.
[IM2] RFC4798 requires us to send service routes via SAFI4, yes. But as we agreed above, it is not necessary, there are options. Moreover, vendors already give us all we need (Juniper is also).
Speaking about the latter document, Section 7 explicitly states that we actually can send service routes without any labels attached to them. I've spent much time discussing it with the authors because the original versions required the sending of service routes as labeled ones.

KV2> Yes that’s an improvement. Thanks for that. But specifying too many options without the actual tradeoffs clearly specified may also confuse users. Based on our experiences,
KV2> if we can atleast recommend ‘not redistributing between SAFIs’, that would be helpful. Especially because rfc4798 has been in widespread use so far.
KV> This dilutes the business logic of these AFs. Ability to carrying explicit-null in SAFI 1 using MNH helps with these scenarios, by not diluting SAFI 4 business logic to carry service routes. Thanks.
[IM2] So, having the alternative (the LU), I cannot agree with you on this part in the way, that we have to create something new to overcome the described problem.

KV2> Thanks for all your input. Just wanted to state my viewpoint above, on why having a way to carry service-label in the SAFI 1 service-family also can be beneficial, just like we carry label in SAFI 128 servce-family.

Thanks
Kaliraj







Juniper Business Use Only
From: Idr <[e-mail]<mailto:[e-mail]>> on behalf of Igor Malyushkin <[e-mail]<mailto:[e-mail]>>
Date: Wednesday, November 15, 2023 at 11:45 AM
To: Susan Hares <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Idr] WG adoption call - draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)
[External Email. Be cautious of content]

Hello folks,

I have some questions. First, it is unclear whether this draft applies to several paths and their next-hop addresses or a single path and its potential next-hop addresses. The text, especially in Section 3, refers to the Add-Paths mechanism as today's alternative, but Add-Paths allows us to propagate several paths without losing any attributes.
Second, the draft specifies that for unlabeled families there can be a labeled next-hop attached via the new attribute. I think that moment should be carefully considered by the WG. From my side, it is strange to advertise any labels to a receiver that does not expect that via families that are not about that.

Thank you!

пт, 10 нояб. 2023 г. в 13:19, Susan Hares <[e-mail]<mailto:[e-mail]>>:
This begins a WG adoption call for draft-kaliraj-idr-multinexthop-attribute-10.txt.

Each author should reply to this message with a message
that indicates whether you know of any IPR on this topic.

During your consideration,  please consider:


a.       Are there any errors or problems with this specification?

b.      Will this specification aid operational networks?

Cheerily, Sue Hares
_______________________________________________
Idr mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!EqDZ5Xgy6P1r1DqauJ3-G_fdixf-M42OMPDJ41fgj9c3Cgs3EuxCPdUgGTckEw2m2hSbX7P6jwCMDhXb1FPb$>