Re: [Idr] WG adoption call - draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)

[Igor Malyushkin <[e-mail]>]

Hi Kaliraj,

Thank you for the detailed comments, mine are below as usual (IM3).

пт, 24 нояб. 2023 г. в 23:43, Kaliraj Vairavakkalai <[e-mail]>:

> Hi Igor, please see my responses inline KV2>
>
>
>
> Thanks
>
> Kaliraj
>
> Juniper Business Use Only
>
> *From: *Igor Malyushkin <[e-mail]>
> *Date: *Friday, November 17, 2023 at 5:24 AM
> *To: *Kaliraj Vairavakkalai <[e-mail]>
> *Cc: *Susan Hares <[e-mail]>, [e-mail] <[e-mail]>
> *Subject: *Re: [Idr] WG adoption call -
> draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)
>
> *[External Email. Be cautious of content]*
>
>
>
> Hi Kaliraj,
>
>
>
> My responses are below ([IM2]).
>
>
>
>
>
> пт, 17 нояб. 2023 г. в 13:32, Kaliraj Vairavakkalai <[e-mail]>:
>
> Hi Igor,
>
>
>
> Please find some responses inline. KV>
>
>
>
> Thanks
>
> Kaliraj
>
>
>
> Juniper Business Use Only
>
> *From: *Igor Malyushkin <[e-mail]>
> *Date: *Wednesday, November 15, 2023 at 11:45 PM
> *To: *Kaliraj Vairavakkalai <[e-mail]>
> *Cc: *Susan Hares <[e-mail]>, [e-mail] <[e-mail]>
> *Subject: *Re: [Idr] WG adoption call -
> draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)
>
> *[External Email. Be cautious of content]*
>
>
>
> Hi Kaliraj,
>
> Thanks for your response! In general, I agree with the list above and
> personally find your work promising. I just want to clarify several things.
> Please, see my inline.
>
>
>
> чт, 16 нояб. 2023 г. в 05:14, Kaliraj Vairavakkalai <[e-mail]>:
>
> Hi Igor, thanks for your comments.
>
>
>
> This MNH draft applies to single path carrying an ordered set of one or
> more nexthops.
>
> [IM] That's about outgoing messages I believe. I asked about the sources
> of it. Let's say, we have two paths for the same destination: x.x.x.x/y
> {1;2;3} and x.x.x.x/y {5} from different peers (and with the different
> next-hops as well). A node selects the latter due to its shorter AS_PATH,
> would it be possible to propagate the route further with the MNH attribute
> and both next-hops?
>
>     KV> Good question. The node may readvertise the best path with a MNH
> containing the nexthop (EP1) of best path as active leg, and nexthop (EP2)
> of inactive path as backup leg. E.g.,
>
>         MNH-X:
>
>           +NFI-X <Primary, Num-Nexthops=2>
>
>                 +FI-X <Action=Push, RelativePref=1>
>
>                      -FA-U <Type=1, EP=EP1>
>
>                 +FI-Y <Action=Push, RelativePref=2>
>
>                      -FA-X <Type=1, EP=EP2>
>
>
>
>  [IM2] Ok, thank you, now I understand it better.
>
>
>
> Yes Addpath advertises multiple paths along with their attributes, but
> only if they have unique nexthops. So as I see it, the real use of Addpath
> is also to advertise multiple nexthops. But Addpath does not specify any
> relationship/order in the nexthops sent, and uses high RIB-out/in scale,
> because of using all attributes.
>
>
>
> Conceptually, if the sending BGP speaker is able to come up with “what
> kind of multiple-nexthop forwarding” is needed at the receiving nodes, then
> that can be conveyed in a single route update in a more expressive manner,
> instead of carrying all paths to receiver and further consuming CPU in
> multipath computation at all receiving-nodes. MNH just provides that
> expressiveness on the wire. How the sending speaker arrives at the
> forwarding-info to be sent in MNH (ECMP, Ordered fallback, WECMP, etc) -
> there can be more than one ways to figure that out, based on different
> usecases - as we are discovering. This just opens up a BGP based standard
> API to the box’s BGP RIB.
>
> [IM] Thanks for the additional clarification.
>
>
>
> And about the point on carrying labels in unlabled families: nodes not
> supporting MNH will just ignore it. Only those that have configured an AF
> to use MNH will use it. So I think there is no surprise element to any old
> BGP speakers receiving the label/MNH. If a receiving node that understands
> MNH does not understand enough of the contents to safely use it, they don’t
> use it (‘Attribute Discard’ approach). But I agree, the error handling will
> evolve as the draft matures.
>
> [IM] I'm worried about the approach when we define some new attributes and
> significantly change the very nature of an address family. Where does this
> road lead us? I consider an address family not only as an encoding
> container but also as some semantics behind it, and it looks like
> additional attributes can alter it in any possible way.
>
>    KV> I get your concern. I consider an address family as having a core
> business logic. And these encoding enhancements should not alter that
> business logic. They should just help in achieving that business logic
> better.
>
>    KV> IOW, encoding deficiencies should not come in the way of achieving
> the business logic better. I agree with you that the core business logic of
> an AF should not be diluted. But I also feel giving any AF a little more
> expressability
>
>    KV> should not hurt its business logic.
>
> [IM2] Well, we can already send prefixes for one AFI with next-hops from
> another. A more detailed text about the path resolution of unlabeled
> families with labeled next-hops would be helpful.
>
>
>
> KV2> Since that is already existing functionality, documenting that
> elsewhere may be good. may be the LU-EPE draft, which makes use of that
> functionality?
>

[IM3] Agree. And, as I understand the process, after the adoption you would
probably be asked to remove the rest use cases from the body.

Additionally, I'm curious about the process of withdrawing a label when a
> connected with it next-hop is alive it shouldn't be withdrawn.
>
>
>
> KV2> I’m presuming this is a general question, not referring to any
> specific text in the draft?
>

[IM3] Yes. I can’t express the exact problem at the moment. But it is clear
to me, that we don’t only have the possibility to advertise service routes
with labels now, but we also have some dynamic situations when a sender has
a reason to re-advertise us a bunch of routes because something has
happened with its underlying MPLS infrastructure, especially for Inter-AS
links. Even when someone advertises service routes via labeled unicast
(please, no), in this situation it leads to the withdrawal of these routes,
because they are tight to their labels. But maybe my understanding is wrong
and having a label attached to SAFI 1 by MNH makes a route non-decoupling
from its label, just like for SAFI 4 and 128. Although, the text says that
NHS nodes delete the attribute and only may attach it again, but not a
must. That's probably another side of this dynamic: some hops can be
labeled, some not.


> KV2> In my view, the node’s liveness and label-advertisement are
> decoupled. A label could signify a service-endpoint at a node or the node
> itself.
>
> KV2> So a ‘label’s association with a BGP FEC’ can be changed/withdrawn
> without the node actually going down.
>

[IM3] I agree, but it is all good only when the sender is LER. An LSR
depends also on some MPLS-transport which have a less stable nature than
service endpoints and even nodes themselves.


>
>
> KV2> Also, to set some baseline, stating my understanding: in BGP the
> (downstream allocated) label itself is not withdrawn by itself. It is not
> part of the key in NLRI. Only the FEC in NLRI (e.g. SAFI 128, SAFI 4) are
> withdrawn, and as a result
>
> KV2> the association with the previously associated label is removed. IOW,
> the (downstream allocated) label advertised in the 8277 NLRI is just used
> in the nexthop at receiving node.
>

[IM3] Agree. SAFI 4/128 gives us the ability to send an IP route and signal
LSP toward its next hop at the same time. A label inside an MNH attribute
can be attached to a route dynamically and anywhere.


> KV2> MNH carries the (downstream allocated) label in an attribute, which
> actually expresses this non-key aspect better.
>

[IM3] I'd say, it expresses it in a broader way, yes. But nothing prevents
us to attach an MNH for SAFI 4 instead and express whatever we want.


> KV2> Only a (upstream allocated) label advertised in AFI 16399 NLRI (MPLS
> Namespaces) can be explicitly withdrawn because in that case, it is part of
> the FEC/key in the NLRI.
>
> I feel the flexibility (carrying labels in unlabled families) itself is
> beneficial. Even today we have usecases (LU-EPE, 6PE, 4PE) where we want to
> impose labels on unlabeled service routes by resolving over labeled
> families. And some of these usecases (6PE, 4PE) require redistribution
> between Internet-families and Labeled-AFs which is risky. Using MNH allows
> safe walled gardens with consistent service, transport AF-layers, avoiding
> such redistribtions. And, I agree that interaction/precedence of the label
> carried in MNH with the label carried in other places on the route need to
> be specified.
>
> [IM] At this very moment, I can't see where this flexibility is justified.
> It more looks like the approach to get rid of the LU.
>
> KV> I feel having transport layer families (like LU) and resolving service
> families over them has the advantage of indirection and BGP PIC.
>
> KV> So I think we cannot get rid of LU as a transport-family, unless the
> indirection is provided by some other means.
>
>      KV> Its core business logic is to carry transport end-points.
>
>      KV> Having flexibility to carry label on service-routes does not mean
> we need to eliminate a transport layer family. it just allows additional
> expressability for AFs at each layer.
>
> [IM2] I agree with everything about the LU and for exactly this reason I
> want to better understand when the LU doesn't suit and we should allocate
> extra labels for service routes.
>
>
>
> KV2> Firstly, one thing I wanted to clarify: just because of carrying a
> label, the business logic of an AF is not altered. E.g. both SAFI 128 and
> SAFI 4 carry labels. But still they have distinct business logic,
>
> KV2> the former is a service-family and latter is a transport-family.
>

[IM3] Let’s take a closer look at your example with SAFI 1 vs. 4. One can,
intentionally oversimplifying, consider SAFI 4 as SAFI 128 for GRT (Global
Routing Table). And, to my bitter disappointment, lots of people actually
use SAFI 4 for service prefixes (for both families). We can see that the
industry decided to use v6 labeled unicast solely for spreading
reachability (notorious RFC4798) while at the same time, its brother is
used in both ways. Some vendors still consider v6 LU not the same way as v4
LU. Thus, we see some flexibility in business logic here depending on AFI
rather than SAFI. So, I cannot say that the SAFI 4 is a transport family
(I’d like to, but I just can’t). I think we will see the same for SAFI 1
and labels inside MNH, unfortunately. People will use it instead of LU
because modern marketing teaches them to use fewer protocols and
technologies.


> LU hasn’t ceased to exist, because SAFI-128 can carry a label. It is used
> in conjunction with SAFI-4. I expect the same thing to happen with SAFI 1
> and SAFI 4.
>

[IM3] The first part of this statement is truth. For the second part
(specifically, for SAFI 1, I have no questions for SAFI 4), I've already
expressed my view. Let's see.

KV2> I think the right question to ask is: in a certain usecase (e.g. 6PE,
> LU-EPE) whether we are carrying a service-label or a transport-label. It is
> possible that we have been carrying a service-label in a transport-family,
>
> KV2> just because there was no other way to carry it in service family so
> far. The explcit-null in 6PE is actually a service-label, it binds to the
> service-fec(IPv6). We just impose it by virtue of recursive resolution over
> a transport-family.
>
> KV2> That is the reason it forces us to use two loopbacks (a workaround)
> when we want to do nexthop-self and EPE-style 6PE simultaneously.
>


[IM3] To me, both cases are about the transport labels, not the service
ones. Having any reachability inside transport families is a compromise,
not a solution. So, the movement toward any labels attached to service
families is very concerning to me.

KV2> Further, taking the e.g. of EPE, there are circumstances, where the
> A/A or A/B relationship of two EPE peers is not expressable using a single
> LU-label, without making an assumption that the two peers give us exactly
> the same
>
> KV2> service-family routes at all times. We know this may not be true
> unless the two bgp-peer-session are on parallel-links between two nodes
> without any bgp policies, and even then momentarily they can get out of
> sync.
>
> KV2> I think such assumptions arise because the EPE label is actually a
> service-label. But we are imposing it by virtue of recursive resolution
> over a transport family.
>

[IM3] That’s interesting because from my side the whole idea behind EPE is
to have distinct labels for all possible peers. We attach a label to a
next-hop and we can’t have the same next-hop for different peers, thus we
have different labels. That makes these labels transport ones, not service.
Any synchronization of receiving routes from different peers comes into
play if we don't want to advertise all routes of these peers because of
MNH. Classical EPE doesn't require it. Is my understanding correct?


> KV2> IF we were able to specify EPE-label in SAFI 1, then different SAFI 1
> routes can carry different EPE-labels, based on ‘per-nexthop’ label
> allocation mode, allocating an EPE label per ECMP/FRR peer nexthop-set
>
> KV2> that the service-route is pointing to. This model allows for a
> ‘Low-fib ASBR’, just another usecase.
>
> KV2> So I think it is beneficial to be able to express service-labels for
> service-families (SAFI 1 also) without depending on a transport family.
>

[IM3] Sorry, but I definitely miss something here, because if we have a
label in a per-next-hop manner attached to services routes of this
next-hop, I cannot see the difference with having a label attached to the
next-hop itself by LU.

Actually, I can’t see how MHN complies with some EPE cases, if we choose a
single route from many peers at node X and send only it, we lean on the
node decision process, having path attributes of the winner in the network.
Path attributes of none-best routes are lost. Some of them (let’s say, some
communities) may be used as input to the decision process of other routers.
We can use local policies at ingress to select appropriate exit points, for
example. But maybe the answer is “Don’t use MNH in this case”.

“Low-FIB ASBR” sounds strange, probably the idea is not to install service
routes in FIB, populating LFIB only, but still advertise them. Isn’t it
possible with LU? I think it is.


> I also don't think that 4PE/6PE require any redistribution between AFIs,
> no one requires us to disseminate service routes as labeled ones, it is
> enough only for their next-hops. In other words, all the described cases
> are already solved by the LU.
>
> KV> I suppose you are mentioning the LU-EPE (
> https://datatracker.ietf.org/doc/html/draft-gredler-idr-bgplu-epe-15
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-gredler-idr-bgplu-epe-15__;!!NEt6yMaO-gk!AqPF6zorBb_vXUUBej0ZWFFpJYB9l8Je5Us7dC0rR9HvOMKNL6c4pudWp9ePqJ0bIlZokbBy7Tueagxo7P__$>)
> way (nexthop-unchanged on SAFI 1 routes) of doing 6PE? I agree in that way,
> redistribution between AFs is not needed.
>
> KV> If the same approach needs to be used with nexthop-self on SAFI 1
> routes, it may need multiple loopbacks, one to advertise explicit-null, and
> another to advertise implicit-null
>
>  [IM2] Yes, the idea behind my previous comment can be expressed in this
> draft (although, I don't like the way it's written).
>
> KV2> I’d appreciate any input to make the LU-EPE draft better. We need to
> revive it and get it adopted as-well. Since it is deployed technology but
> the draft is still an individual draft.
>
[IM3] Got it, thank you.

>
>
> Yes, to preserve a NH in this case we may need an additional loopback. I'm
> not sure that to overcome this "issue" we need to invent something really
> new. Maybe there should be more convincing use cases. :)
>
> KV> But in mechanisms described in rfc4798 or in
> https://www.ietf.org/archive/id/draft-mishra-idr-v4-islands-v6-core-4pe-06.html#section-7.4
> <https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-mishra-idr-v4-islands-v6-core-4pe-06.html*section-7.4__;Iw!!NEt6yMaO-gk!AqPF6zorBb_vXUUBej0ZWFFpJYB9l8Je5Us7dC0rR9HvOMKNL6c4pudWp9ePqJ0bIlZokbBy7TueakmVBZIe$>,
> AFI redistribution is needed. SAFI 1  routes are redistributed into SAFI 4.
>
> [IM2] RFC4798 requires us to send service routes via SAFI4, yes. But as we
> agreed above, it is not necessary, there are options. Moreover, vendors
> already give us all we need (Juniper is also).
>
> Speaking about the latter document, Section 7 explicitly states that we
> actually can send service routes without any labels attached to them. I've
> spent much time discussing it with the authors because the original
> versions required the sending of service routes as labeled ones.
>
>
>
> KV2> Yes that’s an improvement. Thanks for that. But specifying too many
> options without the actual tradeoffs clearly specified may also confuse
> users. Based on our experiences,
>
> KV2> if we can atleast recommend ‘not redistributing between SAFIs’, that
> would be helpful. Especially because rfc4798 has been in widespread use
> so far.
>
> KV> This dilutes the business logic of these AFs. Ability to carrying
> explicit-null in SAFI 1 using MNH helps with these scenarios, by not
> diluting SAFI 4 business logic to carry service routes. Thanks.
>
> [IM2] So, having the alternative (the LU), I cannot agree with you on this
> part in the way, that we have to create something new to overcome the
> described problem.
>
>
>
> KV2> Thanks for all your input. Just wanted to state my viewpoint above,
> on why having a way to carry service-label in the SAFI 1 service-family
> also can be beneficial, just like we carry label in SAFI 128 servce-family.
>
>
>
> Thanks
>
> Kaliraj
>
>
>
>
>
>
>
>
>
>
>
>
>
> Juniper Business Use Only
>
> *From: *Idr <[e-mail]> on behalf of Igor Malyushkin <
> [e-mail]>
> *Date: *Wednesday, November 15, 2023 at 11:45 AM
> *To: *Susan Hares <[e-mail]>
> *Cc: *[e-mail] <[e-mail]>
> *Subject: *Re: [Idr] WG adoption call -
> draft-kaliraj-idr-multinexthop-attribute-10 - (11/10/2023 to 11/24/2023)
>
> *[External Email. Be cautious of content]*
>
>
>
> Hello folks,
>
> I have some questions. First, it is unclear whether this draft applies to
> several paths and their next-hop addresses or a single path and its
> potential next-hop addresses. The text, especially in Section 3, refers to
> the Add-Paths mechanism as today's alternative, but Add-Paths allows us to
> propagate several paths without losing any attributes.
> Second, the draft specifies that for unlabeled families there can be a
> labeled next-hop attached via the new attribute. I think that moment should
> be carefully considered by the WG. From my side, it is strange to advertise
> any labels to a receiver that does not expect that via families that are
> not about that.
>
> Thank you!
>
>
>
> пт, 10 нояб. 2023 г. в 13:19, Susan Hares <[e-mail]>:
>
> This begins a WG adoption call for
> draft-kaliraj-idr-multinexthop-attribute-10.txt.
>
>
>
> Each author should reply to this message with a message
>
> that indicates whether you know of any IPR on this topic.
>
>
>
> During your consideration,  please consider:
>
>
>
> a.       Are there any errors or problems with this specification?
>
> b.      Will this specification aid operational networks?
>
>
>
> Cheerily, Sue Hares
>
> _______________________________________________
> Idr mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/idr
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!EqDZ5Xgy6P1r1DqauJ3-G_fdixf-M42OMPDJ41fgj9c3Cgs3EuxCPdUgGTckEw2m2hSbX7P6jwCMDhXb1FPb$>
>
>