Re: [Idr] I-D Action: draft-kaliraj-idr-multinexthop-attribute-01.txt

[Kaliraj Vairavakkalai <[e-mail]>]

Thanks Gyan, noted.

The below thread seems to be discussing about tie-breaker during path-selection of add-path received routes.
Yes MNH abstract para 2 mentions tie-breaking on Router-ID (or originator-id) just as an example, it could tie-break on something else too, like BGP nexthop.
About this thread – yes the edge-discriminator can be useful to distinguish between two “EBGP-routes advertised in add-path”, especially in nexthop-self scenarios.
But I am not clear on what diversity the two paths will provide, if the nexthops are the same (nexthop-self) for ipv4-unicast routes. Anyway, that may be a separate discussion.


There is also another section in MNH draft that mentions add-path interaction. (https://datatracker.ietf.org/doc/html/draft-kaliraj-idr-multinexthop-attribute-01#section-3.1.2 ).
That is about the following aspect of add-path:

https://datatracker.ietf.org/doc/html/draft-ietf-idr-add-paths-guidelines-08#section-2

  Diverse path: A BGP path associated with a different BGP next-hop and

   BGP router than some other set of paths. The BGP router associated
   with a path is inferred from the ORIGINATOR_ID attribute or, if there
   is none, the BGP Identifier of the peer that advertised the path.


So, the MNH draft suggests to consider the MNH-attribute also, when doing the “different BGP next-hop” decision, to provide path-diversity. This is inline to how implementations today consider “mpls-label” also besides bgp-nexthop, for a labeled-bgp-family, when doing add-path.

May be I should clarify this section with a reference to addpath-guidelines draft.

Thanks
Kaliraj
From: Gyan Mishra <[e-mail]>
Date: Saturday, July 3, 2021 at 1:56 PM
To: Kaliraj Vairavakkalai <[e-mail]>
Cc: Minto Jeyananth <[e-mail]>, Robert Raszuk <[e-mail]>, idr@ietf. org <[e-mail]>
Subject: Re: [Idr] I-D Action: draft-kaliraj-idr-multinexthop-attribute-01.txt
[External Email. Be cautious of content]

Hi Kaliraj

Responses in-line.

There was a discussion recently on RFC 7911 add path related to the tie breaker being missed in the RFC and possibly use path-id or Robert’s fast-connection-restore path possible revival edge discriminator attribute.

https://datatracker.ietf.org/doc/html/draft-pmohapat-idr-fast-conn-restore<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-pmohapat-idr-fast-conn-restore__;!!NEt6yMaO-gk!QVII_XLxGBlX69rN38hKLDLicegdhAE_AwlEABNNzEWJxo9VYk5qeMZgNTIBpvUe$>


https://mailarchive.ietf.org/arch/msg/idr/nyFgEq6e2XLBganie_gVG70UDyM/<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/idr/nyFgEq6e2XLBganie_gVG70UDyM/__;!!NEt6yMaO-gk!QVII_XLxGBlX69rN38hKLDLicegdhAE_AwlEABNNzEWJxo9VYk5qeMZgNZN3lqbU$>

As your draft in the abstract mentions add path tie breaker in the second paragraph may want to read through this thread.

Thank you

Gyan

On Sat, Jul 3, 2021 at 5:39 AM Kaliraj Vairavakkalai <[e-mail]<mailto:[e-mail]>> wrote:
Hi Gyan,

Thank you for the questions and comments. Apologies - I somehow missed these emails with your questions, hence the delay in response.

Please see my response inline.. KV>


From: Gyan Mishra <[e-mail]<mailto:[e-mail]>>
Date: Friday, June 11, 2021 at 7:27 PM
To: Robert Raszuk <[e-mail]<mailto:[e-mail]>>
Cc: Kaliraj Vairavakkalai <[e-mail]<mailto:[e-mail]>>, idr@ietf. org <[e-mail]<mailto:[e-mail]>>, Minto Jeyananth <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Idr] I-D Action: draft-kaliraj-idr-multinexthop-attribute-01.txt
[External Email. Be cautious of content]


Hi Kaliraj

I read through the draft and had a few questions.

This feature parallels add path but is different in the way it is deployed as it a BGP optional transitive path attribute as opposed to a add path capability code which is exchanged P2P between PE-RR peers in a controlled fashion.  In that manner the add paths is generally used within an AS for BGP PIC advertisement of backup paths as well as for BGP multipath load balancing within an AS.

The gap this draft fills is clear to be able to provide a more intelligent add path advertisement feature.  This also fills a very important gap of unequal cost load balancing as in a topology may not have equal cost cost metric for iBGP tie breaker, thus one path ends up being the best path even though multiple paths exist with add paths even with unique RD, which makes it difficult for iBGP load balancing to occur.

I support the development and progression of this draft as this is an import solution to a real world problem of VPN overlay iBGP unequal cost load balancing within the operator core as well as uniform predictable load balancing.

KV> Thank you.

MultiNextHop= MNH - referring to as

Is the intent to use within an AS and if so making the attribute non transitive I think  is a MUST as the next hop is changed for external peering the propagation of the multinexthop attribute is irrelevant.   I think you mentioned next-hop-self which is set on PE-RR peering at the domain ingress node can be used to stop the MNH from being propagated.  However, the next-hop-self is done in ingress inbound upon entering the domain and not leaving the domain so I think the MNH can still get propagated outside the domain.

KV> I had chosen ‘optional transitive’ to be able to pass thru RRs that don’t support it. But we also need to scope it within an AS and not allow it to pass ebgp boundary. And I agree with you the only way to do it for old-speakers is to make it ‘optional non-transitive’. I will correct that.
 Gyan> Perfect
Section 3.1.1 described interaction with top level mandatory transitive next hop attribute code 3 and the BGP Update MP_REACH next hop code 14.

Can you help explain section 3.1.1 interaction


   “When adding a MultiNexthop attribute to an advertised BGP route, the

   speaker MUST put the same next-hop address in the Advertising PNH

   field as it put in the Nexthop field inside NEXT_HOP attribute or

   MP_REACH_NLRI attribute.  Any speaker that recognizes this attribute

   and changes the PNH while re-advertising the route MUST remove the

   MultiNexthop-Attribute in the re-advertisement.  The speaker MAY

   however add a new MultiNexthop-Attribute to the re-advertisement;

   while doing so the speaker MUST record in the "Advertising-PNH" field

   the same next-hop address as used in NEXT_HOP field or MP_REACH_NLRI

   attribute.”






What is PNH?  primary next hop ??

KV> “Protocol Next Hop”. Nexthop ip-address carried in the attr 3, 14.
 Gyan> Ack
So the way I read this is the optional transitive MNH attribute is copied into code 3 and 14.  So if there are 10 NH in the PNH they all get written into top level code 3 and BGP update code 14.

KV> no. only the following advertising-pnh field has the same address as the attr 3, 14 nexthop field when advertising with nexthop-self.
 Gyan> Understood
   Advertising PNH     IPv4 or IPv6 PNH-address (Len = 32 or 128)
                          advertised in NEXT_HOP or MP_REACH_NLRI attr.
                          Used to sanity-check this attribute

KV> The formatting in 01 version of the draft is messed up, I’ll correct it. Pls refer to the 00 version which has proper formatting.

Since this is for “BGP free core” the P routers where Label swap forwarding semantics happens is not applicable as we don’t run BGP.  As the forwarding semantics is not applicable after the VPN label “push” label imposition CE to PE eBGP peering entering the core or “pop and forward” label disposition PE to CE eBGP peering leaving the core.
All the next hops have a VPN label imposed which is the next-hop-self from each PE, which I think would have only the “forwarding” semantics applicable, so I think that is the only one that would apply for MNH iBGP load balancing.

KV> For vpn-family routes we could continue to carry the label in rfc8277 bgp-nlri. And use MNH with 3.4.1.
        If label is used in both rfc8277 bgp-nlri and the MNH 3.4.2, then it would mean the bgp-nlri imposed mpls-label will be pushed on top of the 3.4.2 specified label.
        The main use of 3.4.2 is to be used with inet-unicast routes. That will equate inet-unicast routes with inet-vpn/inet-lu routes. Because using this now we can bind a mpls-label to a inet-unicast service route also.
        So that all these service families are treated equal. This helps in doing option-B for internet routes as-well, avoiding service-route scale on ASBR BNs.
        And yes, ECMP/PIC/WECMP will happen based on information received in “Nexthop Forwarding Semantic TLV”. Thanks.
 Gyan> Ack.  As another possible use case with RSVP-TE could be the per VRF to TE mapping feature which required multiple loopbacks for the next hop rewrite for steering over different next hop paths which is not scalable.  SR filled the gap nicely with SR-TE per VRF steering and per micro and macro flow steering capabilities.  This MNH draft could make per VRF mapping to TE tunnel more scalable as it now does not require a discrete loopback per next hop and static routing over the tunnel making the solution more scalable.  Also with MNH RSVP-TE maybe able to provide SR-TE like per flow traffic steering.  This could help bridge the gap further for operators not able to migrate yet to SR they can now reap some of the SR steering benefits and can extend the life of RSVP-TE or decide to stay with RSVP-TE.
Have you thought about how this draft can work together or  complement the CT draft?

   Also maybe how this draft for load balancing use cases can work with BGP link bandwidth extended community and other load balancing drafts below.

https://datatracker.ietf.org/doc/html/draft-ietf-idr-link-bandwidth-07<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-idr-link-bandwidth-07__;!!NEt6yMaO-gk!QVII_XLxGBlX69rN38hKLDLicegdhAE_AwlEABNNzEWJxo9VYk5qeMZgNX2D8fnY$>

https://datatracker.ietf.org/doc/html/draft-mohanty-bess-ebgp-dmz-03<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-mohanty-bess-ebgp-dmz-03__;!!NEt6yMaO-gk!QVII_XLxGBlX69rN38hKLDLicegdhAE_AwlEABNNzEWJxo9VYk5qeMZgNedxy3FA$>

https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-unequal-lb<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-unequal-lb__;!!NEt6yMaO-gk!QVII_XLxGBlX69rN38hKLDLicegdhAE_AwlEABNNzEWJxo9VYk5qeMZgNbsQ2ON4$>

Kind Regards

Gyan

On Fri, Jun 11, 2021 at 9:10 AM Robert Raszuk <[e-mail]<mailto:[e-mail]>> wrote:
HI

Two more questions ...

8. I am not clear what are trying to describe in Section 8.


   Like any other optional transitive BGP attribute, it is possible that

   this attribute gets propagated thru speakers that don't understand

   this attribute and an error detected by a speaker multiple hops away.

   This is mitigated by requiring the receiving speaker to remove this

   attribute when doing nexthop-self.

First indeed, the attribute may be propagated by BGP speakers which do not understand it, but that in itself is not an error. In those cases partial bit is set but attribute is still valid.

This is also completely orthogonal to setting the next hop self on the path when propagating for example across EBGP.


9. You are providing a lot of analogy to Add-Paths. But in Add-Paths thx to capability negotiation it is mandatory for receiving speaker indicating support to act on it. Here you have chosen for some reason blind propagation as optional transitive which perhaps may be ok for networks which do end to end encapsulation, but I don't think it is going to work in pure IPv4/IPv6 hop by hop lookup - especially in the cases of mixed network elements some supporting the attribute and some not.

Thx,
R.





On Fri, Jun 11, 2021 at 12:09 PM Robert Raszuk <[e-mail]<mailto:[e-mail]>> wrote:
Dear authors,

I have read yr draft with interest as some perhaps recall we have discussed this topic in the past number of times.

Cosmetics:

1. First nit - why do you say label must be 3107bis label ? (3.4.2.  Labeled IP nexthop) MPLS label is a label and I am not sure how one method of label distribution matter here.

2a. What is PNH ? It first occurs in section 3 as "PNH-Len" or "PNH-address", but it is never explained in the draft. Is this Path Next Hop ?

2b. Would it be better to call this new attribute MNH MultiNextHop ?

Tech:

3. In section 3.1.1 you describe how to assure that NH in MP_REACH is also part of MultiNext Hop Attribute. But I do not see any discussion on how to treat or ignore next hops in MP_REACH when a new attribute is present and is valid.

4. In section 3.1.2 you define behaviour of RR advertising paths from non MultiNexthop paths and those which carry new attributes ... But you should make it clear that this is only about 9.1.2.2 step in best path selection (or candidate selection). There can be other criteria before we even get to that step.

5. Now the most important question - how do you plan to handle atomic withdraws ? I assume the plan is to readvertise the path with MNH - the removed next hop. So by implicit withdraw this next hop will be removed. The draft is silent on this. Now if the removed next hop was selected by some receivers as best (due to 9.1.2.2) and it was not arriving as part of MP_REACH this proposal require significant implementation changes on how BGP best path selection is triggered, how it runs, how it populates results to the RIB/FIB. I think a new section is needed in detailed discussing the withdraws.

6. How do you envision max-prefix safety knobs to work here ? On the surface it may seem orthogonal - but it is not. Today folks use this to protect infrastructure from for example operator's mistakes. Here one received path may fill the MNH attribute with 100s of next hops and as being optional and transitive will be distributed to all routers all over the world.

7. Observe that metric to next hops is dynamic. So some implementations capable of next hop tracking register with RIB all next hops and each time metric changes they get a call back. Here we are effectively talking about exploding this 10x or 100x or more ...

Many thx,
Robert


---------- Forwarded message ---------
From: <[e-mail]<mailto:[e-mail]>>
Date: Fri, Jun 11, 2021 at 10:56 AM
Subject: I-D Action: draft-kaliraj-idr-multinexthop-attribute-01.txt
To: <[e-mail]<mailto:[e-mail]>>



A New Internet-Draft is available from the on-line Internet-Drafts directories.


        Title           : BGP MultiNexthop attribute
        Authors         : Kaliraj Vairavakkalai
                          Minto Jeyananth
        Filename        : draft-kaliraj-idr-multinexthop-attribute-01.txt
        Pages           : 12
        Date            : 2021-06-11

Abstract:
   Today, a BGP speaker can advertise one nexthop for a set of NLRIs in
   an Update.  This nexthop can be encoded in either the BGP-Nexthop
   attribute (code 3), or inside the MP_REACH attribute (code 14).

   For cases where multiple nexthops need to be advertised, BGP-Addpath
   is used.  Though Addpath allows basic ability to advertise multiple-
   nexthops, it does not allow the sender to specify desired
   relationship between the multiple nexthops being advertised e.g.,
   relative-preference, type of load-balancing.  These are local
   decisions at the receiving speaker based on path-selection between
   the various additional-paths, which may tie-break on some arbitrary
   step like Router-Id.

   Some scenarios with a BGP-free core may benefit from having a
   mechanism, where egress-node can signal multiple-nexthops along with
   their relationship to ingress nodes.  This document defines a new BGP
   attribute "MultiNexthop" that can be used for this purpose.

   This attribute can be used for both labeled and unlabled BGP
   families.  For labeled-families, it is used for a different purpose
   in "downstream allocation" case than "upstream allocation" scenarios.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-kaliraj-idr-multinexthop-attribute/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-kaliraj-idr-multinexthop-attribute/__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfRLDTqKGj$>

There is also an htmlized version available at:
https://datatracker.ietf.org/doc/html/draft-kaliraj-idr-multinexthop-attribute-01<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-kaliraj-idr-multinexthop-attribute-01__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfRLYOeXy-$>

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-kaliraj-idr-multinexthop-attribute-01<https://urldefense.com/v3/__https:/www.ietf.org/rfcdiff?url2=draft-kaliraj-idr-multinexthop-attribute-01__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfRIE_FnU0$>


Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/<https://urldefense.com/v3/__ftp:/ftp.ietf.org/internet-drafts/__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfROJyomj_$>


_______________________________________________
I-D-Announce mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/i-d-announce<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/i-d-announce__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfREhLtYSR$>
Internet-Draft directories: http://www.ietf.org/shadow.html<https://urldefense.com/v3/__http:/www.ietf.org/shadow.html__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfRNd45oQu$>
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt<https://urldefense.com/v3/__ftp:/ftp.ietf.org/ietf/1shadow-sites.txt__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfROCmEIB8$>
_______________________________________________
Idr mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfRAjWelEr$>
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<https://urldefense.com/v3/__http:/www.verizon.com/__;!!NEt6yMaO-gk!Tp_AgObe4LEZZKd8evxc9YHtDdlil74QLaAsjA-TqTnhbmWFCxhJPUlfRJoHyeCI$>

Gyan Mishra

Network Solutions Architect

Email [e-mail]<mailto:[e-mail]>

M 301 502-1347



Juniper Business Use Only
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<https://urldefense.com/v3/__http:/www.verizon.com/__;!!NEt6yMaO-gk!QVII_XLxGBlX69rN38hKLDLicegdhAE_AwlEABNNzEWJxo9VYk5qeMZgNed4hVOs$>

Gyan Mishra

Network Solutions Architect

Email [e-mail]<mailto:[e-mail]>

M 301 502-1347



Juniper Business Use Only