Re: [Idr] I-D Action: draft-ietf-idr-bgp-fwd-rr-02.txt

[Igor Malyushkin <[e-mail]>]

Robert,

Please, see my inline (IM).

пн, 18 мар. 2024 г. в 18:42, Robert Raszuk <[e-mail]>:

> Igor,
>
> When you are doing next hop self you are setting it to a local loopback
> address. That is your connected route. That next hop will be accessible to
> peering ABR via implicit or explicit null.
>
[IM] It's not necessary to advertise our loopback address as a labeled
route or even by BGP (we can disseminate it via IGP). When we receive a BGP
LU route there must be an unicast route to resolve its NH and an LSP toward
this NH to resolve a label. We may even not install the received route in a
unicast routing table, leaving it only for label forwarding and
auxiliary tables (for further NH resolving). What matters here is that a
remote node (another ABR too) for traffic forwarding will use a label we
allocated. It has nothing in common with our loopback address, its
potential label, and so on.

>
> You are talking about label carried in NLRI (say for SAFI 4). Yes each ABR
> with NHS will allocate a new value and advertise it to all peers and for
> data plane act as LSR.
>

> And as we established those are/would be per prefix labels.
>
> So when on primary ABR my routes + labels go away to region-2 and as
> backup I need to use routes and labels via peering/backup ABR this is per
> prefix adjustment. There is no PIC.
>
[IM] As I said I do not insist on that terminology. Let it be a per-label
protection. But it is a precomputed element right in the node's FIB, which
makes it a fast solution independent of the number of routes. Probably, we
should differentiate failure scenarios here. If we have an interface
failure at ABR, we can switch labels in the affected NHLFEs. If there is a
failure deeper in the network (close to downstream), we can only lean on
the BGP convergence.

>
> The fact that peering ABR set next hop self does not change anything here
> in terms of PIC for two reasons:
>
> a) we do not know which prefixes and labels going to region-2 got affected
> till we run best path on them
>
[IM] I do not understand this point. With the per-prefix label allocation
mode, we know it exactly in advance. This mode's name implies that.

> b) for each prefix received from back ABR label may be different.
>
[IM] Sure, the per-prefix mode tends to be greedy in terms of resources. It
is a problem for Option B, but almost never a problem for Option C.

>
> So to reiterate I am saying that ABR doing local repair can not do any
> repair in prefix independent fashion as there is no single trigger from
> region-2 and there is no single switchover to backup ABR possible (each
> route may have a different per prefix label).
>
[IM] Well, I disagree.

>
> Thx,
> R.
>
>
>
>
> On Mon, Mar 18, 2024 at 3:15 PM Igor Malyushkin <[e-mail]>
> wrote:
>
>> On the contrary, I'm considering the next-hop-self operation at both
>> ABRs. After receiving a labeled route we can't propagate it any further
>> without changing its next hop if we also allocate a new label for its
>> label.  So, we either do both (NHS and SWAP) or nothing (pure RR). This is
>> an LSR mode. We also can put the received label in the FTN table, which
>> makes us an ingress LER (it is useful when an ABR is also a PE). If we
>> receive an implicit null label, we cannot do a SWAP, of course, only POP.
>>
>> For the rest, I agree. But isn't it the reason this document was born?
>>
>> пн, 18 мар. 2024 г. в 18:00, Robert Raszuk <[e-mail]>:
>>
>>> Igor,
>>>
>>> I think you are forgetting that those ABRs set next hop self. When you
>>> do that there is no more label SWAP operation on transport label. The only
>>> choice you have to do PHP or not on penultimate hop.
>>>
>>> Thinking more about this entire model there is one more serious concern.
>>>
>>> Because ABRs set next hop self metric of the link between such ABRs MUST
>>> be higher then any NH cumulative metric in region-2. If not even under
>>> steady state ABR will select as best peer's ABR paths as this is all IBGP
>>> here and with IBGP it is not unusual to get down to next hop metric tie
>>> break in BGP best path selection.
>>>
>>> The design complexity here with different CLUSTER_IDs  on ABRs acting as
>>> RRs (mutual clients) and Next-Hop-Self is really high and
>>> requires very careful planning.
>>>
>>> Cheers,
>>> R.
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Mar 18, 2024 at 1:02 PM Igor Malyushkin <[e-mail]>
>>> wrote:
>>>
>>>> I see this differently. ABR1 has two sets of the same
>>>> infrastructure paths. One set is from original sources (outside his left
>>>> area), and another is from ABR2. Imagine, that the first set is the best
>>>> one, then ABR1 allocates a label for every prefix (and its label) from the
>>>> set and distributes them as transport prefixes toward his right area (and
>>>> to ABR2 too). Effectively, it makes ABR1 an LSR because it performs the
>>>> SWAP for any incoming label. With the per-prefix label allocation mode, it
>>>> is possible to compile these SWAPs with more than one outgoing label.
>>>> Considering, we have the second set of the same paths from ABR2, we can use
>>>> his labels as a backup. So there is a PIC egress for such labels.
>>>>
>>>> Maybe I confused you because I didn't mention labels instead of routes.
>>>> My bad.
>>>>
>>>> To the authors,
>>>>
>>>> AS2 is further divided into two regions. There are three tunnel domains
>>>> in provider's network: The two regions in AS1 use RSVP intra-domain tunnel.
>>>> AS2 also uses RSVP-TE intra-domain tunnels. MPLS forwarding is used within
>>>> these domains and on inter-domain links. BGP LU (AFI/SAFI: 1/4) is the
>>>> transport family providing reachability between PE loopbacks PE25 and
>>>> PE11.
>>>>
>>>> I see here a subtle mistake. There are no two regions in AS1 that can
>>>> use RSVP LSPs, probably it is AS2.
>>>>
>>>> пн, 18 мар. 2024 г. в 15:21, Robert Raszuk <[e-mail]>:
>>>>
>>>>> Hi Igor,
>>>>>
>>>>> On Mon, Mar 18, 2024 at 12:13 PM Igor Malyushkin <
>>>>> [e-mail]> wrote:
>>>>>
>>>>>> Well, maybe there is some gap in terminology. I always considered
>>>>>> this behavior as a PIC, because we can switch between the next hops without
>>>>>> any dependency on the number of prefixes above. An egress characteristic
>>>>>> here is that it happens on a failed next-hop node (an ingress is not aware
>>>>>> at the moment or is just starting to react). But we can find a better name
>>>>>> for this to avoid confusion.
>>>>>>
>>>>>
>>>>> I disagree.
>>>>>
>>>>> If you zoom into this specific scenario the described situation is
>>>>> that say ABR1 looses (all or some)  IBGP sessions outside his left area.
>>>>> Within those session(s) he may have gotten lots of infrastructure routes
>>>>> with lots of next hops.
>>>>>
>>>>> So here it needs to run best path and install all routes one by one
>>>>> into RIB and FIB now pointing towards a peer ABR2.
>>>>>
>>>>> There is no prefix independence here at all. There is no signalling in
>>>>> neither IGP nor BGP that one next hop is lost and we need to use the other
>>>>> one. That would be possible only on PEs not on ABRs.
>>>>>
>>>>> So while it is some sort of local protection it is not PIC.
>>>>>
>>>>> Regards,
>>>>> R.
>>>>>
>>>>>
>>>>>
>>>>>> Speaking about the propagation of withdraws. As I've previously
>>>>>> mentioned, traffic may be sent slightly before (a few milliseconds) or just
>>>>>> in time of a failure. Without "protection" at egress, it will be lost if
>>>>>> ABRs do not exchange their routes (e.g., because of the same CLUSTER ID).
>>>>>> Another moment to consider is that the fast propagation not only
>>>>>> depends on the diameter of the BGP network (the number of BGP hops from a
>>>>>> source of the event to all its potential receivers) but also on the
>>>>>> situation on every such hop (e.g., CPU spikes). In other words, it is not
>>>>>> constant.
>>>>>>
>>>>>> пн, 18 мар. 2024 г. в 14:54, Robert Raszuk <[e-mail]>:
>>>>>>
>>>>>>> > the egress PIC
>>>>>>>
>>>>>>> Except this is not real egress PIC.
>>>>>>>
>>>>>>> In egress PIC ASBRs or PEs receive EBGP paths and rarely act as RRs.
>>>>>>>
>>>>>>> Here we seem to have a case of option C and IBGP domain where ABRs
>>>>>>> are usually redundantly connected and they learn routes over IBGP from each
>>>>>>> site.
>>>>>>>
>>>>>>> I must admit that I have never seen a real practical analysis if in
>>>>>>> such cases we should be doing PIC between ABRs acting as RRs. Especially
>>>>>>> for infrastructure routes.
>>>>>>>
>>>>>>> And btw propagating withdraws via good RRs last time I measured was
>>>>>>> taking at most single milliseconds.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> R.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 18, 2024 at 11:24 AM Igor Malyushkin <
>>>>>>> [e-mail]> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> AFAIK, the egress PIC is a widely deployed feature with
>>>>>>>> labeled paths. One of its characteristics is to preserve traffic in-flight,
>>>>>>>> that was sent just in time of a failure event or slightly after that.
>>>>>>>> Traffic is almost always faster than any control plane stuff. The
>>>>>>>> significant problem with PIC in this case is a possible temporal loop if a
>>>>>>>> destination node fails, but it is a separate topic.
>>>>>>>>
>>>>>>>> My 2 cents.
>>>>>>>>
>>>>>>>> пн, 18 мар. 2024 г. в 08:40, Kaliraj Vairavakkalai <kaliraj=
>>>>>>>> [e-mail]>:
>>>>>>>>
>>>>>>>>> Hi Robert, please see inline. KV>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> Kaliraj
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Juniper Business Use Only
>>>>>>>>>
>>>>>>>>> *From: *Robert Raszuk <[e-mail]>
>>>>>>>>> *Date: *Sunday, March 17, 2024 at 11:28 PM
>>>>>>>>> *To: *Kaliraj Vairavakkalai <[e-mail]>
>>>>>>>>> *Cc: *idr@ietf. org <[e-mail]>
>>>>>>>>> *Subject: *Fwd: I-D Action: draft-ietf-idr-bgp-fwd-rr-02.txt
>>>>>>>>>
>>>>>>>>> *[External Email. Be cautious of content]*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Kaliraj,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thx for posting the new version.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I have one observation or clarification to be made in respect to
>>>>>>>>> text you added in section 4.1:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> > However this approach does not allow the ABR-ABR tunnels to be
>>>>>>>>>
>>>>>>>>> > used as backup path, in the event where an ABR looses all
>>>>>>>>> tunnels
>>>>>>>>>
>>>>>>>>> > to upstream ASBR.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So you are talking about the delta time it takes for ABR which
>>>>>>>>> looses all tunnels to upstream ASBRs to send BGP withdraws for those
>>>>>>>>> learned infrastructure routes - correct ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> KV> Yes. Those withdrawals need to anyway happen, and reach both
>>>>>>>>> the ingress PEs and adjoining/redundant ABR.
>>>>>>>>>
>>>>>>>>> KV> So that they can do BGP PIC repair based on that event.
>>>>>>>>>
>>>>>>>>> KV> Here I am saying that such BGP PIC repair can happen only at
>>>>>>>>> ingress PE
>>>>>>>>>
>>>>>>>>> KV> (which may be multiple BGP hops away), and not at the
>>>>>>>>> adjoining ABR.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So we are talking 10s of milliseconds here from the moment all
>>>>>>>>> such paths are invalidated (which  -the detection and invalidation is
>>>>>>>>> needed in any scenario).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> KV> The BGP update propagation can take longer, based on load on
>>>>>>>>> the BGP propagation path. But BGP PIC itself can’t always
>>>>>>>>>
>>>>>>>>> KV> guarantee 10s of ms restoration. It only guarantees restoring
>>>>>>>>> the traffic without depending on service-prefix scale
>>>>>>>>>
>>>>>>>>> KV> once the unreachability is detected (in this case: BGP
>>>>>>>>> withdrawal is received).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> As you have established each ABR will set next hop self and
>>>>>>>>> advertise routes to local PEs (directly or via yet one more pair of RRs
>>>>>>>>> (RR26 here)). So each PE will already have backup paths all what you are
>>>>>>>>> observing here is the time before PEs invalidate paths advertised by ASBR
>>>>>>>>> which looses upstream tunnels.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> KV> Agreed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So if such failure models are really likely to happen (in spite of
>>>>>>>>> redundant ABR connectivity in each area)  I would rather focus on fast
>>>>>>>>> removal of broken paths from the network with one next hop invalidation
>>>>>>>>> (single BGP or IGP message, single RIB to FIB switchover on PEs) etc ...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> KV> As explained above, that would also happen. But it may take
>>>>>>>>> longer than if the repair happened at the ABR, which is closer to the
>>>>>>>>> failure event.
>>>>>>>>>
>>>>>>>>> KV> Just a tradeoff to be aware of. Thx.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thx,
>>>>>>>>> Robert
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>> From: <[e-mail]>
>>>>>>>>> Date: Sun, Mar 17, 2024 at 6:42 AM
>>>>>>>>> Subject: I-D Action: draft-ietf-idr-bgp-fwd-rr-02.txt
>>>>>>>>> To: <[e-mail]>
>>>>>>>>> Cc: <[e-mail]>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Internet-Draft draft-ietf-idr-bgp-fwd-rr-02.txt is now available.
>>>>>>>>> It is a work
>>>>>>>>> item of the Inter-Domain Routing (IDR) WG of the IETF.
>>>>>>>>>
>>>>>>>>>    Title:   BGP Route Reflector with Next Hop Self
>>>>>>>>>    Authors: Kaliraj Vairavakkalai
>>>>>>>>>             Natrajan Venkataraman
>>>>>>>>>    Name:    draft-ietf-idr-bgp-fwd-rr-02.txt
>>>>>>>>>    Pages:   9
>>>>>>>>>    Dates:   2024-03-16
>>>>>>>>>
>>>>>>>>> Abstract:
>>>>>>>>>
>>>>>>>>>    The procedures in BGP Route Reflection (RR) spec RFC4456
>>>>>>>>> primarily
>>>>>>>>>    deal with scenarios where the RR is reflecting BGP routes with
>>>>>>>>> next
>>>>>>>>>    hop unchanged.  In some deployments like Inter-AS Option C
>>>>>>>>>    (Section 10, RFC4364), the ABRs may perform RR functionality
>>>>>>>>> with
>>>>>>>>>    nexthop set to self.  If adequate precautions are not taken, the
>>>>>>>>>    RFC4456 procedures can result in traffic forwarding loop in such
>>>>>>>>>    deployments.
>>>>>>>>>
>>>>>>>>>    This document illustrates one such looping scenario, and
>>>>>>>>> specifies
>>>>>>>>>    approaches to minimize possiblity of traffic forwarding loop in
>>>>>>>>> such
>>>>>>>>>    deployments.  An example with Inter-AS Option C (Section 10,
>>>>>>>>> RFC4364)
>>>>>>>>>    deployment is used, where RR with next hop self is used at
>>>>>>>>> redundant
>>>>>>>>>    ABRs when they re-advertise BGP transport family routes between
>>>>>>>>>    multiple IGP domains.
>>>>>>>>>
>>>>>>>>> The IETF datatracker status page for this Internet-Draft is:
>>>>>>>>> https://datatracker.ietf.org/doc/draft-ietf-idr-bgp-fwd-rr/
>>>>>>>>> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-idr-bgp-fwd-rr/__;!!NEt6yMaO-gk!Hv7GNYr6n89i4QRD_aXV0QhV0N_J6YWRal9RjghMoB6DdmitfkQrjPi8YKCDbwPDc6YiEq2NYiMTgzkj$>
>>>>>>>>>
>>>>>>>>> There is also an HTMLized version available at:
>>>>>>>>> https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-fwd-rr-02
>>>>>>>>> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-fwd-rr-02__;!!NEt6yMaO-gk!Hv7GNYr6n89i4QRD_aXV0QhV0N_J6YWRal9RjghMoB6DdmitfkQrjPi8YKCDbwPDc6YiEq2NYmrSzwvH$>
>>>>>>>>>
>>>>>>>>> A diff from the previous version is available at:
>>>>>>>>>
>>>>>>>>> https://author-tools.ietf.org/iddiff?url2=draft-ietf-idr-bgp-fwd-rr-02
>>>>>>>>> <https://urldefense.com/v3/__https:/author-tools.ietf.org/iddiff?url2=draft-ietf-idr-bgp-fwd-rr-02__;!!NEt6yMaO-gk!Hv7GNYr6n89i4QRD_aXV0QhV0N_J6YWRal9RjghMoB6DdmitfkQrjPi8YKCDbwPDc6YiEq2NYmTSTQwU$>
>>>>>>>>>
>>>>>>>>> Internet-Drafts are also available by rsync at:
>>>>>>>>> rsync.ietf.org::internet-drafts
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> I-D-Announce mailing list
>>>>>>>>> [e-mail]
>>>>>>>>> https://www.ietf.org/mailman/listinfo/i-d-announce
>>>>>>>>> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/i-d-announce__;!!NEt6yMaO-gk!Hv7GNYr6n89i4QRD_aXV0QhV0N_J6YWRal9RjghMoB6DdmitfkQrjPi8YKCDbwPDc6YiEq2NYmzhCeS9$>
>>>>>>>>> _______________________________________________
>>>>>>>>> Idr mailing list
>>>>>>>>> [e-mail]
>>>>>>>>> https://www.ietf.org/mailman/listinfo/idr
>>>>>>>>>
>>>>>>>>