Re: [mpls] MPLS-RT review of draft-chandra-mpls-ri-rsvp-frr

[Chandrasekar Ramachandran <[e-mail]>]

Hi Mustapha,
Apologies for the delay. I will be prompt for subsequent mails on this thread. In my previous mail, I categorized your concerns into two.
(A) The need for introducing refresh-interval independent behavior into RSVP-TE
(B) Why some kind of local implementation based timers will not be sufficient to support long refresh intervals

Your mail sent in January seems to indicate you are either not convinced on the need for introducing refresh-interval independent behavior, or you are not convinced that the extensions proposed in this draft (or the ones proposed in [draft-ietf-teas-rsvp-te-scaling-rec]) actually make RSVP-TE FRR refresh-interval independent.

Here is my summary response to your points. I have also responded to your comments separately inline.
- The goal of the draft is not to reduce the number of triggered messages between PLR and MP when the protected LSP is being locally repaired. The draft addresses a specific gap to make RSVP-TE refresh-interval independent as documented in [draft-ietf-teas-rsvp-te-scaling-rec]. The key point is that if RSVP-TE routers are not constrained by short refresh timeouts, then the existing RFC 2961 mechanisms, plus some minor additional mechanisms as recommended in [draft-ietf-teas-rsvp-te-scaling-rec] should be sufficient to handle large LSP scale in all cases (except FRR that [draft-chandra-mpls-ri-rsvp-frr] addresses). It should be noted that while reducing the number of triggered backup LSP messages would be additional performance improvement, the goal of RI-RSVP is more generic and also encompasses make-before-break. Hence, the scope of this draft is not limited reducing triggered messages as specified in draft-mtaillon-mpls-summary-frr-rsvpte.
- It should be noted that [draft-ietf-teas-rsvp-te-scaling-rec] does make a specific recommendation that would address the limitation you have pointed out with Message-ID-ACK mechanism. The problem you have brought up is related to the lack of flow control between the RSVP-TE neighbors and Section 2.3 of [draft-ietf-teas-rsvp-te-scaling-rec] does address this limitation. If a router sends some messages that are unacknowledged, then the router can use that as an indication to reduce message rate to that neighbor. In fact, RFC 2961 message-ID-ACK mechanism is the key tool in achieving flow control between RSVP-TE neighbors. Hence, the mechanism proposed in this draft if considered along with [draft-ietf-teas-rsvp-te-scaling-rec] does make RSVP-TE refresh interval independent. Do you have specific cases in mind that are not covered by these drafts?
- The use of implementation specific timers for scaled networks have been problematic in production networks because these timers are ad-hoc and do not provide predictability to the behavior when the deployed scale increase over time. Moreover, your point assumes the network uses short refresh interval so that lack of reliable PathTear does not result in unacceptable level of stale state build up on the routers. In fact, the only two new aspects introduced in this draft leverage PathTear messages (with some minor extension) to reliably remove state from downstream router(s). Could you specifically point out the mechanism(s) in the draft that is/are complex?

Please refer inline for more detailed and specific responses.

Thanks,
Chandra.

> -----Original Message-----
> From: AISSAOUI, Mustapha (Mustapha) [mailto:[e-mail]]
> Sent: Saturday, January 16, 2016 10:00 AM
> To: Chandrasekar Ramachandran <[e-mail]>; Sriganesh Kini
> <[e-mail]>; Loa Andersson <[e-mail]>; Lucy yong
> <[e-mail]>
> Cc: [e-mail]; [e-mail]; draft-chandra-mpls-ri-rsvp-
> [e-mail]; Guijuan Wang 3 <[e-mail]>; Lizhong Jin
> <[e-mail]>
> Subject: RE: MPLS-RT review of draft-chandra-mpls-ri-rsvp-frr
> 
> Hi Chandra,
> Sorry for the late reply. See inline for some follow-up.
> 
> Regards,
> Mustapha.
> 
> > -----Original Message-----
> > From: Chandrasekar Ramachandran [mailto:[e-mail]]
> > Sent: Friday, December 18, 2015 12:41 AM
> > To: Chandrasekar Ramachandran; Aissaoui, Mustapha (Mustapha); Sriganesh
> Kini;
> > Loa Andersson; Lucy yong
> > Cc: [e-mail]; [e-mail]; draft-chandra-mpls-ri-rsvp-
> [e-mail];
> > Guijuan Wang 3; Lizhong Jin
> > Subject: RE: MPLS-RT review of draft-chandra-mpls-ri-rsvp-frr
> >
> > Mustapha,
> > Did my response address your concerns? If I have understood you correctly,
> the
> > concerns fall in two categories and I have tried to address these concerns.
> > (A) The need for introducing refresh-interval independent behavior into RSVP-
> TE
> > (B) Why some kind of local implementation based timers will not be sufficient
> to
> > support long refresh intervals
> >
> > Do you have any specific comment on my responses? Specifically, do you still
> > think the responses did not address (A), or (B) or both?
> >
> > Regards,
> > Chandra.
> >
> > > -----Original Message-----
> > > From: mpls [mailto:[e-mail]] On Behalf Of Chandrasekar
> > > Ramachandran
> > > Sent: Thursday, December 03, 2015 12:58 AM
> > > To: Aissaoui, Mustapha (Mustapha)
> > > <[e-mail]>;
> > > Sriganesh Kini <[e-mail]>; Loa Andersson
> > > <[e-mail]>; Lucy yong <[e-mail]>
> > > Cc: [e-mail]; [e-mail]; draft-chandra-mpls-ri-rsvp-
> > > [e-mail]; Guijuan Wang 3 <[e-mail]>; Lizhong Jin
> > > <[e-mail]>
> > > Subject: Re: [mpls] MPLS-RT review of draft-chandra-mpls-ri-rsvp-frr
> > >
> > > Mustapha,
> > >
> > > > -----Original Message-----
> > > > From: Aissaoui, Mustapha (Mustapha)
> > > > [mailto:mustapha.aissaoui@alcatel-
> > > > lucent.com]
> > > > Sent: Thursday, November 26, 2015 8:48 AM
> > > > To: Sriganesh Kini <[e-mail]>; Loa Andersson
> > > > <[e-mail]>; Lucy yong <[e-mail]>
> > > > Cc: Lizhong Jin <[e-mail]>; Guijuan Wang 3
> > > > <[e-mail]>;
> > > > [e-mail];
> > > > [e-mail]; Aissaoui, Mustapha (Mustapha)
> > > > <[e-mail]>; [e-mail]
> > > > Subject: RE: MPLS-RT review of draft-chandra-mpls-ri-rsvp-frr
> > > >
> > > > Dear authors,
> > > > I read this draft and I believe the intent is to provide a tighter
> > > > control plane synchronization of the PLR and MP roles on a per RSVP
> > > > session basis such that a LSR will know if it is a MP for a given
> > > > RSVP session which requested protection. This is done such that the
> > > > decision to retain or delete the state by the LSR detecting the
> > > > failure of the link to the previous hop (or the failure of the
> > > > previous hop itself) is made with the prior knowledge that the LSR is a MP
> or
> > not on a per RSVP session basis.
> > >
> > > [Chandra] The primary motivation of the draft is to enable LSRs to
> > > support FRR when the LSP scale on the LSR is of the order of hundreds
> > > of thousands. The analysis of the bottlenecks as a consequence of LSP
> > > scale, that can cause disruption of the LSR has already been
> > > documented in RFC 5439. As analyzed in RFC 5439, while there is a
> > > correlation between the percentage increase in refresh time and the
> > > improvement in LSR performance, there is a degree of functionality
> > > that is lost owing to the soft-state nature of the protocol. TE-
> > > SCALE-REC
> > > (https://tools.ietf.org/html/draft-ietf-teas-rsvp-te-scaling-rec-00)
> > > outlines the motivation for refresh independent RSVP (RI-RSVP) for all
> > > types of LSPs (packet or non-packet), and makes recommendations to
> > > enable RSVP implementations eliminate the reliance on short refresh
> > > time. This draft addresses the refresh-interval dependent behavior of
> > > RFC 4090 in order to support RI-RSVP, as facility backup FRR is a
> > > widely deployed feature in production networ  ks.
> 
> MA> It seems to me the main issue with bypass protection is to deal with the
> generation (PLR) and processing (MP) of a large number of *triggered* Path
> messages for the individual protected LSPs at the time the PLR activates the
> bypass LSP.
> 
> I do not see anything new in this draft to address the bottleneck of the state
> refresh mechanism itself. In fact, the draft suggests that after the bypass LSP is
> activated, the state refresh continues with summary refreshes which is the
> solution already existent as per RFC 2961. Based on that, it is misleading to call
> this a Refresh-Independent RSVP FRR and this draft is not changing the fact that
> a PLR can generate and the MP may have to process a Srefresh message with
> potentially a large number of message IDs once the bypass is activated.

[Chandra] The need to generate and process a large number of message IDs will not be a problem as long as the refresh interval is long. If the concern is that transmitting and processing a large number of message IDs would overwhelm the routers, then that is an orthogonal problem for which Section 2.3 of [draft-ietf-teas-rsvp-te-scaling-rec] proposes to solve. Again, [draft-ietf-teas-rsvp-te-scaling-rec] does indeed rely on RFC 2961 Message-Id-ACK extension to accomplish flow control. Even if an implementation supports extensions to reduce the messages between PLR and MP during local repair, we still require long refresh interval in order to relieve the burden of the routers from having to refresh unaffected LSP states and the make-before-break LSP states that occur subsequent to local repair.

Once the refresh interval could be set to an arbitrarily large value, it makes RSVP-TE "refresh interval independent". Is the use of "refresh independent" instead of a longer phrase "refresh-interval independent" a matter of concern here?

> What this mechanism is providing is a bulk notification of bypass activation from
> the PLR to the MP. This can speed up the creation in the control plane of the
> new remote neighbor (PLR) at the MP and associate the existing PSB/RSBs of the
> protected LSPs with this new neighbor. The savings in the creation of the new
> remote neighbor is due to not relying on the receipt from the PLR and the
> processing of a triggered refresh message for each protected LSP. Note here
> there are no issue with the data plane and packets from all LSPs will be
> forwarded properly at the MP since the ILM for the protected LSP is the same
> whether the packet is received from the primary interface or from the bypass
> LSP interface.

[Chandra] Yes, precisely. The data plane can continue to forward traffic but the control plane imposes an artificial restriction that the LSP states should be refreshed within a short refresh timeout. The goal is to remove this restriction and align with the goal of [draft-ietf-teas-rsvp-te-scaling-rec].

> Based on the above, I am having trouble reconciling the limited benefits and the
> complexity of the proposed mechanism.
> I am not opposed for this part of the draft to move forward as an optional
> mechanism but I propose that it gets merged with draft-mtaillon-mpls-summary-
> frr-rsvpte and that the reference to "Refresh-Independent RSVP FRR" be
> removed. Also, the objective of the mechanism to more clearly explained as a
> bulk notification mechanism when bypass is activated.

[Chandra] The goals of draft-mtaillon-mpls-summary-frr-rsvpte and draft-chandra-mpls-ri-rsvp-frr are different. While draft-mtaillon-mpls-summary-frr-rsvpte could be used by RSVP-TE implementation using short refresh interval to reduce message rate between PLR and MP during local repair, draft-chandra-mpls-ri-rsvp-frr could be used by RSVP-TE implementation to set arbitrarily long refresh interval but avoid unacceptable buildup of stale states across FRR local repairs. That's why the only two extensions introduced in draft-chandra-mpls-ri-rsvp-frr involve sending PathTear messages to ensure explicit cleanup of LSP states rather than relying on timeout (which would be long with long refresh interval). Implementation that sets arbitrarily long refresh interval could also use draft-mtaillon-mpls-summary-frr-rsvpte to reduce message rate during local repair.

> > > > However,  the procedures presented in this document are fairly
> > > > complex and go at odds with the original intent of making state
> > > > synchronization based on a soft-state mechanism.
> > >
> > > [Chandra] The tradeoffs between using short or long refresh intervals
> > > has been well understood. Short refresh intervals aid fast
> > > synchronization of states along the path of the LSP, but is
> > > problematic because of the control message traffic that a router has
> > > to handle at high LSP scale. Routers not only must synchronize new
> > > states as promptly as possible, but also must maintain the rate of
> > > periodic Srefresh messages to a level sufficient to refresh all
> > > existing states without being timed out. When the number of LSPs that
> routers
> > carry approach half a million, there will be two problems with the control
> message
> > rate.
> > > (1) As analyzed in RFC 5439, even with RFC 2961 refresh reduction the
> > > size of Srefresh message may become very large, and the processing
> > > required may cause disruption of the LSR.
> > > (2) Apart from the problem of RSVP message processing overhead, there
> > > is also the problem of RSVP-TE becoming a bottleneck preventing the
> > > router to scale other protocols or services.
> 
> MA> This draft does not solve the issue of large Srefresh message. After the bulk
> triggering of the bypass, Srefresh message will still be large because
> implementations will need to make efficient use of the message by sending as
> many Message IDs as possible.

[Chandra] I think this is a problem that is orthogonal to remove the reliance on short refresh interval. As mentioned before, one should also take into account the recommendations in [draft-ietf-teas-rsvp-te-scaling-rec] in order to support LSP scales much higher than what routers run or support today. The point you have raised here has been addressed in Section 2.3 of [draft-ietf-teas-rsvp-te-scaling-rec].
 
> > > > In fact, what I fail to find is a compelling argument or data from
> > > > the field which shows the issue is not resolved via much simpler
> > > > methods which are used in production networks today. I describe this
> > > > in the detailed comments below.
> > >
> > > [Chandra] Some of the mechanisms already deployed in multi-vendor
> > > production networks involve configuring implementation specific timers or
> > delays on LSRs.
> > > Multiple vendors support various timers or delays on Ingress and Transit
> LSRs.
> > > However, in practice the values configured for these timers or delays
> > > are very scale specific, and the values that work at one LSP scale
> > > usually do not work at higher LSP scale. In practice, the "scale" that
> > > impacts the behavior at specific timer or delay value is not only the
> > > number of LSPs carried on the router, but also the number of other
> > > protocol states that reside on the router. It should be noted that the
> > > reliance on such implementation specific timers or delays has been a
> > > major contributor of operational complexity in running RSVP-TE FRR.
> > > Any solution based on such timers or delays while being operationally
> > > simple at one scale ceases to be so at higher scale. It has been
> > > practically found (with existing implementations from multiple
> > > vendors) tha  t it is operationally hard to find out the new better
> > > value as a running production network grows over time. In short, the use of
> such
> > timers or delays has been found to involve guess work that seldom remain
> simple
> > in the long run.
> 
> 
> MA> The timer is for a different issue than the main objective of the draft. Let us
> discuss it in the points below
> 
> > > > Regards,
> > > > Mustapha.
> > > > ---------------------
> > > > 1. Section 3.1 - Problem Description "
> > > > - If the protected LSP on C times out before D receives signaling
> > > > for the backup LSP, then D would receive PathTear from C prior to
> > > > receiving signaling for the backup LSP, thus resulting in deleting
> > > > the LSP state. This would be possible at scale even with default
> > > > refresh time.
> > > > "
> > > > MA> Since each LSR in the path of a RSVP session which requested
> > > > MA> protection
> > > > has to assume it can be a MP without prior knowledge, a simpler
> > > > method is to reset the refresh timeout for each session as soon as
> > > > the link to the previous hop failed. In fact, a user configurable MP
> > > > timeout upon failure, independent of the refresh timeout, can be
> > > > provided to tune it to the desired value to give enough time to the
> > > > Path message to be received via the bypass LSP.
> > >
> > > [Chandra] The option of resetting the refresh timeout may not be
> > > viable if long refresh interval (of the order of tens of minutes) is
> > > applied on the LSPs. That leaves the other option of providing a "wait
> > > timer" (independent of refresh time) that is configurable on the MP.
> However, it is
> > fairly clear that the "wait timer"
> > > that the operator should configure on MP will be problematic for two
> reasons.
> > > The operator should carefully analyze the performance impact of an
> > > existing timer/delay value if and when (a) the LSP scale on the same
> > > router increases, and (2) the LSP scale increases on other routers
> > > around it (that may potentially become upstream PLRs)!
> 
> MA> The use of a refresh-timeout independent timer has been shown to work in
> production networks. Most of the time, the value is conservative enough to
> cover a large scale network.

[Chandra] Relying on implementation specific timers does not provide predictability of behavior as the deployed network scale increases. There are cases in deployed networks where the use of such timers have increased the complexity of network operations (a) when network scale increases with the same software image, and (b) when network scale increase is also accompanied by software upgrades. The general point is that increasing refresh interval to arbitrarily long values without any accompanying mechanism to introducing predictable behavior in the protocol may increase operational complexity. I have provided detailed discussion on this in previous mails.

> > > > 2. Section 3.1 - Problem Description:
> > > > "
> > > > - If upon the link failure C is to keep state until its timeout,
> > > > then with long refresh interval this may result in a large amount of
> > > > stale state on C. Alternatively, if upon the link failure C is to
> > > > delete the state and send PathTear to D, this would result in
> > > > deleting the state on D, thus deleting the LSP. D needs a reliable
> > > > mechanism to determine whether it is MP or not to overcome this
> > > > problem.
> > > > "
> > > > MA> What is exactly the issue with state timeout being retained
> > > > MA> until the
> > > > refresh timeout? You refer to this as "stale" state but in fact it
> > > > is desirable to keep the state until node D created the backup PSB.
> > > > Also, remember that head-end node will perform global revertive MBB
> > > > and may tear down the LSP before the state timeout.
> > >
> > > [Chandra] As described in the first response in this mail, the
> > > motivation driving the draft is to eliminate RSVP-TE's reliance of
> > > short refresh intervals. If router C were to retain the LSP state
> > > until time out, without any additional procedures that provide an
> > > explicit indication to router C on when the LSP state is no longer
> > > required, then router C would retain the LSP state potentially for
> > > hours if the refresh interval is long. So, router C will not only
> > > store the state of the LSP but also periodically send the Path message
> > > (in Srefresh) downstream - thereby unnecessarily consuming resources that
> > could potentially be utilized for other LSPs.
> > > It should also be noted that the transit router cannot assume that
> > > Ingress LSR will be able to complete global repair within a particular
> > > time frame. The transit LSR procedures should be able to handle cases
> > > when global repair does not complete for some valid reason for an extended
> > period of time.
> 
> MA> Clearly, you are not taking away the reliance on refresh messages. Any ad-
> hoc mechanism which is not reliable such as a PathTear may still keep the state
> until it times out. Furthermore, relying exclusively on Message ID ACk as
> proposed in draft-ietf-teas-rsvp-te-scaling-rec in a scaled network is not good.
> Data from deployment networks I am familiar with has shown that the
> retransmission of messages which are not acknowledged when the control
> plane churn is happening is causing much more churn. Our advice for our
> customers was to disable the requirement to receive a Message ID Ack.

[Chandra] I disagree with both the points above.
- The extensions introduced in the draft only involve leveraging PathTear mechanism in two slightly modified ways. If PathTear messages are reliably sent by RSVP routers (i.e. implement RFC 2961 for tear down messages also and also implement flow control described in draft-ietf-teas-rsvp-te-scaling-rec), then the operator could set arbitrarily long refresh intervals. If the concern is the use of "refresh independent" instead of "refresh-interval independent", then I can add clarification to the draft.
- The cause of churn in RSVP-TE deployments running at high scale in terms of LSPs is the lack of flow control in RSVP-TE implementations. It is possible to implement flow control if implementations can leverage Message-id-ACKs suitably as recommended in [draft-ietf-teas-rsvp-te-scaling-rec].

> > > > 3. Section 3.1 - Problem Statement:
> > > > "
> > > > - If head-end A attempts to tear down LSP after step 1 but before
> > > > step 2 of the above sequence, then B may receive the tear down
> > > > message before step 2 and delete the LSP state from its state
> > > > database. If B deletes its state without informing D, with long
> > > > refresh interval this could cause (large) buildup of stale state on
> > > > D.
> > > > "
> > > > MA> I am not sure I understand the issue here. If B acting as a PLR
> > > > MA> receives a
> > > > PathTear from A, all it needs to do is to check that the primary
> > > > path neighbor (C in this case) is in down or in cleanup state and
> > > > send the PathTear over the bypass before deleting its own local state