Re: [mpls] draft-atlas-mpls-te-express-path-02 (was MPLS-RT review of ...)

[Alia Atlas <[e-mail]>]

Hi Curtis,



Thanks for your detailed comments and suggestions.  My responses are in-line, as always.



Alia



-----Original Message-----
From: Curtis Villamizar [mailto:[e-mail]]
Sent: Tuesday, April 23, 2013 12:59 PM
To: MPLS WG Mailing List; draft-atlas-mpls-te-express-path authors; The Great and Mighty MPLS Co-Chairs
Cc: [e-mail]
Subject: draft-atlas-mpls-te-express-path-02 (was MPLS-RT review of ...)





Loa, authors, et al,



So far I have seen two of the four MPLS-RT reviews (maybe I missed the others).  I've commented before on the mailing list about this draft but at this point I would like to provide a detailed review.



I think there are very major issues with this document as it now stands.  IMO these issues should be addressed before the draft is even accepted as a WG document.



You can choose to consider this during MPLS-RT review or after.



Curtis









Major issues:



  Jitter and loss are very close to meaningless if queueing delays and

  loss are not considered.  Links that are losing packets at the link

  layer are generally taken down.  Oscillations can occur if queueing

  delay and loss is considered, such as measurements at a low priority

  and therefore possible to congest or measurements which are

  otherwise affected by traffic load.  Unless there is adequate

  discussion of stability and mechanisms to insure stability, then

  jitter and loss should be removed.



[Alia] I have changed the second paragraph of the introduction to be:



" Queuing latency is specifically excluded to insure freedom from

   oscillations and stability issues that have plagued prior attempts to

   use delay as a routing metric.  If application traffic which follows

   path based upon latency constraints, the same traffic might be in an

   Expedited Forwarding Per-Hop-Behavior [RFC3246] with minimal queuing

   delay or another PHB with potentially very substantial per-hop

   queuing delay.  Only traffic which experiences relatively low

   congestion, such as Expedited Forwarding traffic, will experience

   delays very close to the sum of the reported link delays"



Removing queuing delay is the agreed result for draft-ietf-ospf-te-metric-extensions.

As far as jitter and loss goes, these can still be somewhat meaningful.  For instance, jitter does allow capturing the differences in serialization delays - which is relevant in some parts of the network.  The acceptable loss to take a link down may vary.   I don't actually see the reasoning or need to remove jitter and loss from the draft - given that we have constrained the measurements in the draft-ietf-ospf-te-metric-extensions to not include queuing delay.



General:



  It needs to be much more clearly stated that the NPO (s/SLA/NPO/g,

  see nits below) is specific to a link, and not an NPO per LSP.  A

  "non-conformance to NPO" flag in the IGP link advertisement is

  intractable if there are multiple NPO being applied to the link.



[Alia] As you may recall from draft-ietf-ospf-te-metric-extensions, the Anomalous bit is specified per characteristic advertised per link.   This draft (draft-atlas-mpls-te-express-path-02) describes the Anomalous bit as clearly inside a links' sub-TLV as in Sec 2.3.1:



"If the answer to (a) is no for latency SLAs, then any link which has

the Anomalous bit set in the Unidirectional Link Delay sub-

TLV[I-D.ietf-ospf-te-metric-extensions]

[I-D.previdi-isis-te-metric-extensions] should be removed from the

topology before a CSPF calculation is used to compute a new path."



  The use of a "non-conformance to NPO" flag as the only means to

  alert the set of LSP ingress of a significant change is also at best

  a poor solution.  For example, a change from 2 msec to 15 msec may

  be a problem for some LSP.  That same change may not be a problem

  for others, such as LSPs where only this one hop is needed.  So

  advertising out of NPO in the IGP doesn't help the second case.  LSP

  ingress must evaluate the path delay constraint, each time a change

  in IGP link advertised delay is received.



[Alia] Sure - different LSPs may have different requirements as to the Anomalous flag.  Using it doesn't replace paying attention to the value that is advertised.  The anomalous flag is reporting that the link is not complying to the expected performance - this can indicate that there is something strange going on and some LSPs should avoid using that link due to administrative policy.



  The use of a "non-conformance to NPO" flag solely as a constraint

  makes sense, where some LSP are configured to exclude links with

  this flag set.



[Alia] Right - case (a) and (c) for Sec 2.3



  For path delay computation it would also be useful if the NPO

  threshhold were advertised.  In some cases, it may make sense to

  minimize or place bounds on the sum of NPO delays.



[Alia] I think that should be a comment on draft-ietf-ospf-te-metric-extensions.  This draft doesn't define what is flooded.   A question though is whether in the same network it would make sense to consider both the NPO delays and the actual measured delays?  If only one or the other - then that could be a local router's decision as to which is flooded.



Specific:



  Abstract:



    Remove mention of jitter and loss.





[Alia] I am interested in the opinion of the WG on this.  In my view, this draft is describing how the information flooded via the new sub-TLVs in draft-ietf-ospf-te-metric-extensions-04 should be used.  Loss and jitter are included in that draft; of course, that draft could change as well - but I'd like to hear stronger arguments for why it is harmful to include them than that they're only relevant when queuing behavior is included and that can lead to oscillations.  I am, perhaps stubbornly, not convinced that they are irrelevant.



  1.  Introduction:



    This sentence is awkward but I think I know what you are trying to

    say.



      The method suggested is not optimal for both minimizing path

      cost and additional constraints, such as latency; optimal

      solutions are computationally complex.



    Please consider this replacement.



      Methods of optimizing path selection for multiple parameters are

      generally computationally complex.  The method proposed here are

      to make use of either a single metric in path selection, such as

      minimal path delay, or to make use of another single metric,

      such as the existing TE metric with additional constraints, such

      as target link delay bounds and target path delay bounds.



[Alia] Thanks for the better text - put in...



    Please note:



      The path selection mechanisms described in this document apply

      to paths that are fully computed by the head-end of the LSP and

      then signaled in an ERO where every sub-object is strict.  This

      allows the head-end to consider IGP-distributed performance data

      without requiring the ability to signal the performance

      constraints in an object of the RSVP Path message.



    There is work in MPLS or CCAMP by George Swallow and others on

    cummulative metrics in RSVP-TE with delay as a major motivation.

    Perhaps that work should be considered.



[Alia]  I believe you are referring to http://tools.ietf.org/html/draft-ietf-ccamp-te-metric-recording-01?  That draft appears to me to be about how to get measurements for latency, jitter, and cost for a given Forwarding Adjacency or Routing Adjacency so that those values can be advertised into the IGP.  I think this draft is providing a means to collect the data to flood in draft-ietf-ospf-te-metrics-04.



I did add the last sentence in the following:



"This document does not specify how a router determines what values

to advertise by the IGP; it does assume that the constraints specified

in [I-D.ietf-ospf-te-metric-extensions] and [I-D.previdi-isis-te-metric-extensions] are followed.  Mechanisms for determining latency and delay variation for Forwarding

Adjacencies and Routing Adjacencies are defined in [I-D.ietf-ccamp-te-metric-recording]."





    This paragraph could use rewording:



      When considering performance-based data, it is obvious that

      there are additional contributors beyond just the links.

      Clearly end-to-end latency is a combination of router latency,

      queuing latency, physical link latency and other factors.

      However, if application traffic requires paths to be selected

      based upon latency constraints, the same traffic might be in an

      Expedited Forwarding Per-Hop- Behavior[RFC3246] with minimal

      queuing delay or another PHB with known maximal per-hop queuing

      delay.  While traversing a router can cause delay, that can be

      included in the advertised link delay.



    What you are getting at is that where traffic levels can't

    possibly have a significant impact the measurements, such as low

    levels of EF traffic in a WAN with high geographic delays, and

    delay meansured with EF priority, stability is not impacted if

    delay measurements are considered.  In this case, loss MUST be

    zero or the EF service is broken (replace routers and try again).

    OTOH jitter will increase as traffic levels increase, even in such

    a case where the traffic is only a few percent, potentially

    causing oscillations and impacting stability.  For this reason,

    jitter and loss should not be considered.



    I will leave the rewording up to you.  I suggest that you create a

    subsection of "Introduction" which discusses potential

    oscillations and stability in greater detail.  If you like, I will

    provide some text.



[Alia] I do hear your concern that jitter could cause oscillations and impact stability.  I am not fully persuaded that this is a practical issue - given the delays in ingress re-optimization, not trying to minimize the jitter, and the ability for an LSP to reserve bandwidth.  I would be interested in what text you might provide and a focused discussion on whether this is something that needs to have restrictions clearly described to avoid oscillations.



For now, I have changed this paragraph to:



"When considering performance-based data, it is obvious that there

are additional contributors beyond just the links. Clearly end-to-end

latency is a combination of router latency, queuing latency, physical

link latency and other factors.  While traversing a router can cause

delay, that can be included in the advertised link delay.  As

described in [I-D.ietf-ospf-te-metric-extensions] and

[I-D.previdi-isis-te-metric-extensions], queuing delay

should not be included in the measurements advertised by OSPF or ISIS.



Queuing latency is specifically excluded to insure freedom from

oscillations and stability issues that have plagued prior attempts to

use delay as a routing metric.  If application traffic which follows

path based upon latency constraints, the same traffic might be in an

Expedited Forwarding Per-Hop-Behavior [RFC3246] with

minimal queuing delay or another PHB with potentially very substantial

per-hop queuing delay.  Only traffic which experiences relatively low

congestion, such as Expedited Forwarding traffic, will experience

delays very close to the sum of the reported link delays."



  1.1.  Basic Requirements:



    Drop "packet loss, jitter" from the first numeric item.  Otherwise

    OK except use of the word SLA.  See nits below.



[Alia] Changed SLAs to NPO



    After the list please state that "For existing MPLS constraints,

    corresponding RSVP-TE signaling allows midpoint LSR to use Path

    Tear and/or notification and would use this mechanism to

    accomplish items 3-6.  In the absense of RSVP-TE signaling

    corresponding to these new constraints, new mechanisms at the LSP

    ingress are needed."  Alternately, consider citing the work by

    George Swallow et al and explain how that solves it in the same

    way that a change in admin attr of a link would (or should, poor

    implementations not withstanding).



[Alia] Frankly, I'm confused by this.  I don't agree that RSVP-TE signaling extensions would solve, for instance, (3):



"3. Ability to periodically verify that a TE tunnel's current LSP

complies with its configured end-to-end performance requirements."



Even if the ingress signaled the end-to-end performance requirements, I don't see how that stops the ingress from verifying compliance?



Similarly, only the ingress could do:

"   4.  Ability to move tunnels, using make-before-break, based upon

       computed end-to-end performance complying with configuration



   5.  Ability to move tunnels away from any link that is violating an

       underlying SLA



   6.  Ability to optionally avoid setting up tunnels using any link

       that is violating an SLA, regardless of whether end-to-end

       performance would still meet requirements."



I have looked for the additional work that you are talking about by George Swallow unsuccessfully.  Can you find a pointer to explain what you're talking about?  The anomalous bits provide a trigger mechanism that a router can use to tell the ingress to do (5); different admin attributes could do a similar behavior - and similarly for (6) - but again that is the flooding for notification aspect.  I think the lack of RSVP-TE extensions doesn't cause an issue - these are handled by IGP instead and thus don't have to be signaled per LSP.



I am not irrevocably opposed to RSVP-TE extensions - if we really need them for practical use-cases.  This draft was trying to do the minimum that is sufficient and then, if and when there is a need for more, what extra is needed could be better defined.



  2.1.  End-to-End Constraints:



    Note: I've requested that jitter and loss be dropped from

    draft-ietf-ospf-te-metric-extensions and

    draft-previdi-isis-te-metric-extensions for the same potential

    oscillations and stability reasons cited above.



[Alia] Yes - I think we clearly need to have a good email discussion with those interested about what oscillations might actually occur and what could be done to prevent that.



    s/While it has been possible to compute a CSPF/It is possible to

    compute a CSPF/

    s/Instead of this approach to minimize path latency, an/An

    alternative to this approach to minimize path latency is an

    approach to place a upper bound on path latency.  An/

    Note: both approaches are valid.



    Delete next paragraph starting with "This is illustrated as

    follows."  This seems to be taken from email and is good email

    discussion but not needed in the draft.



[Alia] I've put in some pseudo-code so I'm ok with taking the example out - but there does seem to have been confusion even with it in.



    Delete next paragraph starting with "An end-to-end bound on delay

    variation".  Lets get rid of jitter altogether.  (Let the old SNA

    networks be damned. :)



    Delete next paragraph starting with "For link loss".  Get rid of

    link loss altogether.



[Alia] Not done - discussion is needed.  I understand that you really really really don't want to see loss or delay variation in any of these drafts.



  2.2.  Link Constraints:



    Drop delay variation and link loss.



    If we are dealing with EF traffic then using Unidirectional

    Available Bandwidth and Residual Bandwidth makes no sense.  If we

    are dealing with low priority traffic and we are using

    Unidirectional Available Bandwidth and Residual Bandwidth in path

    selection will be prone to oscillations and network instability.

    Therefore delete the entire second paragraph (the one starting

    with "When doing path selection for TE tunnels,".



[Alia]  If we are trying to avoid congesting the link, then Residual Bandwidth is important and useful regardless of the LSP traffic class.  What is your specific concern with oscillation for bandwidth?  Why is it different than for the bandwidths already advertised and used for TE path computation?   Come on - the Residual Bandwidth is just the Link Capacity minus that reserved by RSVP-TE - so pretty darn similar characteristics to the Unreserved Bandwidth per priority...   The Unidirectional Available bandwidth does include an actual traffic measurement - that is averaged over a reasonable interval, that can only change with limited frequency - and then the LSPs need to be reoptimized to use them.  Please explain precisely the oscillation concern with a clear example.





    Also delete the entire third paragraph (starting with "Similarly,

    only links whose loss is").  As stated earlier, use of loss is

    either a NOOP (low volume of EF traffic) or can lead to

    oscillations.  Get rid of loss entirely.



  2.3.  Links out of SLA:



    s/SLA/NPO/g (see nits below).



    General: A better mechanism than an "Anomalous State" flag is

    needed to provide notifications of change.  One mechanism would be

    to put the commulative constraint and the cummulative total in the

    ERO and RRO.  A link adding delay can then notify the ingress of

    any LSP for which the commulative constraint is violated.  See

    work by Swallow et al and perhaps align with that work.



[Alia] That could be done as well - but the signaling extensions seem like overkill for the basic problem of letting the ingress compute a complete ERO.  Carrying both the cumulative constraint and the cumulative total just means that the midpoint detects a changed link and then has to verify the performance on each LSP and individually signal it.  Having an anomalous flag provides a succinct notification to the ingress which can then do the verification and be known to have the updated information.  This solution also requires that the midpoints all support it before it can be used.  An advantage of the ingress computation is that only the ingress needs to have updated code.



    The "Anomalous State" flag is helpful for c in the list, but not

    for b.  The case where a link is out of NPO by 1 msec then changes

    to out of NPO by 10s of msec is an example.  The flag is already

    set and therefore the trigger is unavailable.



[Alia] Right - but LSPs that are particular sensitive (i.e. a or c) can have been moved already.   Having the Anomalous flag set is expected to be unusual.  I agree that it is more a hint for (b) than a full solution.



  2.3.1.  Use of Anomalous Links for New Paths



    Delete second paragraph regarding jitter and loss.



  2.3.2.  Links entering the Anomalous State



    This section ignores two cases in which the Anomalous State does

    not change but a change makes the path violate a delay

    constraint.  The first is where all of the links are within NPO

    but a change to a link has make the path delay sum exceed the path

    delay constraint.  The second is where a link which is already in

    the Anomalous State by a small margin but the path delay is still

    within the constraint.  A large change in delay at that point will

    not affect the Anomalous State since it is already set.



[Alia]  The Anomalous bit is NOT meant to replace reading the actual value associated with the link and checking each potentially affected LSP.  In the case of (b), it is a hint to focus on those LSPs first.



    A better means of handling case (b) in "2.3.  Links out of SLA" is

    needed.



[Alia]  I've added the following paragraph at the end of 2.3.2:

"It is not sufficient to just look at the Anomalous bit in order to

determine when TE tunnels must have their compliance verified.  When

changing to set, the Anomalous bit merely provides a hint that

interested TE tunnels for case (b) should have their continued

compliance verified."



  2.3.3.  Links leaving the Anomalous State



    Same issue as in "2.3.2.  Links entering the Anomalous State".  A

    better means of handling case (b) in "2.3.  Links out of SLA" is

    needed.





[Alia]  I've added the following sentence:

"The hint provided by the Anomalous state change may help optimize when to recompute for a better path."



XML version nits:



  CV: you really should remove the template comments.



[Alia]  I find them helpful for when I want to do additional things... and very very few people read the XML.



  You should also enable strict mode.  For example, you have one

  author too many and strict would catch that.



[Alia]  So does basic arithmetic :)   It will be resolved before the draft is passed to the RFC editor.



Other nits:



  The "A" in SLA is "Agreement" as in contract.  The acronym NPO for

  network performance objective seems to be in vogue for that reason.

  IETF since diffserv has wanted to steer clear of making

  recommendations regarding provider contracts (agreements) with

  customers, peer, or anyone else.



[Alia]  True - changed



  I agree with Sri on the suggestions to change the title, short name

  and document filename but I'm not fond of his suggested new names.

  Authors please suggest new title, short name, and filename.



[Alia]  I did do a new short name and title.  As for filename, we'll deal with that if/when the draft is adopted as a WG draft.



Thanks again,

Alia





------- Forwarded Message



On 4/5/13 9:50 PM, "Loa Andersson" <[e-mail]<mailto:[e-mail]>> wrote:

>

>Xiaohu, Sri, Rajiv and Pranjal,

>

>You have been selected as an MPLS Review team reviewers for

>draft-atlas-mpls-te-express-path-02.

>

>Note to authors: You have been CC'd on this email so that you can know

>that this review is going on. However, please do not review your own

>document.

>

>Reviews should comment on whether the document is coherent, is it

>useful (ie, is it likely to be actually useful in operational

>networks), and is the document technically sound?  We are interested in

>knowing whether the document is ready to be considered for WG adoption

>(ie, it doesn't have to be perfect at this point, but should be a good

>start).

>

>Reviews should be sent to the document authors, WG co-chairs and WG

>secretary, and CC'd to the MPLS WG email list. If necessary, comments

>may be sent privately to only the WG chairs.

>

>Are you able to review this draft by April 20, 2013?

>

>Thanks, Loa

>(as MPLS WG chair)

>

>/Loa

>--

>

>

>Loa Andersson                        email: [e-mail]<mailto:[e-mail]>

>Senior MPLS Expert                          [e-mail]<mailto:[e-mail]>

>Huawei Technologies (consultant)     phone: [phone]



_______________________________________________

mpls mailing list

[e-mail]<mailto:[e-mail]>

https://www.ietf.org/mailman/listinfo/mpls



------- End of Forwarded Message