RE: OPS-Dir review of draft-ietf-mpls-multipath-use-03

["Black, David" <[e-mail]>]

> I think we are in agreement and I will edit the document to reflect
> your comments shortly.  I think Adrian is AD on this and Loa the
> shepard, so I'll wait for a go-ahead from either of them before
> submiting the edits.

Yes, we're in agreement, and indeed, "sandwich" is a term I coined in my
review - it seemed descriptive of an MPLS-TP label in a stack between
two ELI/EL pairs.

Thanks,
--David

> -----Original Message-----
> From: Curtis Villamizar [mailto:[e-mail]]
> Sent: Tuesday, January 21, 2014 8:08 PM
> To: Black, David
> Cc: [e-mail]; [e-mail]; [e-mail]; draft-ietf-mpls-
> [e-mail]
> Subject: Re: OPS-Dir review of draft-ietf-mpls-multipath-use-03
>
>
> In message <[e-mail]>
> "Black, David" writes:
> >
> > Curtis,
> >
> > > Thanks for the review and for pointing out lack of clarity due to
> > > underlying assumptions about the use of RFC 4206 (and perhaps use of
> > > RFC 4201 as well for large MPLS LSP over MPLS-TP).
> > >
> > > Sometimes it is too easy to forget that not all readers are going to
> > > be as steeped in MPLS "as deployed today" details as a relatively
> > > small number of long time MPLS participants are.
> > >
> > > Adding a few paragraphs to the introduction and a few concluding
> > > paragraphs may significantly improve clarity for people who have not
> > > been following MPLS closely for a decade or more without requiring
> > > them to read the references and figure out usage details themselves.
> > >
> > > Please let me know how you feel about the suggested additions to the
> > > document.
> >
> > Most of them look good - details below.  Many thanks for the prompt and
> > comprehensive response.  I have a few more suggestions in what follows
> > - anything that I don't comment on below is fine with me.
>
> Thank you for the quick turnaround on this as well.
>
> > -- Introduction
> >
> > > There is an underlying assumption in the draft that the reader
> > > understands that MPLS-in-MPLS exists today, most commonly but not
> > > exclusively in the form of RFC 4206 hierarchy.  RFC 4206 provides a
> > > means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some
> > > level of Packet Switch Capable (PSC) hierarchy.  It is also possible
> > > to make use of the management plane, using CLI or NETCONF or the MPLS
> > > cross connect MIB to program a multiple label MPLS push operation.
> > >
> > > If a very experienced reviewer could miss that, then the introduction
> > > could be improved by better stating assumptions.  It is also the case
> > > that many participants in MPLS, particularly those primarily
> > > interested in MPLS-TP, began participation long after RFC 4206 was
> > > written and would benefit by better stated assumptions in the intro.
> > >
> > > I suggest that the following paragraphs be added to the introduction.
> >
> > The suggested new paragraphs will definitely improve the introduction.
> >
> > -- [A] Summary text in Introduction appears to be wrong.
> >
> > > > Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above
> > > > exception appears to concern the reverse, and Section 4 on carrying MPLS
> > > > over MPLS-TP allows for MPLS LSPs that exceed component link capacity.
> > > > Was the exception intended to be for an MPLS-TP LSP that exceeds the
> > > > capacity of any single component link?
> >
> > [... snip ...]
> >
> > > To answer your question directly: the exception is a very large MPLS
> > > LSP can't be carried by a single MPLS-TP LSP.  The solution is to use
> > > multiple MPLS-TP LSP.  The MPLS-TP LSR need not be not "aware" that
> > > this is occurring.
> >
> > It looks like  my confusion stemmed from the original text only
> > discussing only what won't work and referencing a section that
> > concerns a different encapsulation order than the one that causes
> > the problem.  Including the "MAY NOT" -> "SHOULD NOT" change, I suggest:
> >
> > OLD
> >    When an MPLS LSP
> >    exceeds the capacity of any single component link it MAY be carried
> >    by a network using multipath techniques, but MAY NOT be carried by a
> >    single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
> >    imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
> >    fate sharing constraints and MPLS-TP LM OAM packet ordering
> >    constraints (see Section 3.1).
> > NEW
> >    When an MPLS LSP
> >    exceeds the capacity of any single component link it MAY be carried
> >    by a network using multipath techniques, but SHOULD NOT be carried by a
> >    single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
> >    imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
> >    fate sharing constraints and MPLS-TP LM OAM packet ordering
> >    constraints (see Section 3.1).  Instead, multiple MPLS-TP LSPs SHOULD
> >    be used to carry a large MPLS LSP (see Section 4).
>
> OK.  Good addition.
>
> > -- [B] Section 3.2 needs a summary.
> >
> > The proposed summary looks good, and helps with [C], but the phrase
> > "access and edge" needs qualification in the following sentence:
> >
> >    However access and edge is the least likely to support
> >    RFC 6790 in the near term.
> >
> > "access and edge" what? e.g., areas of the network? LSRs?
>
> Oh.  Good point.
>
> Core, edge, aggregation, and access are terms commonly used for the
> tiers in provider networks.  Core is central, high traffic.  Access is
> furthest out with lower traffic (think of subscriber connections for
> consumer Internet).
>
> Perhaps I can just change the sentence to:
>
>   However equipment furthest from a provider's network core is the
>   least likely to support RFC 6790 in the near term.
>
> I think we all know "core" at this point so furthest from core might
> be easier to understand.
>
> > -- [C] Entropy label "sandwich" seems peculiar and at least needs an
> >  explanation.
>
> That was your term.  I didn't propose adding Entropy label "sandwich"
> to the text.  I think this is just an excerpt from your prior email as
> a marker in following up on it.  Please correct me if I am wrong.
>
> > > The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following
> > > paragraph.
> > >
> > >    Alternately, the need for requirement MP#1 can be eliminated if
> > >    every MPLS-TP LSP created by an MPLS-TP ingress makes use of an
> > >    Entropy Label Indicator (ELI) and Entropy Label (EL) below the
> > >    MPLS-TP label [RFC6790].  This would require that all MPLS-TP LSR
> > >    in a deployment support Entropy Label, which may render it
> > >    impractical in many deployments.
> > >
> > > Please not the "Alternately" at the beginning and "impractical in many
> > > deployments" at the end.
>
> That should have said "Please note".
>
> > > The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs
> > > above that but assumes knowledge of RFC 6790.  I think the following
> > > change within this paragraph would clear things up:
> >
> > That change is helpful, but my "sandwich" concern was actually raised
> > by text two paragraphs further down:
> >
> >    MPLS-TP LSPs can be carried as client LSPs within an MPLS server LSP
> >    if an Entropy Label Indicator (ELI) and Entropy Label (EL) is pushed
> >    below the server layer LSP label(s) in the label stack, just above
> >    the MPLS-TP LSP label entry [RFC6790].
> >
> > Read after the "Alternately" paragraph, this text appeared to suggest
> > a "sandwich" that would have an ELI/EL pair both above and below the
> > MPLS-TP label.  That was clearly not intended, so I suggest inserting
> > "When Entropy Label Indicators (ELIs) and Entropy Labels (ELs) are not
> > applied by MPLS-TP ingresses," before the start of the first sentence
> > above, so that the result is:
> >
> >    When Entropy Label Indicator (ELIs) and Entropy Labels (ELs) are
> >    not applied by MPLS-TP ingresses, MPLS-TP LSPs can be carried as
> >    client LSPs within an MPLS server LSP if an ELI and EL are pushed
> >    below the server layer LSP label(s) in the label stack, just above
> >    the MPLS-TP LSP label entry [RFC6790].
>
> OK.  I will make that replacement.
>
> > -- [D] Discussion of multipath impacts on MPLS OAM
> >
> > > > Section 3.2 discusses OAM, and contains this important
> > > > requirement for OAM traffic:
> > > >
> > > >    An Entropy Label must be used to insure that all of the
> > > >    MPLS-TP payload and OAM traffic are carried on the same component,
> > > >    except during very infrequent transitions due to load balancing.
> > > >
> > > > [D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in
> > > > Section 4; that should be added.
> > >
> > > There is no impact on MPLS OAM traffic.  There are three cases to
> > > consider.
> > >
> > >   1. Where MPLS is carried over a single MPLS-TP all traffic flows on
> > >      one link.  MPLS OAM is unaffected and need not use multipath
> > >      support in LSP Ping.
> > >
> > >   2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP
> > >      LSP is carried over one link thanks to the fixed EL value.
> > >      MPLS-TP OAM is unaffected.
> > >
> > >   3.  Where MPLS is carried over MPLS (an existing case) or over
> > >       multiple MPLS-TP, the multipath support in LSP Ping (as
> > >       imperfect as it may already be) is used and LSP Ping (RFC 4379)
> > >       operation is unaffected.
> > >
> > > If needed something can be added to state the above (maybe with less
> > > informal wording).  What is obvious to some people may not be obvious
> > > to everyone.  Briefly stating the almost obvious is usually not a bad
> > > thing.
> >
> > Points 1-3 are useful explanation as complement to the Entropy label
> > requirement to correctly handle OAM traffic for the other encapsulation.
> >
> > I like your suggestion to add this text as part of a new concluding
> > Section 5, and the rest of the proposed conclusion section text looks
> > good.
> >
> > Thanks,
> > --David
>
> Thanks again.
>
> I think we are in agreement and I will edit the document to reflect
> your comments shortly.  I think Adrian is AD on this and Loa the
> shepard, so I'll wait for a go-ahead from either of them before
> submiting the edits.
>
> Curtis
>
>
> > > -----Original Message-----
> > > From: Curtis Villamizar [mailto:[e-mail]]
> > > Sent: Tuesday, January 21, 2014 2:23 PM
> > > To: Black, David
> > > Cc: [e-mail]; [e-mail]; draft-ietf-mpls-multipath-
> > > [e-mail]
> > > Subject: Re: OPS-Dir review of draft-ietf-mpls-multipath-use-03
> > >
> > >
> > > In message <[e-mail]>
> > > "Black, David" writes:
> > > >
> > > > I have reviewed this document as part of the operations directorate's
> > > > ongoing effort to review all IETF documents being processed by the
> > > > IESG.  These comments were written primarily for the benefit of the
> > > > operations area directors.  Document editors and WG chairs should treat
> > > > these comments just like any other last call comments.
> > > >
> > > > Summary:
> > > > This draft is on the right track but has open issues, described in the
> > > review.
> > > >
> > > > I apologize for this review showing up a couple of days after the end of
> > > > IETF Last Call.
> > > >
> > > > This draft discusses considerations for what are effectively MPLS-in-
> MPLS
> > > > tunnels for multipathing - an LSP carries another LSP, with the inner
> > > > (client) LSPs being distributed across multiple outer (server) LSPs that
> > > > use different paths.  This draft is motivated by considerations specific
> > > > to MPLS-TP that complicate multipathing when one of the two MPLS
> instances
> > > > is MPLS-TP.
> > >
> > > Skipping to the discussion quite a ways down, there seems to be some
> > > confusion about what is being proposed here.  While MPLS-in-MPLS is
> > > being used, no new form of MPLS-in-MPLS is being proposed.
> > >
> > > There is an underlying assumption in the draft that the reader
> > > understands that MPLS-in-MPLS exists today, most commonly but not
> > > exclusively in the form of RFC 4206 hierarchy.  RFC 4206 provides a
> > > means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some
> > > level of Packet Switch Capable (PSC) hierarchy.  It is also possible
> > > to make use of the management plane, using CLI or NETCONF or the MPLS
> > > cross connect MIB to program a multiple label MPLS push operation.
> > >
> > > If a very experienced reviewer could miss that, then the introduction
> > > could be improved by better stating assumptions.  It is also the case
> > > that many participants in MPLS, particularly those primarily
> > > interested in MPLS-TP, began participation long after RFC 4206 was
> > > written and would benefit by better stated assumptions in the intro.
> > >
> > > I suggest that the following paragraphs be added to the introduction.
> > >
> > >    MPLS LSPs today are able to function as a server layer and carry
> > >    client MPLS LSPs.  When control plane signaling is used, forwarding
> > >    adjacency (FA) advertisements are used to inform the set of LSR of
> > >    Packet Switching Capable (PSC) LSP within the MPLS topology
> > >    [RFC4206].  Client MPLS LSP at a higher layer (lower PSC number)
> > >    may signal their intention to use PSC LSP as hops in the RSVP-TE
> > >    Explicit Route Object (ERO).  LSR with no explicit support for RFC
> > >    4206 see the PSC LSP as ordinary links and therefore use them.
> > >
> > >    An MPLS LSP that has been set up using RSVP-TE appears to its
> > >    ingress LSR as a viable IP next hop to a distant LSR.  If LDP is
> > >    used and bidirectional RSVP-TE LSP connectivity is available, then
> > >    LDP signaling can be set up among the RSVP-TE LSP endpoints and LDP
> > >    can make use of the RSVP-TE LSP as an LDP hop.  This is another
> > >    form of existing MPLS-in-MPLS use.
> > >
> > >    MPLS LSPs may also make use of hierarchy that is configured through
> > >    the management plane rather than signaled using RSVP-TE.
> > >
> > >    These existing forms of MPLS-in-MPLS may traverse multipath hops
> > >    such as Ethernet LAG [IEEE802.1AX] or MPLS Link Bundling [RFC4201].
> > >    MPLS-TP brings with it a new set of requirements not considered in
> > >    past deployments of the various forms of MPLS-in-MPLS where
> > >    multipath was in use.  This document merely discusses use of
> > >    existing forwarding and protocol mechanisms that can support the
> > >    case where either the client layer LSPs or the server layer LSPs
> > >    are MPLS-TP and where multipath is used.
> > >
> > > If there are no objections I will add these paragraphs.
> > >
> > > Now on to the specific comments.
> > >
> > > > I consider the following to be open issues that need attention (each
> > > > letter is used to tag the issue in the text that follows - [B] occurs
> > > > multiple times, as there are multiple aspect to that issue):
> > > >
> > > > [A] Summary text in Introduction appears to be wrong.
> > > > [B] Section 3.2 needs a summary.
> > > > [C] Entropy label "sandwich" seems peculiar and at least needs an
> explanation.
> > > > [D] Discussion of multipath impacts on MPLS OAM should be added to
> Section
> > > 4.
> > > >
> > > > --- Comments on draft text
> > > >
> > > > [A] Section 1 (Introduction) 2nd paragraph:
> > > >
> > > >    RFC 5654 requirement 33 requires the capability to carry a client
> > > >    MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer [RFC5654].
> > > >    This is possible in all cases with one exception.  When an MPLS LSP
> > > >    exceeds the capacity of any single component link it MAY be carried
> > > >    by a network using multipath techniques, but MAY NOT be carried by a
> > > >    single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
> > > >    imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
> > > >    fate sharing constraints and MPLS-TP LM OAM packet ordering
> > > >    constraints (see Section 3.1).
> > > >
> > > > Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above
> > > > exception appears to concern the reverse, and Section 4 on carrying MPLS
> > > > over MPLS-TP allows for MPLS LSPs that exceed component link capacity.
> > > > Was the exception intended to be for an MPLS-TP LSP that exceeds the
> > > > capacity of any single component link?
> > >
> > > There is no problem today with MPLS LSP carried by MPLS-TP LSP except
> > > the existing case where MPLS LSP capacity is too big.  For example, a
> > > 500 Gb/s LSP is possible today carried over N x 100 Gb/s or N x 10
> > > Gb/s multipath links but could not be carried over a single MPLS-TP
> > > LSP which would be limited to a single 100 Gb/s or 10 Gb/s link.  So
> > > the exception is where the MPLS LSP is too big to fit in any possible
> > > single MPLS-TP LSP.
> > >
> > > To answer your question directly: the exception is a very large MPLS
> > > LSP can't be carried by a single MPLS-TP LSP.  The solution is to use
> > > multiple MPLS-TP LSP.  The MPLS-TP LSR need not be not "aware" that
> > > this is occurring.
> > >
> > > > Also, "MAY NOT" is not an RFC 2119 term (for good reason - it's
> equivalent
> > > > to "MAY OR MAY NOT").  I think "cannot" was intended here, but "MUST
> NOT"
> > > > is another possibility.
> > >
> > > Good point.  I suggest we replace the MAY NOT with SHOULD NOT.  Many
> > > people would insist that it should be MUST NOT, but in reality
> > > networks SHOULD NOT be congested but sometimes they are.
> > >
> > > > [B] Section 3.2 contains a lot of useful details, but it could really
> use
> > > > a summary at the end on the options and recommendations for meeting the
> > > > four requirements (this summary could be a new Section 3.3).  It's a bit
> > > > difficult to discern what's going on, as I had to read this section more
> > > > than once in order to discern the following:
> > >
> > > OK.  A Summary paragraph can be added.  See below.  Perhaps conclusion
> > > section can be added.  See far below.
> > >
> > > > [C] In Section 3.2, use of an MPLS entropy label to meet requirement
> MP#1
> > > > requires that it be "below the MPLS-TP label" whereas, to meet
> requirement
> > > > MP#2, the MPLS entropy label needs to be "just above the MPLS-TP LSP
> label".
> > > > This combination appears to suggest "sandwiching" the MPLS-TP label
> between
> > > > two entropy labels that effectively carry the same information (albeit,
> > > > applied and removed at different places in the network) - was that
> intended?
> > > > This is important, as there seems to be no reasonable alternative
> > > > to the entropy label for meeting requirement MP#2, and the draft's
> > > > alternative to the entropy label for meeting requirement MP#1 is "some
> form
> > > > of configuration" - the entropy label would seem to be preferred to that
> > > > based on my reading of this draft and the comment in RFC 5706 section
> 2.2
> > > > that "Anything that can be configured can be misconfigured."
> > >
> > > The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following
> > > paragraph.
> > >
> > >    Alternately, the need for requirement MP#1 can be eliminated if
> > >    every MPLS-TP LSP created by an MPLS-TP ingress makes use of an
> > >    Entropy Label Indicator (ELI) and Entropy Label (EL) below the
> > >    MPLS-TP label [RFC6790].  This would require that all MPLS-TP LSR
> > >    in a deployment support Entropy Label, which may render it
> > >    impractical in many deployments.
> > >
> > > Please not the "Alternately" at the beginning and "impractical in many
> > > deployments" at the end.
> > >
> > > The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs
> > > above that but assumes knowledge of RFC 6790.  I think the following
> > > change within this paragraph would clear things up:
> > >
> > >  OLD
> > >
> > >    MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI)
> > >    and Entropy Label (EL) are added to the label stack [RFC6790].  For
> > >    those client LSP that are MPLS-TP LSP, a single EL value must be
> > >    chosen.  For those client LSP that are MPLS LSP, per packet entropy
> > >    below the top label must, for practical reasons, be used to
> > >    determine the entropy label value.
> > >
> > >  NEW
> > >
> > >    MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI)
> > >    and Entropy Label (EL) are added to the label stack by the midpoint
> > >    LSR for the client MPLS-TP LSP, at the ingress of the MPLS LSP
> > >    [RFC6790].  For those client LSP that are MPLS-TP LSP, a single EL
> > >    value must be chosen.  For those client LSP that are MPLS LSP, per
> > >    packet entropy below the top label must, for practical reasons, be
> > >    used to determine the entropy label value.  The resulting label
> > >    stack contains the server MPLS LSP label, ELI, EL and the client
> > >    LSP label.
> > >
> > > The above rewording is simply more explicit about the label stack
> > > order and which LSR puts the ELI and EL in the label stack.
> > >
> > > > [B] A summary of requirements for implementation and deployment at the
> end
> > > > of Section 3.2 would have made this much more obvious.  That should be
> > > > provided in addition to addressing the above question about duplicate
> use
> > > > of the MPLS entropy label.
> > >
> > > The following summary can be added to the end of Section 3.2:
> > >
> > >    Two methods of adding an Entropy Label are described above.  The
> > >    MPLS-TP ingress must have a means to determine which links can
> > >    support MPLS-TP in selecting a path (MP#4).  Administrative
> > >    attributes can satisfy that requirement.  If the MPLS-TP LSR is
> > >    capable of adding ELI/EL to the label stack, this method is
> > >    preferred.  However access and edge is the least likely to support
> > >    RFC 6790 in the near term.  For portions of the topology where an
> > >    MPLS-TP is carried within a server layer MPLS LSP the ingress of
> > >    the server layer MPLS LSP can add ELI/EL using a fixed EL value per
> > >    client LSP, except those known not to require MPLS-TP treatment.
> > >    There are numerous ways to determine which client LSP are MPLS-TP
> > >    LSP and which are not.  While this determination is out of scope
> > >    and will vary among deployments, configuration or the presence of
> > >    specific attribute affinities in RSVP-TE signaling are among the
> > >    likely means to do so.
> > >
> > > If there are no objections I will add the above paragraph.
> > >
> > > > --- Relevant Q&A based on RFC 5706 Appendix A
> > > >
> > > > I'm omitting management concerns, as both MPLS and MPLS-TP have
> extensive
> > > > management infrastructure, and this draft's notion of stacking LSPs so
> > > > that one instance of MPLS (or MPLS-TP) can be a server to (i.e., carry)
> > > > another is not novel to this draft.
> > > >
> > > > A.1.1.  Has deployment been discussed?
> > > >
> > > > [B] Yes, although Section 3.2 is weak on deployment requirements (some
> of
> > > > which are implementation requirements) and could use a summary.
> > >
> > > The use of RFC 4206 is assumed throughout the discussion of signaling
> > > in section 3.2.  This assumption is made more clear by suggested
> > > additions to the introduction.
> > >
> > > > A.1.2.  Has installation and initial setup been discussed?
> > > >
> > > > No, although because this draft reuses existing functionality in
> different
> > > > configurations, this really reduces to the A.1.9 configuration question
> > > > below.  In practice, quite a bit of operator knowledge and insight is
> > > > required to get initial setup right, e.g., as traffic engineering LSPs
> to
> > > > avoid congestion problems (needed to meet requirement MP#3) requires
> > > > significant knowledge of network structure and expected traffic flows.
> > > > This should be obvious to anyone who works w/MPLS, but it's probably
> > > > worth stating for completeness.
> > >
> > > FA advertisements defined in RFC 4206 make it possible for ingress to
> > > signal viable paths.  Either the use of Ethernet Link Aggregation or
> > > MPLS Link bundling using the all-ones component link (RFC 4201) allows
> > > the total capacity of a multipath to be advertised.  Meeting the
> > > requirement to balance traffic over the component links in a multipath
> > > (MP#3) is then a local matter and requires no further signaling
> > > extensions.
> > >
> > > > A.1.6.  Have suggestions for verifying correct operation been discussed?
> > > >
> > > > Yes and no.  Section 3.2 discusses OAM, and contains this important
> > > > requirement for OAM traffic:
> > > >
> > > >    An Entropy Label must be used to insure that all of the
> > > >    MPLS-TP payload and OAM traffic are carried on the same component,
> > > >    except during very infrequent transitions due to load balancing.
> > > >
> > > > [D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in
> > > > Section 4; that should be added.
> > >
> > > There is no impact on MPLS OAM traffic.  There are three cases to
> > > consider.
> > >
> > >   1. Where MPLS is carried over a single MPLS-TP all traffic flows on
> > >      one link.  MPLS OAM is unaffected and need not use multipath
> > >      support in LSP Ping.
> > >
> > >   2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP
> > >      LSP is carried over one link thanks to the fixed EL value.
> > >      MPLS-TP OAM is unaffected.
> > >
> > >   3.  Where MPLS is carried over MPLS (an existing case) or over
> > >       multiple MPLS-TP, the multipath support in LSP Ping (as
> > >       imperfect as it may already be) is used and LSP Ping (RFC 4379)
> > >       operation is unaffected.
> > >
> > > If needed something can be added to state the above (maybe with less
> > > informal wording).  What is obvious to some people may not be obvious
> > > to everyone.  Briefly stating the almost obvious is usually not a bad
> > > thing.  Please tell me if a small subsection on "Imopact on MPLS and
> > > MPLS-TP OAM" is needed.
> > >
> > > Note that the limitations of LSP Ping are currently being discussed in
> > > the MPLS WG in the context of extensions to LSP Ping to better support
> > > Entropy Label.  A few limitations not related to EL may get fixed in
> > > the process.
> > >
> > > > A.1.9.  Is configuration discussed?
> > > >
> > > > [B] Not much. Configuration is required to meet requirements MP#3 and
> MP#4,
> > > > and is one of the options for MP#1.  A summary of what has to be
> configured
> > > > as part of the summary at the end of Section 3.2 would be a good idea.
> > >
> > > Perhaps a worked example appendix could be added but I don't think
> > > this is common and may not be necessary with some of the
> > > clarifications suggested so far.
> > >
> > > > A.2.* [skipped, see above]
> > > >
> > > > A.3 Documentation
> > > >
> > > > There's quite a bit of operational material contained in this draft; a
> > > > separate section on operations and management considerations would not
> > > > be appropriate.
> > > >
> > > > There is a useful implementation status section which appears to imply
> > > > that MPLS-TP over MPLS is not currently deployable due to absence of
> > > > entropy label support, and because deployed MPLS equipment does not
> > > > generally support multipathing.  Nonetheless, data center experience
> > > > indicates significant value and widespread use of multipathing based
> > > > on link aggregation and ECMP; corresponding benefits can be expected
> > > > for use of multipathing in the Internet beyond data centers.
> > >
> > > If you would like I can add a conclusion section as Section 5 with
> > > these paragraphs:
> > >
> > >    MPLS equipment deployed in the core currently supports multipath.
> > >    For large service providers, core LSR must support some form of
> > >    multipath to be deployable.  Deployed MPLS access and edge
> > >    equipment is often oblivious to the use of multipath in the core.
> > >    It is expected that at least first generation MPLS-TP equipment
> > >    will be oblivious to the use of multipath in the core.  This first
> > >    generation MPLS-TP equipment is deployable in a core using
> > >    multipath, with no adverse impact to RSVP-TE signaling, if the edge
> > >    equipment can support administrative attributes (RFC 3209) and the
> > >    core equipment can support ELI/EL and put a per-LSP fixed EL value
> > >    on any LSP that indicates a particular attribute affinity or can
> > >    identify client MPLS-TP LSP through some other means.
> > >
> > >    There are no issues carrying MPLS over MPLS-TP, except when the
> > >    MPLS LSP is too big to be carried by a single MPLS-TP LSP.  Most
> > >    MPLS core equipment and some edge equipment can configure an MPLS
> > >    Link Bundle [RFC4201] over multiple component links where the
> > >    component links are themselves MPLS LSP.  This existing capability
> > >    can be used to carry large MPLS LSP and overcome the limited
> > >    capacity of any single server layer MPLS-TP LSP.
> > >
> > > If this conclusion section is added, then the three bullets above
> > > regarding impact on OAM can be put there.  The three bullets can be
> > > added as follows:
> > >
> > >    MPLS OAM and MPLS-TP OAM are unaffected in the following cases
> > >    proposed in this document:
> > >
> > >      1. Where MPLS is carried over a single MPLS-TP all traffic flows
> > >         on one link.  MPLS OAM is unaffected and need not use
> > >         multipath support in LSP Ping.
> > >
> > >      2. Where MPLS-TP is carried over MPLS, all traffic for that
> > >         MPLS-TP LSP is carried over one link thanks to the fixed EL
> > >         value.  MPLS-TP OAM is unaffected.
> > >
> > >      3.  Where MPLS is carried over MPLS (an existing case) or over
> > >          multiple MPLS-TP, the multipath support in LSP Ping is used
> > >          and LSP Ping (RFC 4379) operation is unaffected.
> > >
> > > I think adding the above as a conclusion section as Section 5 would
> > > improve clarity and therefore is worth doing.
> > >
> > > > --- Editorial comments/nits:
> > > >
> > > > - Section 3.1, after RFC 5960 quotes:
> > > >
> > > >    [RFC5960] paragraph 3 requires that packets within a specific ordered
> > > >
> > > > Insert "Section 3.1.1.", before "paragraph 3" and likewise for
> "paragraph 6"
> > > > later in the same paragraph in the draft.
> > >
> > > GenArt review caught this.  Done.
> > >
> > > > - Section 3.2, requirement MP#3:
> > > >
> > > >    MP#3  It SHOULD be possible to insure that MPLS-TP LSPs will not be
> > > >
> > > > "insure" -> "ensure" or "assure".  Surely this draft does not envision
> > > > operators taking out insurance policies on MPLS TP LSP behavior ;-).
> > >
> > > Good catch.  I think "ensure" is the right word.
> > >
> > > > - Section 3.2, last paragraph on p.8:
> > > >
> > > >    An MPLS-TP LSP may not traverse multipath links on the path where
> > > >    MPLS-TP forwarding requirements cannot be met.
> > > >
> > > > "may not" is meaningless - I believe "MUST NOT" was intended here.
> > >
> > > I suggested SHOULD NOT above.
> > >
> > > > - Section 4, middle of 3rd para:
> > > >
> > > > Editorial suggestions for clarity:
> > > >
> > > > OLD
> > > >             For those LSP that are larger than component link
> > > >     capacity, their capacity are not integer multiples of convenient
> > > >     capacity increments such as 10Gb/s.
> > > > NEW
> > > >             For those LSPs that are larger than a component link
> > > >    capacity, the LSP capacities need not be (and often are not)
> > > >    integer multiples of convenient capacity increments such as 10Gb/s.
> > >
> > > Good suggestion.  GenArt review suggested adding "generally" making
> > > the statement read "their capacity are generally not integer
> > > multiples".  Your suggested rewording may be more clear.
> > >
> > > Also "10Gb/s" should be "10 Gb/s".
> > >
> > > > In particular, "are not" seems wrong, as the "integer multiples"
> > > > case is possible, so I changed "are not" to "need not be (and often
> > > > are not)" in the suggested new text.
> > > >
> > > > Thanks,
> > > > --David
> > > > ----------------------------------------------------
> > > > David L. Black, Distinguished Engineer
> > > > EMC Corporation, 176 South St., Hopkinton, MA  01748
> > > > [phone]             FAX: [phone]
> > > > [e-mail]        Mobile: [phone]
> > > > ----------------------------------------------------
> > >
> > > Thanks for the review and for pointing out lack of clarity due to
> > > underlying assumptions about the use of RFC 4206 (and perhaps use of
> > > RFC 4201 as well for large MPLS LSP over MPLS-TP).
> > >
> > > Sometimes it is too easy to forget that not all readers are going to
> > > be as steeped in MPLS "as deployed today" details as a relatively
> > > small number of long time MPLS participants are.
> > >
> > > Adding a few paragraphs to the introduction and a few concluding
> > > paragraphs may significantly improve clarity for people who have not
> > > been following MPLS closely for a decade or more without requiring
> > > them to read the references and figure out usage details themselves.
> > >
> > > Please let me know how you feel about the suggested additions to the
> > > document.
> > >
> > > Curtis
>