RE: OPS-Dir review of draft-ietf-mpls-multipath-use-03

"Black, David" <david.black@emc.com> Wed, 22 January 2014 00:02 UTC

Return-Path: <david.black@emc.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D7BB81A0260; Tue, 21 Jan 2014 16:02:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.536
X-Spam-Level:
X-Spam-Status: No, score=-4.536 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_I_LETTER=-2, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.535, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hd7k7sLMCekh; Tue, 21 Jan 2014 16:02:44 -0800 (PST)
Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) by ietfa.amsl.com (Postfix) with ESMTP id 3E6911A0225; Tue, 21 Jan 2014 16:02:43 -0800 (PST)
Received: from maildlpprd04.lss.emc.com (maildlpprd04.lss.emc.com [10.253.24.36]) by mailuogwprd01.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s0M02fZN012270 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jan 2014 19:02:42 -0500
X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s0M02fZN012270
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1390348963; bh=xzCObmqSS8B84Tukb1D8ud+K38U=; h=From:To:CC:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=QBb4PRc37bQuK1QBOFetFFlDVKVJkFLBL6Q1SPOdkewpuQy2L6kWxPlRTC65YfsF9 wqyYwC6+d+n4vWQPisOXrJJbGdyWWu+1Ml6jhQrmQLtYJDYZW1SZM9VeryntROcPIB T8d6VRPjPj2Kaz1EKry1ZAjWlmDu1vhcETrUa2kM=
X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s0M02fZN012270
Received: from mailusrhubprd53.lss.emc.com (mailusrhubprd53.lss.emc.com [10.106.48.18]) by maildlpprd04.lss.emc.com (RSA Interceptor); Tue, 21 Jan 2014 16:02:36 -0800
Received: from mxhub03.corp.emc.com (mxhub03.corp.emc.com [10.254.141.105]) by mailusrhubprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s0M02YPi002932 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 21 Jan 2014 19:02:35 -0500
Received: from mx15a.corp.emc.com ([169.254.1.107]) by mxhub03.corp.emc.com ([10.254.141.105]) with mapi; Tue, 21 Jan 2014 19:02:34 -0500
From: "Black, David" <david.black@emc.com>
To: "curtis@ipv6.occnc.com" <curtis@ipv6.occnc.com>
Date: Tue, 21 Jan 2014 19:02:33 -0500
Subject: RE: OPS-Dir review of draft-ietf-mpls-multipath-use-03
Thread-Topic: OPS-Dir review of draft-ietf-mpls-multipath-use-03
Thread-Index: Ac8W3kd3cn8I8nSsRQ6C7ksWTmbcMwAHK7fQ
Message-ID: <8D3D17ACE214DC429325B2B98F3AE712026F047CC2@MX15A.corp.emc.com>
References: Your message of "Sun, 19 Jan 2014 21:05:17 -0500." <8D3D17ACE214DC429325B2B98F3AE712026F047819@MX15A.corp.emc.com> <201401211923.s0LJNRPm064842@maildrop2.v6ds.occnc.com>
In-Reply-To: <201401211923.s0LJNRPm064842@maildrop2.v6ds.occnc.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Sentrion-Hostname: mailusrhubprd53.lss.emc.com
X-RSA-Classifications: DLM_1, public, GIS Solicitation
Cc: "draft-ietf-mpls-multipath-use.all@tools.ietf.org" <draft-ietf-mpls-multipath-use.all@tools.ietf.org>, "ops-dir@ietf.org" <ops-dir@ietf.org>, "ietf@ietf.org" <ietf@ietf.org>, "Black, David" <david.black@emc.com>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Jan 2014 00:02:49 -0000

Curtis,

> Thanks for the review and for pointing out lack of clarity due to
> underlying assumptions about the use of RFC 4206 (and perhaps use of
> RFC 4201 as well for large MPLS LSP over MPLS-TP).
>
> Sometimes it is too easy to forget that not all readers are going to
> be as steeped in MPLS "as deployed today" details as a relatively
> small number of long time MPLS participants are.
>
> Adding a few paragraphs to the introduction and a few concluding
> paragraphs may significantly improve clarity for people who have not
> been following MPLS closely for a decade or more without requiring
> them to read the references and figure out usage details themselves.
>
> Please let me know how you feel about the suggested additions to the
> document.

Most of them look good - details below.  Many thanks for the prompt and
comprehensive response.  I have a few more suggestions in what follows
- anything that I don't comment on below is fine with me.

-- Introduction

> There is an underlying assumption in the draft that the reader
> understands that MPLS-in-MPLS exists today, most commonly but not
> exclusively in the form of RFC 4206 hierarchy.  RFC 4206 provides a
> means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some
> level of Packet Switch Capable (PSC) hierarchy.  It is also possible
> to make use of the management plane, using CLI or NETCONF or the MPLS
> cross connect MIB to program a multiple label MPLS push operation.
>
> If a very experienced reviewer could miss that, then the introduction
> could be improved by better stating assumptions.  It is also the case
> that many participants in MPLS, particularly those primarily
> interested in MPLS-TP, began participation long after RFC 4206 was
> written and would benefit by better stated assumptions in the intro.
>
> I suggest that the following paragraphs be added to the introduction.

The suggested new paragraphs will definitely improve the introduction.

-- [A] Summary text in Introduction appears to be wrong.

> > Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above
> > exception appears to concern the reverse, and Section 4 on carrying MPLS
> > over MPLS-TP allows for MPLS LSPs that exceed component link capacity.
> > Was the exception intended to be for an MPLS-TP LSP that exceeds the
> > capacity of any single component link?

[... snip ...]

> To answer your question directly: the exception is a very large MPLS
> LSP can't be carried by a single MPLS-TP LSP.  The solution is to use
> multiple MPLS-TP LSP.  The MPLS-TP LSR need not be not "aware" that
> this is occurring.

It looks like  my confusion stemmed from the original text only
discussing only what won't work and referencing a section that
concerns a different encapsulation order than the one that causes
the problem.  Including the "MAY NOT" -> "SHOULD NOT" change, I suggest:

OLD
   When an MPLS LSP
   exceeds the capacity of any single component link it MAY be carried
   by a network using multipath techniques, but MAY NOT be carried by a
   single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
   imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
   fate sharing constraints and MPLS-TP LM OAM packet ordering
   constraints (see Section 3.1).
NEW
   When an MPLS LSP
   exceeds the capacity of any single component link it MAY be carried
   by a network using multipath techniques, but SHOULD NOT be carried by a
   single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
   imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
   fate sharing constraints and MPLS-TP LM OAM packet ordering
   constraints (see Section 3.1).  Instead, multiple MPLS-TP LSPs SHOULD
   be used to carry a large MPLS LSP (see Section 4).

-- [B] Section 3.2 needs a summary.

The proposed summary looks good, and helps with [C], but the phrase
"access and edge" needs qualification in the following sentence:

   However access and edge is the least likely to support
   RFC 6790 in the near term.

"access and edge" what? e.g., areas of the network? LSRs?

-- [C] Entropy label "sandwich" seems peculiar and at least needs an explanation.

> The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following
> paragraph.
>
>    Alternately, the need for requirement MP#1 can be eliminated if
>    every MPLS-TP LSP created by an MPLS-TP ingress makes use of an
>    Entropy Label Indicator (ELI) and Entropy Label (EL) below the
>    MPLS-TP label [RFC6790].  This would require that all MPLS-TP LSR
>    in a deployment support Entropy Label, which may render it
>    impractical in many deployments.
>
> Please not the "Alternately" at the beginning and "impractical in many
> deployments" at the end.
>
> The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs
> above that but assumes knowledge of RFC 6790.  I think the following
> change within this paragraph would clear things up:

That change is helpful, but my "sandwich" concern was actually raised
by text two paragraphs further down:

   MPLS-TP LSPs can be carried as client LSPs within an MPLS server LSP
   if an Entropy Label Indicator (ELI) and Entropy Label (EL) is pushed
   below the server layer LSP label(s) in the label stack, just above
   the MPLS-TP LSP label entry [RFC6790].

Read after the "Alternately" paragraph, this text appeared to suggest
a "sandwich" that would have an ELI/EL pair both above and below the
MPLS-TP label.  That was clearly not intended, so I suggest inserting
"When Entropy Label Indicators (ELIs) and Entropy Labels (ELs) are not
applied by MPLS-TP ingresses," before the start of the first sentence
above, so that the result is:

   When Entropy Label Indicator (ELIs) and Entropy Labels (ELs) are
   not applied by MPLS-TP ingresses, MPLS-TP LSPs can be carried as
   client LSPs within an MPLS server LSP if an ELI and EL are pushed
   below the server layer LSP label(s) in the label stack, just above
   the MPLS-TP LSP label entry [RFC6790].

-- [D] Discussion of multipath impacts on MPLS OAM

> > Section 3.2 discusses OAM, and contains this important
> > requirement for OAM traffic:
> >
> >    An Entropy Label must be used to insure that all of the
> >    MPLS-TP payload and OAM traffic are carried on the same component,
> >    except during very infrequent transitions due to load balancing.
> >
> > [D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in
> > Section 4; that should be added.
>
> There is no impact on MPLS OAM traffic.  There are three cases to
> consider.
>
>   1. Where MPLS is carried over a single MPLS-TP all traffic flows on
>      one link.  MPLS OAM is unaffected and need not use multipath
>      support in LSP Ping.
>
>   2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP
>      LSP is carried over one link thanks to the fixed EL value.
>      MPLS-TP OAM is unaffected.
>
>   3.  Where MPLS is carried over MPLS (an existing case) or over
>       multiple MPLS-TP, the multipath support in LSP Ping (as
>       imperfect as it may already be) is used and LSP Ping (RFC 4379)
>       operation is unaffected.
>
> If needed something can be added to state the above (maybe with less
> informal wording).  What is obvious to some people may not be obvious
> to everyone.  Briefly stating the almost obvious is usually not a bad
> thing.

Points 1-3 are useful explanation as complement to the Entropy label
requirement to correctly handle OAM traffic for the other encapsulation.

I like your suggestion to add this text as part of a new concluding
Section 5, and the rest of the proposed conclusion section text looks
good.

Thanks,
--David

> -----Original Message-----
> From: Curtis Villamizar [mailto:curtis@ipv6.occnc.com]
> Sent: Tuesday, January 21, 2014 2:23 PM
> To: Black, David
> Cc: ops-dir@ietf.org; ietf@ietf.org; draft-ietf-mpls-multipath-
> use.all@tools.ietf.org
> Subject: Re: OPS-Dir review of draft-ietf-mpls-multipath-use-03
>
>
> In message <8D3D17ACE214DC429325B2B98F3AE712026F047819@MX15A.corp.emc.com>
> "Black, David" writes:
> >
> > I have reviewed this document as part of the operations directorate's
> > ongoing effort to review all IETF documents being processed by the
> > IESG.  These comments were written primarily for the benefit of the
> > operations area directors.  Document editors and WG chairs should treat
> > these comments just like any other last call comments.
> >
> > Summary:
> > This draft is on the right track but has open issues, described in the
> review.
> >
> > I apologize for this review showing up a couple of days after the end of
> > IETF Last Call.
> >
> > This draft discusses considerations for what are effectively MPLS-in-MPLS
> > tunnels for multipathing - an LSP carries another LSP, with the inner
> > (client) LSPs being distributed across multiple outer (server) LSPs that
> > use different paths.  This draft is motivated by considerations specific
> > to MPLS-TP that complicate multipathing when one of the two MPLS instances
> > is MPLS-TP.
>
> Skipping to the discussion quite a ways down, there seems to be some
> confusion about what is being proposed here.  While MPLS-in-MPLS is
> being used, no new form of MPLS-in-MPLS is being proposed.
>
> There is an underlying assumption in the draft that the reader
> understands that MPLS-in-MPLS exists today, most commonly but not
> exclusively in the form of RFC 4206 hierarchy.  RFC 4206 provides a
> means to advertise forward adjacencies (FA) where MPLS-in-MPLS at some
> level of Packet Switch Capable (PSC) hierarchy.  It is also possible
> to make use of the management plane, using CLI or NETCONF or the MPLS
> cross connect MIB to program a multiple label MPLS push operation.
>
> If a very experienced reviewer could miss that, then the introduction
> could be improved by better stating assumptions.  It is also the case
> that many participants in MPLS, particularly those primarily
> interested in MPLS-TP, began participation long after RFC 4206 was
> written and would benefit by better stated assumptions in the intro.
>
> I suggest that the following paragraphs be added to the introduction.
>
>    MPLS LSPs today are able to function as a server layer and carry
>    client MPLS LSPs.  When control plane signaling is used, forwarding
>    adjacency (FA) advertisements are used to inform the set of LSR of
>    Packet Switching Capable (PSC) LSP within the MPLS topology
>    [RFC4206].  Client MPLS LSP at a higher layer (lower PSC number)
>    may signal their intention to use PSC LSP as hops in the RSVP-TE
>    Explicit Route Object (ERO).  LSR with no explicit support for RFC
>    4206 see the PSC LSP as ordinary links and therefore use them.
>
>    An MPLS LSP that has been set up using RSVP-TE appears to its
>    ingress LSR as a viable IP next hop to a distant LSR.  If LDP is
>    used and bidirectional RSVP-TE LSP connectivity is available, then
>    LDP signaling can be set up among the RSVP-TE LSP endpoints and LDP
>    can make use of the RSVP-TE LSP as an LDP hop.  This is another
>    form of existing MPLS-in-MPLS use.
>
>    MPLS LSPs may also make use of hierarchy that is configured through
>    the management plane rather than signaled using RSVP-TE.
>
>    These existing forms of MPLS-in-MPLS may traverse multipath hops
>    such as Ethernet LAG [IEEE802.1AX] or MPLS Link Bundling [RFC4201].
>    MPLS-TP brings with it a new set of requirements not considered in
>    past deployments of the various forms of MPLS-in-MPLS where
>    multipath was in use.  This document merely discusses use of
>    existing forwarding and protocol mechanisms that can support the
>    case where either the client layer LSPs or the server layer LSPs
>    are MPLS-TP and where multipath is used.
>
> If there are no objections I will add these paragraphs.
>
> Now on to the specific comments.
>
> > I consider the following to be open issues that need attention (each
> > letter is used to tag the issue in the text that follows - [B] occurs
> > multiple times, as there are multiple aspect to that issue):
> >
> > [A] Summary text in Introduction appears to be wrong.
> > [B] Section 3.2 needs a summary.
> > [C] Entropy label "sandwich" seems peculiar and at least needs an explanation.
> > [D] Discussion of multipath impacts on MPLS OAM should be added to Section
> 4.
> >
> > --- Comments on draft text
> >
> > [A] Section 1 (Introduction) 2nd paragraph:
> >
> >    RFC 5654 requirement 33 requires the capability to carry a client
> >    MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer [RFC5654].
> >    This is possible in all cases with one exception.  When an MPLS LSP
> >    exceeds the capacity of any single component link it MAY be carried
> >    by a network using multipath techniques, but MAY NOT be carried by a
> >    single MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
> >    imposed by MPLS-TP Operations, Administration, and Maintenance (OAM)
> >    fate sharing constraints and MPLS-TP LM OAM packet ordering
> >    constraints (see Section 3.1).
> >
> > Section 3.1 concerns carrying MPLS-TP over MPLS, whereas the above
> > exception appears to concern the reverse, and Section 4 on carrying MPLS
> > over MPLS-TP allows for MPLS LSPs that exceed component link capacity.
> > Was the exception intended to be for an MPLS-TP LSP that exceeds the
> > capacity of any single component link?
>
> There is no problem today with MPLS LSP carried by MPLS-TP LSP except
> the existing case where MPLS LSP capacity is too big.  For example, a
> 500 Gb/s LSP is possible today carried over N x 100 Gb/s or N x 10
> Gb/s multipath links but could not be carried over a single MPLS-TP
> LSP which would be limited to a single 100 Gb/s or 10 Gb/s link.  So
> the exception is where the MPLS LSP is too big to fit in any possible
> single MPLS-TP LSP.
>
> To answer your question directly: the exception is a very large MPLS
> LSP can't be carried by a single MPLS-TP LSP.  The solution is to use
> multiple MPLS-TP LSP.  The MPLS-TP LSR need not be not "aware" that
> this is occurring.
>
> > Also, "MAY NOT" is not an RFC 2119 term (for good reason - it's equivalent
> > to "MAY OR MAY NOT").  I think "cannot" was intended here, but "MUST NOT"
> > is another possibility.
>
> Good point.  I suggest we replace the MAY NOT with SHOULD NOT.  Many
> people would insist that it should be MUST NOT, but in reality
> networks SHOULD NOT be congested but sometimes they are.
>
> > [B] Section 3.2 contains a lot of useful details, but it could really use
> > a summary at the end on the options and recommendations for meeting the
> > four requirements (this summary could be a new Section 3.3).  It's a bit
> > difficult to discern what's going on, as I had to read this section more
> > than once in order to discern the following:
>
> OK.  A Summary paragraph can be added.  See below.  Perhaps conclusion
> section can be added.  See far below.
>
> > [C] In Section 3.2, use of an MPLS entropy label to meet requirement MP#1
> > requires that it be "below the MPLS-TP label" whereas, to meet requirement
> > MP#2, the MPLS entropy label needs to be "just above the MPLS-TP LSP label".
> > This combination appears to suggest "sandwiching" the MPLS-TP label between
> > two entropy labels that effectively carry the same information (albeit,
> > applied and removed at different places in the network) - was that intended?
> > This is important, as there seems to be no reasonable alternative
> > to the entropy label for meeting requirement MP#2, and the draft's
> > alternative to the entropy label for meeting requirement MP#1 is "some form
> > of configuration" - the entropy label would seem to be preferred to that
> > based on my reading of this draft and the comment in RFC 5706 section 2.2
> > that "Anything that can be configured can be misconfigured."
>
> The "MPLS MPLS-TP ELI EL" sandwich is suggested in the following
> paragraph.
>
>    Alternately, the need for requirement MP#1 can be eliminated if
>    every MPLS-TP LSP created by an MPLS-TP ingress makes use of an
>    Entropy Label Indicator (ELI) and Entropy Label (EL) below the
>    MPLS-TP label [RFC6790].  This would require that all MPLS-TP LSR
>    in a deployment support Entropy Label, which may render it
>    impractical in many deployments.
>
> Please not the "Alternately" at the beginning and "impractical in many
> deployments" at the end.
>
> The other sandwich "MPLS ELI EL MPLS-TP" is described two paragraphs
> above that but assumes knowledge of RFC 6790.  I think the following
> change within this paragraph would clear things up:
>
>  OLD
>
>    MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI)
>    and Entropy Label (EL) are added to the label stack [RFC6790].  For
>    those client LSP that are MPLS-TP LSP, a single EL value must be
>    chosen.  For those client LSP that are MPLS LSP, per packet entropy
>    below the top label must, for practical reasons, be used to
>    determine the entropy label value.
>
>  NEW
>
>    MPLS server layer LSP ingress LSR an Entropy Label Indicator (ELI)
>    and Entropy Label (EL) are added to the label stack by the midpoint
>    LSR for the client MPLS-TP LSP, at the ingress of the MPLS LSP
>    [RFC6790].  For those client LSP that are MPLS-TP LSP, a single EL
>    value must be chosen.  For those client LSP that are MPLS LSP, per
>    packet entropy below the top label must, for practical reasons, be
>    used to determine the entropy label value.  The resulting label
>    stack contains the server MPLS LSP label, ELI, EL and the client
>    LSP label.
>
> The above rewording is simply more explicit about the label stack
> order and which LSR puts the ELI and EL in the label stack.
>
> > [B] A summary of requirements for implementation and deployment at the end
> > of Section 3.2 would have made this much more obvious.  That should be
> > provided in addition to addressing the above question about duplicate use
> > of the MPLS entropy label.
>
> The following summary can be added to the end of Section 3.2:
>
>    Two methods of adding an Entropy Label are described above.  The
>    MPLS-TP ingress must have a means to determine which links can
>    support MPLS-TP in selecting a path (MP#4).  Administrative
>    attributes can satisfy that requirement.  If the MPLS-TP LSR is
>    capable of adding ELI/EL to the label stack, this method is
>    preferred.  However access and edge is the least likely to support
>    RFC 6790 in the near term.  For portions of the topology where an
>    MPLS-TP is carried within a server layer MPLS LSP the ingress of
>    the server layer MPLS LSP can add ELI/EL using a fixed EL value per
>    client LSP, except those known not to require MPLS-TP treatment.
>    There are numerous ways to determine which client LSP are MPLS-TP
>    LSP and which are not.  While this determination is out of scope
>    and will vary among deployments, configuration or the presence of
>    specific attribute affinities in RSVP-TE signaling are among the
>    likely means to do so.
>
> If there are no objections I will add the above paragraph.
>
> > --- Relevant Q&A based on RFC 5706 Appendix A
> >
> > I'm omitting management concerns, as both MPLS and MPLS-TP have extensive
> > management infrastructure, and this draft's notion of stacking LSPs so
> > that one instance of MPLS (or MPLS-TP) can be a server to (i.e., carry)
> > another is not novel to this draft.
> >
> > A.1.1.  Has deployment been discussed?
> >
> > [B] Yes, although Section 3.2 is weak on deployment requirements (some of
> > which are implementation requirements) and could use a summary.
>
> The use of RFC 4206 is assumed throughout the discussion of signaling
> in section 3.2.  This assumption is made more clear by suggested
> additions to the introduction.
>
> > A.1.2.  Has installation and initial setup been discussed?
> >
> > No, although because this draft reuses existing functionality in different
> > configurations, this really reduces to the A.1.9 configuration question
> > below.  In practice, quite a bit of operator knowledge and insight is
> > required to get initial setup right, e.g., as traffic engineering LSPs to
> > avoid congestion problems (needed to meet requirement MP#3) requires
> > significant knowledge of network structure and expected traffic flows.
> > This should be obvious to anyone who works w/MPLS, but it's probably
> > worth stating for completeness.
>
> FA advertisements defined in RFC 4206 make it possible for ingress to
> signal viable paths.  Either the use of Ethernet Link Aggregation or
> MPLS Link bundling using the all-ones component link (RFC 4201) allows
> the total capacity of a multipath to be advertised.  Meeting the
> requirement to balance traffic over the component links in a multipath
> (MP#3) is then a local matter and requires no further signaling
> extensions.
>
> > A.1.6.  Have suggestions for verifying correct operation been discussed?
> >
> > Yes and no.  Section 3.2 discusses OAM, and contains this important
> > requirement for OAM traffic:
> >
> >    An Entropy Label must be used to insure that all of the
> >    MPLS-TP payload and OAM traffic are carried on the same component,
> >    except during very infrequent transitions due to load balancing.
> >
> > [D] OTOH, there is no discussion of OAM traffic for the MPLS LSP in
> > Section 4; that should be added.
>
> There is no impact on MPLS OAM traffic.  There are three cases to
> consider.
>
>   1. Where MPLS is carried over a single MPLS-TP all traffic flows on
>      one link.  MPLS OAM is unaffected and need not use multipath
>      support in LSP Ping.
>
>   2. Where MPLS-TP is carried over MPLS, all traffic for that MPLS-TP
>      LSP is carried over one link thanks to the fixed EL value.
>      MPLS-TP OAM is unaffected.
>
>   3.  Where MPLS is carried over MPLS (an existing case) or over
>       multiple MPLS-TP, the multipath support in LSP Ping (as
>       imperfect as it may already be) is used and LSP Ping (RFC 4379)
>       operation is unaffected.
>
> If needed something can be added to state the above (maybe with less
> informal wording).  What is obvious to some people may not be obvious
> to everyone.  Briefly stating the almost obvious is usually not a bad
> thing.  Please tell me if a small subsection on "Imopact on MPLS and
> MPLS-TP OAM" is needed.
>
> Note that the limitations of LSP Ping are currently being discussed in
> the MPLS WG in the context of extensions to LSP Ping to better support
> Entropy Label.  A few limitations not related to EL may get fixed in
> the process.
>
> > A.1.9.  Is configuration discussed?
> >
> > [B] Not much. Configuration is required to meet requirements MP#3 and MP#4,
> > and is one of the options for MP#1.  A summary of what has to be configured
> > as part of the summary at the end of Section 3.2 would be a good idea.
>
> Perhaps a worked example appendix could be added but I don't think
> this is common and may not be necessary with some of the
> clarifications suggested so far.
>
> > A.2.* [skipped, see above]
> >
> > A.3 Documentation
> >
> > There's quite a bit of operational material contained in this draft; a
> > separate section on operations and management considerations would not
> > be appropriate.
> >
> > There is a useful implementation status section which appears to imply
> > that MPLS-TP over MPLS is not currently deployable due to absence of
> > entropy label support, and because deployed MPLS equipment does not
> > generally support multipathing.  Nonetheless, data center experience
> > indicates significant value and widespread use of multipathing based
> > on link aggregation and ECMP; corresponding benefits can be expected
> > for use of multipathing in the Internet beyond data centers.
>
> If you would like I can add a conclusion section as Section 5 with
> these paragraphs:
>
>    MPLS equipment deployed in the core currently supports multipath.
>    For large service providers, core LSR must support some form of
>    multipath to be deployable.  Deployed MPLS access and edge
>    equipment is often oblivious to the use of multipath in the core.
>    It is expected that at least first generation MPLS-TP equipment
>    will be oblivious to the use of multipath in the core.  This first
>    generation MPLS-TP equipment is deployable in a core using
>    multipath, with no adverse impact to RSVP-TE signaling, if the edge
>    equipment can support administrative attributes (RFC 3209) and the
>    core equipment can support ELI/EL and put a per-LSP fixed EL value
>    on any LSP that indicates a particular attribute affinity or can
>    identify client MPLS-TP LSP through some other means.
>
>    There are no issues carrying MPLS over MPLS-TP, except when the
>    MPLS LSP is too big to be carried by a single MPLS-TP LSP.  Most
>    MPLS core equipment and some edge equipment can configure an MPLS
>    Link Bundle [RFC4201] over multiple component links where the
>    component links are themselves MPLS LSP.  This existing capability
>    can be used to carry large MPLS LSP and overcome the limited
>    capacity of any single server layer MPLS-TP LSP.
>
> If this conclusion section is added, then the three bullets above
> regarding impact on OAM can be put there.  The three bullets can be
> added as follows:
>
>    MPLS OAM and MPLS-TP OAM are unaffected in the following cases
>    proposed in this document:
>
>      1. Where MPLS is carried over a single MPLS-TP all traffic flows
>         on one link.  MPLS OAM is unaffected and need not use
>         multipath support in LSP Ping.
>
>      2. Where MPLS-TP is carried over MPLS, all traffic for that
>         MPLS-TP LSP is carried over one link thanks to the fixed EL
>         value.  MPLS-TP OAM is unaffected.
>
>      3.  Where MPLS is carried over MPLS (an existing case) or over
>          multiple MPLS-TP, the multipath support in LSP Ping is used
>          and LSP Ping (RFC 4379) operation is unaffected.
>
> I think adding the above as a conclusion section as Section 5 would
> improve clarity and therefore is worth doing.
>
> > --- Editorial comments/nits:
> >
> > - Section 3.1, after RFC 5960 quotes:
> >
> >    [RFC5960] paragraph 3 requires that packets within a specific ordered
> >
> > Insert "Section 3.1.1.", before "paragraph 3" and likewise for "paragraph 6"
> > later in the same paragraph in the draft.
>
> GenArt review caught this.  Done.
>
> > - Section 3.2, requirement MP#3:
> >
> >    MP#3  It SHOULD be possible to insure that MPLS-TP LSPs will not be
> >
> > "insure" -> "ensure" or "assure".  Surely this draft does not envision
> > operators taking out insurance policies on MPLS TP LSP behavior ;-).
>
> Good catch.  I think "ensure" is the right word.
>
> > - Section 3.2, last paragraph on p.8:
> >
> >    An MPLS-TP LSP may not traverse multipath links on the path where
> >    MPLS-TP forwarding requirements cannot be met.
> >
> > "may not" is meaningless - I believe "MUST NOT" was intended here.
>
> I suggested SHOULD NOT above.
>
> > - Section 4, middle of 3rd para:
> >
> > Editorial suggestions for clarity:
> >
> > OLD
> >             For those LSP that are larger than component link
> >     capacity, their capacity are not integer multiples of convenient
> >     capacity increments such as 10Gb/s.
> > NEW
> >             For those LSPs that are larger than a component link
> >    capacity, the LSP capacities need not be (and often are not)
> >    integer multiples of convenient capacity increments such as 10Gb/s.
>
> Good suggestion.  GenArt review suggested adding "generally" making
> the statement read "their capacity are generally not integer
> multiples".  Your suggested rewording may be more clear.
>
> Also "10Gb/s" should be "10 Gb/s".
>
> > In particular, "are not" seems wrong, as the "integer multiples"
> > case is possible, so I changed "are not" to "need not be (and often
> > are not)" in the suggested new text.
> >
> > Thanks,
> > --David
> > ----------------------------------------------------
> > David L. Black, Distinguished Engineer
> > EMC Corporation, 176 South St., Hopkinton, MA  01748
> > +1 (508) 293-7953             FAX: +1 (508) 293-7786
> > david.black@emc.com        Mobile: +1 (978) 394-7754
> > ----------------------------------------------------
>
> Thanks for the review and for pointing out lack of clarity due to
> underlying assumptions about the use of RFC 4206 (and perhaps use of
> RFC 4201 as well for large MPLS LSP over MPLS-TP).
>
> Sometimes it is too easy to forget that not all readers are going to
> be as steeped in MPLS "as deployed today" details as a relatively
> small number of long time MPLS participants are.
>
> Adding a few paragraphs to the introduction and a few concluding
> paragraphs may significantly improve clarity for people who have not
> been following MPLS closely for a decade or more without requiring
> them to read the references and figure out usage details themselves.
>
> Please let me know how you feel about the suggested additions to the
> document.
>
> Curtis