Re: [mpls] MPLS-RT review of draft-villamizar-mpls-multipath-use

[Curtis Villamizar <[e-mail]>]

Dave,

When explicitly referencing the requirements, there was enough to
create a small subsection (3.1).  So now section 3 is:

   3.  MPLS as a Server Layer for MPLS-TP
     3.1.  MPLS-TP Forwarding and Server Layer Requirements
     3.2.  Methods of Supporting MPLS-TP client LSPs over MPLS

In section 3.2 I now use the phrase "MPLS-TP forwarding requirements"
and refer back to section 3.1 rather than using either the phrase
"MPLS-TP OAM fate-sharing requirements" or the phrase "MPLS-TP packet
ordering requirements" in section 3.2.

I did some rewording and additions to improve clarity and be more
specific in section 3, as requested in general comments by reviewers.
I'll send annotated diffs in a little while so that you and other
MPLS-RT expert reviewers can make sure all comments (so far) are
addressed and the changes are to your liking.

Regards,

Cuertis


In message <[e-mail]>
David Allan I writes:

Hi Curtis

Thanks for the reply, and yes LM has a strict ordering constraint, while FM wants fate sharing, I was brain-farting on the PM/LM component of OAM when reading it and thinking FM which is the usual discussion on this list....

So now I understand the motivation for the way you had phrased the text. Not spreading an individual TP LSP out across multiple links actually satifies both so IMO there is a dual motivation for the requirement. If referring to the OAM ordering requirement, I'd suggest calling it "the strict packet ordering requirement that TP Performance monitoring requires", so it cannot be confused with simply in order delivery of OAM PDUs in isolation; the OAM PDU's relative position with respect to the interleaved real traffic actually matters.

I also have no issue with the digression on capacity allocation determinism and SLA monitoring vs. actual behavior. Other that that I would observe actual mileage observed by the different modes of operation depends on the actual traffic profile, e.g. large number of small flows vs. small number of large flows....

Cheers
D

-----Original Message-----
From: Curtis Villamizar [mailto:[e-mail]] 
Sent: Wednesday, January 23, 2013 2:19 PM
To: David Allan I
Cc: [e-mail]; [e-mail]; Martin Vigoureux; [e-mail]; Loa Andersson
Subject: Re: MPLS-RT review of draft-villamizar-mpls-multipath-use


In message <[e-mail]>
David Allan I writes:
> 
> HI Curtis: 
>  
> Some snipping to help readability and replies prefixed by [Dave]
>  
> <snipped>
>  
> If it is unclear to you, then the document is not sufficiently clear 
> and needs fixing.  See below.
>  
> [Dave] Great....
>  
> > Nit:
> >  
> > Section 1 para 2: Refers to MPLS-TP packet ordering as the constraint. 
> > I'm a bit confused by this, any aggregate that shares an ordering 
> > constraint does not get spread across parallel links while a set of 
> > traffic COULD be in theory load spread across a set of MPLS-TP paths 
> > so long as individual ordering constraints were respected. Is this 
> > text document really discussing midbox spreading out of load that 
> > respects ordering constraints but would mean MPLS-TP OAM no longer 
> > fate shared?  Should be reworded if so. Clarified otherwise.
>  
> Section 1 paragraph 2 contains:
>  
>    RFC 5654 requirement 33 requires the capability to carry a client
>    MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer
>    [RFC5654].  This is possible in all cases with one exception.  When
>    an MPLS LSP exceeds the capacity of any single component link it
>    may be carried by a network using multipath techniques, but may not
>    be carried by an MPLS-TP LSP due to the inherent MPLS-TP capacity
>    limitation imposed by MPLS-TP OAM packet ordering constraints.
>  
> The sentence in question is:
>  
>    When an MPLS LSP exceeds the capacity of any single component link
>    it may be carried by a network using multipath techniques, but may
>    not be carried by an MPLS-TP LSP due to the inherent MPLS-TP
>    capacity limitation imposed by MPLS-TP OAM packet ordering
>    constraints.
>  
> Perhaps I could clarify this by changing "carried by an MPLS-TP LSP"
> to "carried by a single MPLS-TP LSP".
>  
> The "MPLS-TP OAM packet ordering constraints" is the "fate sharing"
> which is the reason that MPLS-TP prohibits using ECMP (aka multipath, 
> though ECMP is only one form of multipath) with MPLS-TP.  This is why 
> an MPLS LSP (client layer) with capacity greater than a component link 
> cannot be carried over a *single* MPLS-TP LSP.
>  
> Since this is explained in later sections I could make the 
> substitution suggested above or I could drop the last sentence and 
> mention that limitation later.
>  
> [Dave] the sentence I had a problem with was "but may not be carried 
> by an MPLS-TP LSP due to the inherent MPLS-TP capacity limitation 
> imposed by MPLS-TP OAM packet ordering constraints."

The phrase you are objecting to is "packet ordering constraints".

Carlos suggested that I reference or quote RFC 5960 which would clear up exactly what the dataplane requirements are.

If direct LM OAM is supported, then there is a strict packet reordering requirement (can't reorder the LM OAM packet relative to payload packets or accuracy suffers) so maybe I need to cite RFC 6374 Section 2.9.4 "Equal Cost Multipath" and Section 4.2.10 "Message Loss and Packet Misorder Conditions".

> It is not an ordering constraint, it is a fate sharing constraint. It 
> read fairly strangely to me as to what the problem was you were trying 
> to expose. IMO the TP requirements are focused on both determinism of 
> commitment of network resources for network planning & SLA purposes, 
> and ability to fully and reliably instrument the connectivity. IMO 
> multipath violates the latter, whether it violates the former would be 
> a topic of debate unresolvable in the reality of LAG. LAG simply 
> scoping the domain of uncertainty for OAM purposes to a link, and 
> requiring extra link OAM and inheritance...

This is an aside, but since you brought this up I'll respond.

Begin aside.

You made two implied assertions.  1) MPLS-TP is more deterministic, 2) MPLS-TP is necessary to reliably instrument the connectivity.

A circuit swtiched (or MPLS-TP) network is more deterministic with respect to how much capacity is allocated.  When the capacity in question is a large aggregate of traffic, the packet switched (or
MPLS) network is more deterministic with respect to individual customer experience.  For example, if many LSP travel across a core and if one city pair is exceeding its expected capactiy but others are slidghtly underutilizing their expected capactiy, then in the circuit switched or MPLS-TP network individual flows for that city pair experience congestion, whereas in a packet switched or MPLS network, capacity is shared and therefore no congestion is experienced.

In MPLS core networks the SLA traffic is a fraction of traffic and does fine with little more than TC marking and diffserv treatment.

The determinism you describe may be an issue in metro networks, such as where mobile backhaul is mixed with pedestrian Internet traffic and capacity is scarse.

Kireeti Kompella and other wrote a paper about two years ago on exactly this topic and it might help if I dug up the reference and included in the document.

The latter point regarding reliably instrumenting the connectivity is valid.  But if the benefit of instrumenting is for a minority of the traffic (SLA traffic) and the cost is a reduction in network efficiency for the vast majority of traffic (plain old Internet), then there is a substantial increase in cost to provide the instrumented service and then it may not be worth it.  This is why MPLS-TP carried over MPLS is of interest.

End aside.

> [Dave] To make the discussion short, replacing "OAM packet ordering 
> constraints" with "OAM fate sharing constraints" should be sufficient.

In practice the "OAM fate sharing constraints" and "OAM packet ordering constraints" are almost one in the same where LAG or ECMP are the cause of the misordering.  LM OAM requires strict packet ordering, not just fate sharing.

I'll add the references to RFC 5960 and RFC 6374 and let those RFCs state the requirements.

> > Main issue:
> >  
> > Section 3: I think this needs a bit of work, it starts to discuss 
> > tractable solutions but IMO is technically incomplete. IF an entropy 
> > label and ELI (where all traffic associated with the MPLS_TP LSP 
> > shares a common randomly selected entropy label value, which is not
> > stated) are the only mechanisms of ensuring proper treatment (with 
> > all transit nodes implementing entropy and ELI processing such that 
> > seeking sources of entropy is capped for such LSPs), and the ingress 
> > node (which by some means SHOULD be able to know it is a TP LSP as 
> > it has layer visibility) and the egress node can strip entropy and 
> > ELI then a solution exists for proper fate sharing and ordering of a 
> > TP LSP over MPLS. It is IMO not quiet eluciated completely.
>  
> Your paragraph above describes the solution, but then says "It is IMO 
> not quiet eluciated completely."
>  
> [Dave] Well I know you and I know the solution, but someone newer to 
> the subject would not necessarily be able to infer it from the 
> document in its current state.

OK.  Fair enough.

> So lets parse your paragraph and see what the draft is missing:
>  
> > Section 3: I think this needs a bit of work, it starts to discuss 
> > tractable solutions but IMO is technically incomplete.
>  
>     OK ...
>  
> > IF an entropy label and ELI (where all traffic associated with the 
> > MPLS_TP LSP shares a common randomly selected entropy label value, 
> > which is not stated)
>  
>     You mention "IF an entropy label and ELI (where all traffic
>     associated with the MPLS_TP LSP shares a common randomly selected
>     entropy label value, which is not stated)".  Regarding the "which
>     is not stated" part of that phrase I can change the following
>     paragraph:
>  
>       OLD:
>  
>         MPLS-TP LSP can be carried as client LSP within an MPLS server
> 	LSP if an Entropy Label Indicator (ELI) and entropy label (EL)
> 	is added after the server layer LSP label(s) in the label
> 	stack, just above the MPLS-TP LSP label entry [RFC6790].
> 	[...]
>  
>       NEW:
>  
>         MPLS-TP LSP can be carried as client LSP within an MPLS server
>         LSP if an Entropy Label Indicator (ELI) and Entropy Label (EL)
>         is added after the server layer LSP label(s) in the label
>         stack, just above the MPLS-TP LSP label entry [RFC6790].  The
>         value of EL can be randomly selected at LSP setup time and the
>         same EL value used for all packets of the MPLS-TP LSP.  [...]
>  
> [Dave] That edit works for me.

Its in the xml source and will be in the next iteration.

>     One sentence is added to the end.  (and Entropy Label is capitalized).
>  
> > are the only mechanisms of ensuring proper treatment
>  
>     Actually link-bundling is mentioned as another mechanism.
>  
> > (with all transit nodes implementing entropy and ELI processing such 
> > that seeking sources of entropy is capped for such LSPs),
>  
>     The requirement to terminate search for additional entropy below
>     the EL is a requirement of RFC 6790.  The intent is to say that
>     non-compliant midpoint LSR would in this case cause packet order
>     problems where in MPLS only (no TP carried) use of RFC 6790, those
>     non-compliant midpoint LSR are tolerable as long as they can get
>     enough entropy from the traffic to adequately load split.
>  
> [Dave] Given the relative newness of 6790, I'm assuming some statement 
> to the effect that this solution is not necessarily backwards 
> compatible with existing deployments needs to be said.

OK.  Most of the draft-villamizar-mpls-multipath-extn is about fixing the limitation that MPLS-TP LSP need to be identified in signaling and that the ingress needs to know which links can't support MPLS-TP requirements.

This text addresses the need to know which links can't support MPLS-TP.

   MP#4  Where an RSVP-TE control plane is used, it MUST be possible
   	 for an ingress LSR which is setting up an MPLS-TP or MPLS LSP
   	 to determine at CSPF time whether a link or MPLS PSC LSP
   	 within the topology can support the MPLS-TP requirements of
   	 the LSP.

I'll explicitly mention that legacy LAG and legacy ECMP that do not support ELI and EL are among the "link or MPLS PSC LSP within the topology" that cannot "can support the MPLS-TP requirements of the LSP."  Without protocol extensions, the ingress can still know this if color (administrative attributes in RFC 3209) is used to identify these links.  Its worth mentioning this, so thanks for bringing it up.

So far all I say is:

   Requirement MP#4 can be supported using administrative attributes.
   Administrative attributes are defined in [RFC3209].  Some
   configuration is required to support this.

I'll preceed that with:

   Links or MPLS LSP which traverse LAG or ECMP with adjacent LSR that
   do not support [RFC6790] cannot support MPLS-TP requirements.  This
   is the reason for including requirement MP#4.  The ingress of an
   MPLS-TP LSP must be able to avoid these links or MPLS LSP.

I think this addresses your concern.  [btw- sometimes it is better to state what is obvious to some readers than confuse the others readers, so I agree that this is an improvement.]

> > and the ingress node (which by some means SHOULD be able to know it 
> > is a TP LSP as it has layer visibility)
>  
>     That is discussed in the following paragraph:
>  
>        There is currently no signaling mechanism defined to support
>        requirement MP#1.  In the absense of a signaling extension,
>        MPLS-TP can be identified through some form of configuration,
>        such as configuration which provides an MPLS-TP compatible
>        server layer to all LSP arriving on a specific interface or
>        originating from a specific set of ingress LSR.  Alternately an
>        MPLS-TP LSP can be created with and Entropy Label Indicator
>        (ELI) and entropy label (EL) below the MPLS-TP label [RFC6790].
>  
>     Obviously the EL can be added at the MPLS-TP ingress LSR.  This
>     paragraph is considering the case where a set of MPLS-TP LSR are
>     not EL capable but also do not have multipath enabled, but
>     multipath is enabled elsewhere (ie: the core where lots of traffic
>     needs to get aggregated into a smaller number of LSP).  The lack
>     of a signaling extension is noted as a (fixable) limitation.  The
>     fix comes in a later draft (draft-villamizar-mpls-multipath-extn).
>  
> > and the egress node can strip entropy and ELI
>  
>     That is a hard requirement of RFC 6790 for an LSR claiming to
>     support RFC 6790.
>  
> > then a solution exists for proper fate sharing and ordering of a TP 
> > LSP over MPLS.
>  
>      OK ... agreed.
>  
> > It is IMO not quiet eluciated completely.
>  
>      You seem to have figured out all of the details.
>  
> If there are additional clarifications that are needed besides the 
> ones suggested above, please point out what details of the solution 
> are still not clear in the existing text.
>  
> > If I'm unclear on any of the above, am happy to discuss...
>  
> Thanks.  Please let me know if the above clarifications are sufficient 
> or whether additional clarifications are needed.
>  
> [Dave] I think with the change of ordering constraint to fate sharing 
> constraint in section 1, the EL "fixed value" addition and some 
> disclaimer about 6790 support in the MPLS network, I would be happy. 
> I'm clearer on what you were saying.

If the changes above don't adequately address your concerns then please say so.

> Best
> Dave

Cheers,

Curtis