Re: [mpls] MPLS-RT review of draft-villamizar-mpls-multipath-use

[Curtis Villamizar <[e-mail]>]

In message <[e-mail]>
David Allan I writes:
> 
> HI Curtis: 
>  
> Some snipping to help readability and replies prefixed by [Dave]
>  
> <snipped>
>  
> If it is unclear to you, then the document is not sufficiently clear
> and needs fixing.  See below.
>  
> [Dave] Great....
>  
> > Nit:
> >  
> > Section 1 para 2: Refers to MPLS-TP packet ordering as the constraint. 
> > I'm a bit confused by this, any aggregate that shares an ordering 
> > constraint does not get spread across parallel links while a set of 
> > traffic COULD be in theory load spread across a set of MPLS-TP paths 
> > so long as individual ordering constraints were respected. Is this 
> > text document really discussing midbox spreading out of load that 
> > respects ordering constraints but would mean MPLS-TP OAM no longer 
> > fate shared?  Should be reworded if so. Clarified otherwise.
>  
> Section 1 paragraph 2 contains:
>  
>    RFC 5654 requirement 33 requires the capability to carry a client
>    MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer
>    [RFC5654].  This is possible in all cases with one exception.  When
>    an MPLS LSP exceeds the capacity of any single component link it
>    may be carried by a network using multipath techniques, but may not
>    be carried by an MPLS-TP LSP due to the inherent MPLS-TP capacity
>    limitation imposed by MPLS-TP OAM packet ordering constraints.
>  
> The sentence in question is:
>  
>    When an MPLS LSP exceeds the capacity of any single component link
>    it may be carried by a network using multipath techniques, but may
>    not be carried by an MPLS-TP LSP due to the inherent MPLS-TP
>    capacity limitation imposed by MPLS-TP OAM packet ordering
>    constraints.
>  
> Perhaps I could clarify this by changing "carried by an MPLS-TP LSP"
> to "carried by a single MPLS-TP LSP".
>  
> The "MPLS-TP OAM packet ordering constraints" is the "fate sharing"
> which is the reason that MPLS-TP prohibits using ECMP (aka multipath,
> though ECMP is only one form of multipath) with MPLS-TP.  This is why
> an MPLS LSP (client layer) with capacity greater than a component link
> cannot be carried over a *single* MPLS-TP LSP.
>  
> Since this is explained in later sections I could make the
> substitution suggested above or I could drop the last sentence and
> mention that limitation later.
>  
> [Dave] the sentence I had a problem with was "but may not be carried
> by an MPLS-TP LSP due to the inherent MPLS-TP capacity limitation
> imposed by MPLS-TP OAM packet ordering constraints."

The phrase you are objecting to is "packet ordering constraints".

Carlos suggested that I reference or quote RFC 5960 which would clear
up exactly what the dataplane requirements are.

If direct LM OAM is supported, then there is a strict packet
reordering requirement (can't reorder the LM OAM packet relative to
payload packets or accuracy suffers) so maybe I need to cite RFC 6374
Section 2.9.4 "Equal Cost Multipath" and Section 4.2.10 "Message Loss
and Packet Misorder Conditions".

> It is not an ordering constraint, it is a fate sharing constraint. It
> read fairly strangely to me as to what the problem was you were trying
> to expose. IMO the TP requirements are focused on both determinism of
> commitment of network resources for network planning & SLA purposes,
> and ability to fully and reliably instrument the connectivity. IMO
> multipath violates the latter, whether it violates the former would be
> a topic of debate unresolvable in the reality of LAG. LAG simply
> scoping the domain of uncertainty for OAM purposes to a link, and
> requiring extra link OAM and inheritance...

This is an aside, but since you brought this up I'll respond.

Begin aside.

You made two implied assertions.  1) MPLS-TP is more deterministic, 2)
MPLS-TP is necessary to reliably instrument the connectivity.

A circuit swtiched (or MPLS-TP) network is more deterministic with
respect to how much capacity is allocated.  When the capacity in
question is a large aggregate of traffic, the packet switched (or
MPLS) network is more deterministic with respect to individual
customer experience.  For example, if many LSP travel across a core
and if one city pair is exceeding its expected capactiy but others are
slidghtly underutilizing their expected capactiy, then in the circuit
switched or MPLS-TP network individual flows for that city pair
experience congestion, whereas in a packet switched or MPLS network,
capacity is shared and therefore no congestion is experienced.

In MPLS core networks the SLA traffic is a fraction of traffic and
does fine with little more than TC marking and diffserv treatment.

The determinism you describe may be an issue in metro networks, such
as where mobile backhaul is mixed with pedestrian Internet traffic and
capacity is scarse.

Kireeti Kompella and other wrote a paper about two years ago on
exactly this topic and it might help if I dug up the reference and
included in the document.

The latter point regarding reliably instrumenting the connectivity is
valid.  But if the benefit of instrumenting is for a minority of the
traffic (SLA traffic) and the cost is a reduction in network
efficiency for the vast majority of traffic (plain old Internet), then
there is a substantial increase in cost to provide the instrumented
service and then it may not be worth it.  This is why MPLS-TP carried
over MPLS is of interest.

End aside.

> [Dave] To make the discussion short, replacing "OAM packet ordering
> constraints" with "OAM fate sharing constraints" should be
> sufficient.

In practice the "OAM fate sharing constraints" and "OAM packet
ordering constraints" are almost one in the same where LAG or ECMP are
the cause of the misordering.  LM OAM requires strict packet ordering,
not just fate sharing.

I'll add the references to RFC 5960 and RFC 6374 and let those RFCs
state the requirements.

> > Main issue:
> >  
> > Section 3: I think this needs a bit of work, it starts to discuss 
> > tractable solutions but IMO is technically incomplete. IF an entropy 
> > label and ELI (where all traffic associated with the MPLS_TP LSP 
> > shares a common randomly selected entropy label value, which is not
> > stated) are the only mechanisms of ensuring proper treatment (with all 
> > transit nodes implementing entropy and ELI processing such that 
> > seeking sources of entropy is capped for such LSPs), and the ingress 
> > node (which by some means SHOULD be able to know it is a TP LSP as it 
> > has layer visibility) and the egress node can strip entropy and ELI 
> > then a solution exists for proper fate sharing and ordering of a TP 
> > LSP over MPLS. It is IMO not quiet eluciated completely.
>  
> Your paragraph above describes the solution, but then says "It is IMO
> not quiet eluciated completely."
>  
> [Dave] Well I know you and I know the solution, but someone newer to
> the subject would not necessarily be able to infer it from the
> document in its current state.

OK.  Fair enough.

> So lets parse your paragraph and see what the draft is missing:
>  
> > Section 3: I think this needs a bit of work, it starts to discuss 
> > tractable solutions but IMO is technically incomplete.
>  
>     OK ...
>  
> > IF an entropy label and ELI (where all traffic associated with the 
> > MPLS_TP LSP shares a common randomly selected entropy label value, 
> > which is not stated)
>  
>     You mention "IF an entropy label and ELI (where all traffic
>     associated with the MPLS_TP LSP shares a common randomly selected
>     entropy label value, which is not stated)".  Regarding the "which
>     is not stated" part of that phrase I can change the following
>     paragraph:
>  
>       OLD:
>  
>         MPLS-TP LSP can be carried as client LSP within an MPLS server
> 	LSP if an Entropy Label Indicator (ELI) and entropy label (EL)
> 	is added after the server layer LSP label(s) in the label
> 	stack, just above the MPLS-TP LSP label entry [RFC6790].
> 	[...]
>  
>       NEW:
>  
>         MPLS-TP LSP can be carried as client LSP within an MPLS server
>         LSP if an Entropy Label Indicator (ELI) and Entropy Label (EL)
>         is added after the server layer LSP label(s) in the label
>         stack, just above the MPLS-TP LSP label entry [RFC6790].  The
>         value of EL can be randomly selected at LSP setup time and the
>         same EL value used for all packets of the MPLS-TP LSP.  [...]
>  
> [Dave] That edit works for me.

Its in the xml source and will be in the next iteration.

>     One sentence is added to the end.  (and Entropy Label is capitalized).
>  
> > are the only mechanisms of ensuring proper treatment
>  
>     Actually link-bundling is mentioned as another mechanism.
>  
> > (with all transit nodes implementing entropy and ELI processing such 
> > that seeking sources of entropy is capped for such LSPs),
>  
>     The requirement to terminate search for additional entropy below
>     the EL is a requirement of RFC 6790.  The intent is to say that
>     non-compliant midpoint LSR would in this case cause packet order
>     problems where in MPLS only (no TP carried) use of RFC 6790, those
>     non-compliant midpoint LSR are tolerable as long as they can get
>     enough entropy from the traffic to adequately load split.
>  
> [Dave] Given the relative newness of 6790, I'm assuming some statement
> to the effect that this solution is not necessarily backwards
> compatible with existing deployments needs to be said.

OK.  Most of the draft-villamizar-mpls-multipath-extn is about fixing
the limitation that MPLS-TP LSP need to be identified in signaling and
that the ingress needs to know which links can't support MPLS-TP
requirements.

This text addresses the need to know which links can't support
MPLS-TP.

   MP#4  Where an RSVP-TE control plane is used, it MUST be possible
   	 for an ingress LSR which is setting up an MPLS-TP or MPLS LSP
   	 to determine at CSPF time whether a link or MPLS PSC LSP
   	 within the topology can support the MPLS-TP requirements of
   	 the LSP.

I'll explicitly mention that legacy LAG and legacy ECMP that do not
support ELI and EL are among the "link or MPLS PSC LSP within the
topology" that cannot "can support the MPLS-TP requirements of the
LSP."  Without protocol extensions, the ingress can still know this if
color (administrative attributes in RFC 3209) is used to identify
these links.  Its worth mentioning this, so thanks for bringing it up.

So far all I say is:

   Requirement MP#4 can be supported using administrative attributes.
   Administrative attributes are defined in [RFC3209].  Some
   configuration is required to support this.

I'll preceed that with:

   Links or MPLS LSP which traverse LAG or ECMP with adjacent LSR that
   do not support [RFC6790] cannot support MPLS-TP requirements.  This
   is the reason for including requirement MP#4.  The ingress of an
   MPLS-TP LSP must be able to avoid these links or MPLS LSP.

I think this addresses your concern.  [btw- sometimes it is better to
state what is obvious to some readers than confuse the others
readers, so I agree that this is an improvement.]

> > and the ingress node (which by some means SHOULD be able to know it is 
> > a TP LSP as it has layer visibility)
>  
>     That is discussed in the following paragraph:
>  
>        There is currently no signaling mechanism defined to support
>        requirement MP#1.  In the absense of a signaling extension,
>        MPLS-TP can be identified through some form of configuration,
>        such as configuration which provides an MPLS-TP compatible
>        server layer to all LSP arriving on a specific interface or
>        originating from a specific set of ingress LSR.  Alternately an
>        MPLS-TP LSP can be created with and Entropy Label Indicator
>        (ELI) and entropy label (EL) below the MPLS-TP label [RFC6790].
>  
>     Obviously the EL can be added at the MPLS-TP ingress LSR.  This
>     paragraph is considering the case where a set of MPLS-TP LSR are
>     not EL capable but also do not have multipath enabled, but
>     multipath is enabled elsewhere (ie: the core where lots of traffic
>     needs to get aggregated into a smaller number of LSP).  The lack
>     of a signaling extension is noted as a (fixable) limitation.  The
>     fix comes in a later draft (draft-villamizar-mpls-multipath-extn).
>  
> > and the egress node can strip entropy and ELI
>  
>     That is a hard requirement of RFC 6790 for an LSR claiming to
>     support RFC 6790.
>  
> > then a solution exists for proper fate sharing and ordering of a TP 
> > LSP over MPLS.
>  
>      OK ... agreed.
>  
> > It is IMO not quiet eluciated completely.
>  
>      You seem to have figured out all of the details.
>  
> If there are additional clarifications that are needed besides the
> ones suggested above, please point out what details of the solution
> are still not clear in the existing text.
>  
> > If I'm unclear on any of the above, am happy to discuss...
>  
> Thanks.  Please let me know if the above clarifications are sufficient
> or whether additional clarifications are needed.
>  
> [Dave] I think with the change of ordering constraint to fate sharing
> constraint in section 1, the EL "fixed value" addition and some
> disclaimer about 6790 support in the MPLS network, I would be
> happy. I'm clearer on what you were saying.

If the changes above don't adequately address your concerns then
please say so.

> Best
> Dave

Cheers,

Curtis