Re: [mpls] AD review of draft-ietf-mpls-forwarding

[Curtis Villamizar <[e-mail]>]

In message <033c01cf1c1b$612c9480$2385bd80$@olddog.co.uk>
"Adrian Farrel" writes:
 
> Curtis,
> Thanks for all this.
> I'm good with the changes recorded here.
> Time to post AFAIK.
>  
> A


Adrian,

I just checked the diffs after submitting 05 and I want to make two very
minor changes.

 OLD

   ... renamed these from the term "reserved labels" used in [RFC3032]
   "special purpose labels".

 NEW

   ... renamed these from the term "reserved labels" used in [RFC3032]
   to "special purpose labels".

Added the word "to" as in "rename from X to Y".

 OLD

   registry reachable at IANA's pages at [1].

 NEW

   registry reachable at IANA's pages at http://www.iana.org.

The above is the return of a xml2rfc eref bug.  Need to hand edit the
.txt file or fix the python source again.  Didn't catch this since
there was no change in the xml file.

Should I hold off on these or submit a 06 version just to fix this?

Curtis


> > -----Original Message-----
> > From: Curtis Villamizar [mailto:[e-mail]]
> > Sent: 28 January 2014 04:09
> > To: [e-mail]
> > Cc: [e-mail]; [e-mail];
> > [e-mail]
> > Subject: Re: [mpls] AD review of draft-ietf-mpls-forwarding
> > 
> > 
> > In message <022d01cf1bb4$1624fde0$426ef9a0$@olddog.co.uk>
> > "Adrian Farrel" writes:
> > >
> > > Hello Curtis,
> > >
> > > These replies modulo the conversation with Carlos. I find I can't read that
> > > conversation and back apply the results here :-(
> > 
> > I'll summarize the outcome of that conversation with Carlo at the
> > bottom so we can have one thread.
> > 
> > > [snip]
> > >
> > > > > Your acronym list is commendably thorough, but a little enthusiastic.
> > > >
> > > > If I provide a section with a list of acronyms, do I still have to
> > > > expand on first use.  If so, AC, NSP, OAM, and a few others appear
> > > > before that section.
> > >
> > > Afraid so :-(
> > 
> > OK.  Expanded except RFC-Editor listed well known abbreviations.
> > 
> > > > Given that this takes up only 17 lines in a 50 page document I'd
> > > > rather be thorough.
> > >
> > > Yup. OK. Go for it.
> > >
> > > [snip]
> > >
> > > > > I think Curtis may have heard this before :-)
> > > > > The "preferred" (by the RFC editor) expansion of ECMP is
> > > > > "Equal-Cost Multipath"
> > > >
> > > > The form without the hyphen is more common, even among recent
> > > > documents.  I prefer to keep it without the hyphen.
> > >
> > > A discussion for a rainy day with the RFC Editor.
> > > Leave as is.
> > 
> > Cheers.
> > 
> > > > > Section 1.3 bullet 5
> > > > >
> > > > >    5.  The implementer and system designer MUST support pseudowire
> > > > >        control word (CW) if MPLS-TP is supported or if ACH [RFC5586] is
> > > > >        being used on a pseudowire.
> > > > >
> > > > > The wording is a bit odd. "The implementation and system design..."?
> > > > >
> > > > > Ditto bullets 6 and 7
> > > >
> > > > Target audience is explained in Section 1.4.  If you like I can flip
> > > > Section 1.3 and 1.4 so target audience is first, then use of the roles
> > > > called for in the target audience section won't seem quite so odd.
> > >
> > > No issue with the section order.
> > > Just puzzled by "the implementer MUST support" when I (pedantically) thought
> > > that the implementer supporting something might not be the same as the
> > > implementation supporting it.
> > >
> > > Not a big deal.
> > 
> > Good point.  Its changed.
> > 
> > > > > Section 1.3
> > > > >
> > > > > While there is not wrong with the statements made in the bullets, some
> > > > > of the later ones refer to recent additions to the MPLS suite. Yet the
> > > > > list is presented as "there were some misconceptions." Clearly the
> > > > > early silicon did not have misconceptions about the inclusion of entropy
> > > > > labels.
> > > > >
> > > > > Just tweak the words at the top of the list?
> > > >
> > > > I'd like to keep that as is and split into two lists.  The second list
> > > > would have the last two items (fat-pw and EL).  The first list would
> > > > end with "implement CW" (sic).
> > > >
> > > >  OLD
> > > >
> > > >    6.  The implementer and system designer SHOULD support adding a
> > > >        pseudowire Flow Label [RFC6391].  Deployments MAY enable this
> > > >        feature for appropriate pseudowire types.  See Section 2.4.3.
> > > >
> > > >    7.  The implementer and system designer SHOULD support adding an MPLS
> > > >        entropy label [RFC6790].  Deployments MAY enable this feature.
> > > >        See Section 2.4.4.
> > > >
> > > >  NEW
> > > >
> > > >    The following statements provide clarification regarding more
> > > >    recent requirements that are often missed.
> > > >
> > > >    1.  The implementer and system designer SHOULD support adding a
> > > >        pseudowire Flow Label [RFC6391].  Deployments MAY enable this
> > > >        feature for appropriate pseudowire types.  See Section 2.4.3.
> > > >
> > > >    2.  The implementer and system designer SHOULD support adding an MPLS
> > > >        entropy label [RFC6790].  Deployments MAY enable this feature.
> > > >        See Section 2.4.4.
> > > >
> > > > I've made this change.  Let me know if this is not OK.
> > >
> > > That's fine.
> > >
> > > > > 2.1.1
> > > > >
> > > > > Maybe the first paragraph should clarify "special purpose labels at
> > > > > the top of the label stack"
> > > >
> > > > That phrase doesn't appear anywhere in the document.  Exactly what am
> > > > I clarifying?  What to do with unknown special purpose labels is in
> > > > the last paragraph of this subsection.
> > >
> > > WTF?
> > > You have to wonder where these senile ADs get their ideas.
> > >
> > > Complete brain fart. Sorry.
> > >
> > > [snip]
> > >
> > > > > While per platform label space is mentioned in 2.1.7 I wonder whether
> > > > > more information on per platform and per interface label spaces is
> > > > > needed. I recall early implementations that got very confused when
> > > > > parallel interfaces used the same label for different purposes.
> > > > >
> > > > > I guess the point there is that you cannot assume that your neighbor
> > > > > is or is not using the per platform label space.
> > > > >
> > > > > Upstream label allocation may also come into this.
> > > >
> > > > The only mention of label allocation is that MPLS FRR bypass method
> > > > (more formally known as facilitles backup) uses platform label space.
> > > >
> > > > The only reason platform label space is of any significance in a
> > > > document about forwarding is "The use of platform label space impacts
> > > > the size of the LSR ILM for LSR with a very large number of
> > > > interfaces."
> > > >
> > > > Label allocation, per platform or per interface and upstream or
> > > > downstream, is not a forwarding issue.  It is a software issue
> > > > and a matter of getting the protocol bits right.  Therefore I think
> > > > expanding any further on label allocation should be out of scope.
> > >
> > > OK. I'm convinced.
> > >
> > > > > 2.1.8.1
> > > > >
> > > > >    3.  If the edge is not using pseudowire control word (CW) and the
> > > > >        core is using multipath, reordering will be far more common.  If
> > > > >        this is occurring, the best solution is to use CW on the edge,
> > > > >        rather than try to fix the reordering using resequencing.
> > > > >
> > > > > Completely agree, but isn't the sequence number contained in a control
> > > > > word meaning that the resequencing could, in any case, not be done
> > > > > without using a control word?
> > > >
> > > > I suppose you can't fix reordering caused by not using CW without the
> > > > sequence number in the CW.  That is going to require fixing the text.
> > > >
> > > >  OLD
> > > >
> > > >    3.  If the edge is not using pseudowire control word (CW) and the
> > > >        core is using multipath, reordering will be far more common.
> > > >        If this is occurring, the best solution is to use CW on the
> > > >        edge, rather than try to fix the reordering using resequencing.
> > > >
> > > >  NEW
> > > >
> > > >    3.  If the edge is not using pseudowire control word (CW) and the
> > > >        core is using multipath, reordering will be far more common.
> > > >        If this is occurring, using CW on the edge will solve the
> > > >        problem.  Without CW, resequencing is not possible since the
> > > >        sequence number is contained in the CW.
> > > >
> > > > That was a big oops on our part.
> > >
> > > Well, on the scale of IETF oopsies, I don't think you score too high. Maybe
> > > Narten could run a weekly script?
> > >
> > > [snip]
> > >
> > > > > Should 2.2 distinguish the order of magnitude of replication at branch
> > > > > nodes? This impacts the replication method used (some devices make a
> > > > > copy and cycle around, some devices can do multiple copies at once). On
> > > > > the whole is no different from IP multicast processing except (as you
> > > > > note) that each outgoing packet may be different by its label value.
> > > >
> > > > Is it possible to quantify the fanout?  YMMV?
> > > >
> > > > The only thing I could say is that an implementation may need to make
> > > > lots of copies in some roles (access routers for example).
> > > >
> > > > Making a copy and cycling yields poor performance but for low
> > > > multicast traffic volumes might be OK.  But you are right - some
> > > > mostly low-end-ish chips to this.
> > > >
> > > > I'm not sure I can describe how multicast with high fanout is done
> > > > without wading into implementation details of specific vendors.
> > > >
> > > > Perhaps the best I can do is add this:
> > > >
> > > >    Careful consideration should be given to the performance
> > > >    characteristics of high fanout multicast for equipment that is
> > > >    intended to be used in such a role.
> > > >
> > > > I'll add this before the last paragraph in the section.
> > >
> > > That works.
> > >
> > > > > 2.4
> > > > >
> > > > > So obvious you didn't say it?
> > > > >
> > > > >    In order to support an adequately balanced load distribution across
> > > > >    multiple links, IP header information must be used.  Common practice
> > > > >    today is to reinspect the IP headers at each LSR and use the label
> > > > >    stack and IP header information in a hash performed at each LSR.
> > > > >    Further details are provided in Section 2.4.5.
> > > > >
> > > > > Missing is the statement that a single "flow" must not be distributed
> > > > > across multiple paths because of the implication for potentially
> > > > > significant packet misordering. And feeding that is a common requirement
> > > > > that such packet misordering must not occur because applications and
> > > > > transport protocol implementations cannot survive such misordering.
> > > >
> > > > Yes.  That requirement was missed.  Add new second paragraph to this
> > > > subsection.
> > > >
> > > >    The Differentiated Services requirements for good reasons dictate
> > > >    that packets within a common microflow SHOULD NOT be reordered
> > > >    [RFC2474].  Service providers generally impose stronger
> > > >    requirements, commonly requiring that packets within a microflow
> > > >    MUST NOT be reordered except in rare circumstances such as load
> > > >    balancing across multiple links or path change for load balancing
> > > >    or path change for other reason.
> > > >
> > > > Another SP requirement is stated here and I'm quite sure this
> > > > requirement is well accepted.
> > >
> > > Looks good.
> > >
> > > > > 2.4.2 uses "composite link" and "component link". I suggest picking just
> > > > > one term.
> > > >
> > > > They are two different things.  Two or more component links make up a
> > > > composite link.  Knowing that, give it another read please.
> > > >
> > > > I'd rather not cite draft-ietf-rtgwg-cl-requirements as an
> > > > informational reference just for this one term.  In favor of citing
> > > > it, draft-ietf-rtgwg-cl-requirements is moving along.  Against citing
> > > > it is there is far less than a ground swell of providers calling for
> > > > the full set of things asked for in draft-ietf-rtgwg-cl-requirements.
> > >
> > > Yes. Sorry. It has been a looooooooong time since I had a pass on the CL
> > > document. Atrophy.
> > 
> > Its also in Gwiz.8080 or something like that.
> > 
> > > > > 2.4.5.1 notes that special purpose and extended special purpose labels
> > > > > need to be excluded from the hash. Good.
> > > > > But it seems that some special purpose labels will indicate that the
> > > > > next label stack entry contains a label with special meaning. (ELI is
> > > > > an example that we specifically don't have to worry about.)
> > > > > How do we handle that?
> > > > > Should we be dividing up the extended special purpose label space to
> > > > > have one set of code points meaning "just this label is special" and
> > > > > another set meaning "this label is special and the next label stack
> > > > > entry is magic"?
> > > >
> > > > I did list ELI (bullet 2) before the more general rule of not useing
> > > > special purpose labels.  The ELI is not used, just the EL, so the text
> > > > could be considered correct as-is.
> > > >
> > > > So far the only special purpose label that is not just ignored and
> > > > skipped over is ELI.
> > > >
> > > > Regarding this being magic -- All of this is somewhat programable
> > > > specialized silicon magic.  The silicon generally has some form of
> > > > very fast, very light weight parsing engine at the front of the
> > > > pipeline.  One thing it does is pick out fields for load balance.
> > > >
> > > > The better silicon hashes as it goes rather that pick out a set of
> > > > fields and then hashes that set of fields when its done.  When it sees
> > > > 13 it skips and hashes the next thing and stops hashing completely.
> > > > If it sees 0-12,14 it skips and continues.  If it sees 15 it skips two
> > > > labels and continues.  Its should be programable enough that if
> > > > someone defines a new ELI like label it is likely to be able to deal
> > > > with it.
> > > >
> > > > The not so good silicon has this all so hard wired that it won't be
> > > > able to do ELI without at least a respin.
> > > >
> > > > At most I could add "If a new special purpose label or extended
> > > > special purpose label is defined which requires special load balance
> > > > processing then, as is the case for the ELI label, a spacial action
> > > > may be needed rather than skipping the special purpose label or
> > > > extended special purpose label."  I really don't think this is needed.
> > >
> > > You're right, and my worry is more about the special-purpose draft and the
> > > consequences of possibly adding other special purpose labels that have child
> > > labels associated. We certainly don't want to have to retrain the silicon at
> > > transit LSRs to specially know what to do for each new special-purpose
> label.
> > > Currently we propose that you don't hash on a special purpose label, but you
> > can
> > > carry on hashing immediately after.
> > >
> > > If I introduce the foo-label, your silicon will recognise it as special
> purpose,
> > > but it I say the label after the foo-label is magic you won't know that.
> > >
> > > A way to fix this is to have (just punting here) the top bit of the extended
> > > special purpose label range set mean "magic label follows".
> > 
> > I added this:
> > 
> >    4.  If a new special purpose label or extended special purpose
> >        label is defined which requires special load balance
> >        processing, then, as is the case for the ELI label, a spacial
> >        action may be needed rather than skipping the special purpose
> >        label or extended special purpose label.
> > 
> > This will be the new bullet 4, bumping down the old 4 and 5.
> > 
> > > > > An issue that arises from the multipath support (2.4.5.1) is that
> > > > > hardware assumes that after a label stack entry with the S-bit set,
> > > > > there are only three possible next bytes...
> > > > > - a control word (indicated by b0000 or b0001)
> > > > > - an IPv4 header
> > > > > - an IPv6 header
> > > > > This is the case regardless of how the LSP was set up, and the next
> > > > > bytes cannot ever be further MPLS stack entries.
> > > >
> > > > Right.  Note that in (5) is says that some SP will require IP headers
> > > > and some will require an ability to disable IP headers.
> > > >
> > > > The rule is really look for 4, 6, or anything else in the first
> > > > nibble.  If 4 or 6 assume IP.  If anything else stop.
> > > >
> > > > And yes if the payload is MPLS after a S-bit you have a screwed up
> > > > MPLS implementation to start with and you won't get load balance on
> > > > any set of MPLS labels after the first S-bit.  This is a fact of life
> > > > in the field and is as it should be.
> > > >
> > > > > While this comes up 2.4.5.1 it may merit further discussion in an
> > > > > earlier section of the document.
> > > >
> > > > This text is part of 2.4. ("MPLS Multipath Techniques").  The third
> > > > paragraph contains "Further details are provided in Section 2.4.5."
> > > > Section 2.4.5. is "Fields Used for Multipath Load Balance".
> > > >
> > > > > I note that discussion of support of PWs without the CW drives you
> > > > > to say that hashing beyond the S-bit should be a configurable option
> > > > > which would (of course) support any payload including MPLS in MPLS
> > > > > with repeated bottom of stack. However, you might want to specifically
> > > > > preclude that.
> > > >
> > > > It says the same thing here in bullet 5 regarding being configurable.
> > > > The wording "ability to disable" is same as "configurable option".
> > > >
> > > > At no point in this document do we imply that looking beyond the S-bit
> > > > means looking at anything beyond the S-bit other than looking for IP.
> > > > This is very clear in [RFC4385] and [RFC4928] which is cited in the
> > > > text about PW CW.
> > > >
> > > > All it says in the places discusing PW is that without CW the traffic
> > > > might get reordered.
> > > >
> > > > If you feel that we at any point imply that lack of PW CW allows
> > > > looking at anything past the S-bit rather than just looking for an IP
> > > > header please point to where and we will have to correct that.  I
> > > > looked at all occurances of CW and did not find anything.
> > > >
> > > > Bullet 5 is very clear that a 4 or 6 has to be found in the first
> > > > nibble of payload.
> > >
> > > OK. I misread bullet 5.
> > 
> > OK
> > 
> > > [snip]
> > >
> > > Thanks.
> > > Adrian
> > 
> > Summary of conversation with Carlos:
> > 
> > Adrian wrote:
> > >>   Tunneling encapsulations which may carry MPLS, such as MPLS in GRE,
> > >>   L2TP, or LDP, are out of scope.
> > >>
> > >> I think s/LDP/UDP/
> > 
> > Carlos noted some wording issues in this sentence and suggested adding
> > citations for each.  After some discussion:
> > 
> >  OLD (original)
> > 
> >    Tunneling encapsulations which may carry MPLS, such as MPLS in GRE,
> >    L2TP, or LDP, are out of scope.
> > 
> >  NEW
> > 
> >    Tunneling encapsulations carrying MPLS, such as MPLS in IP
> >    [RFC4023], MPLS in GRE [RFC4023], MPLS in L2TPv3 [RFC4817], or MPLS
> >    in UDP [I-D.ietf-mpls-in-udp], are out of scope.
> > 
> > He also suggested the following.
> > 
> >  OLD
> > 
> >    Some IP encapsulations support tunneling, such as IP-in-IP, GRE,
> >    L2TPv3, and IPSEC.  These provide a greater source of entropy which
> >    some provider networks carrying large amounts of tunneled traffic
> > -   may need.
> >    The use of tunneling header information is out of scope for this
> >    document.
> > 
> >  NEW
> > 
> >    Some IP encapsulations support tunneling, such as IP-in-IP, GRE,
> >    L2TPv3, and IPSEC.  These provide a greater source of entropy which
> >    some provider networks carrying large amounts of tunneled traffic
> > +   may need, for example as used in [RFC5640] for GRE and L2TPv3.
> >    The use of tunneling header information is out of scope for this
> >    document.
> > 
> > As part of that conversation it was noted that the following also had
> > no citations.
> > 
> >  OLD
> > 
> >    Support for other protocols that share a common Layer-4 header such
> > -   as RTP, UDP-lite, SCTP and DCCP
> >    SHOULD be provided, particularly for edge or access equipment where
> >    additional entropy may be needed.
> > 
> >  NEW
> > 
> >    Support for other protocols that share a common Layer-4 header such
> > +   as RTP [RFC3550], UDP-Lite [RFC3828], SCTP [RFC4960] and DCCP
> > +   [RFC4340]
> >    SHOULD be provided, particularly for edge or access equipment where
> >    additional entropy may be needed.
> > 
> > btw- this is wrt fields used in load balancing and is saying in effect
> > "use more than just port 6 and 17 (UDP and TCP)".
> > 
> > Other than that, I looked at the list of informational references and
> > found the following do affect MPLS forwarding and therefore should be
> > promoted to normative.
> > 
> >    [RFC4950]  Bonica, R., Gan, D., Tappan, D., and C. Pignataro, "ICMP
> >               Extensions for Multiprotocol Label Switching", RFC 4950,
> >               August 2007.
> > 
> >    [RFC5082]  Gill, V., Heasley, J., Meyer, D., Savola, P., and C.
> >               Pignataro, "The Generalized TTL Security Mechanism
> >               (GTSM)", RFC 5082, October 2007.
> > 
> >    [RFC5085]  Nadeau, T. and C. Pignataro, "Pseudowire Virtual Circuit
> >               Connectivity Verification (VCCV): A Control Channel for
> >               Pseudowires", RFC 5085, December 2007.
> > 
> >    [RFC5332]  Eckert, T., Rosen, E., Aggarwal, R., and Y. Rekhter, "MPLS
> >               Multicast Encapsulations", RFC 5332, August 2008.
> > 
> >    [RFC5880]  Katz, D. and D. Ward, "Bidirectional Forwarding Detection
> >               (BFD)", RFC 5880, June 2010.
> > 
> >    [RFC5884]  Aggarwal, R., Kompella, K., Nadeau, T., and G. Swallow,
> >               "Bidirectional Forwarding Detection (BFD) for MPLS Label
> >               Switched Paths (LSPs)", RFC 5884, June 2010.
> > 
> >    [RFC5885]  Nadeau, T. and C. Pignataro, "Bidirectional Forwarding
> >               Detection (BFD) for the Pseudowire Virtual Circuit
> >               Connectivity Verification (VCCV)", RFC 5885, June 2010.
> > 
> >    [RFC6374]  Frost, D. and S. Bryant, "Packet Loss and Delay
> >               Measurement for MPLS Networks", RFC 6374, September 2011.
> > 
> >    [RFC6375]  Frost, D. and S. Bryant, "A Packet Loss and Delay
> >               Measurement Profile for MPLS-Based Transport Networks",
> >               RFC 6375, September 2011.
> > 
> >    [RFC6378]  Weingarten, Y., Bryant, S., Osborne, E., Sprecher, N., and
> >               A. Fulignoli, "MPLS Transport Profile (MPLS-TP) Linear
> >               Protection", RFC 6378, October 2011.
> > 
> >    [RFC6427]  Swallow, G., Fulignoli, A., Vigoureux, M., Boutros, S.,
> >               and D. Ward, "MPLS Fault Management Operations,
> >               Administration, and Maintenance (OAM)", RFC 6427, November
> >               2011.
> > 
> >    [RFC6428]  Allan, D., Swallow Ed. , G., and J. Drake Ed. , "Proactive
> >               Connectivity Verification, Continuity Check, and Remote
> >               Defect Indication for the MPLS Transport Profile", RFC
> >               6428, November 2011.
> > 
> >    [RFC6720]  Pignataro, C. and R. Asati, "The Generalized TTL Security
> >               Mechanism (GTSM) for the Label Distribution Protocol
> >               (LDP)", RFC 6720, August 2012.
> > 
> >    [I-D.ietf-mpls-psc-updates]
> >               Osborne, E., "Updates to PSC", draft-ietf-mpls-psc-
> >               updates-00 (work in progress), October 2013.
> > 
> > Upgrades were mostly due to BFD and MPLS-TP OAM.  For example, direct
> > LM needs to be in hardware and most things related to protection.
> > 
> > Note that "Updates to PSC" affects MPLS-TP protection state machine
> > which needs hardware assist if not direct support in hardware in order
> > to be fast.  The race condition may be the most severe problem with
> > RFC 6378 state machine.
> > 
> > If you think any of the above should be put back into informative,
> > please let me know.
> > 
> > Curtis