Re: RtgDir review: draft-ietf-rtgwg-cl-requirement-10

[Curtis Villamizar <[e-mail]>]

Lou,

This top posting is to inform the entire Cc of an email thread
regarding a major point in these comments.

The summary message of that thread with a recent response from Alia is
concatonated to the end of this email as it is a plain text email.

As promised in that email, I have edited my draft response sent to you
and others below.  The further edits to your RtgDir review are marked
with *CV*.  The comments not preceded by *CV* are as sent in the
preview prior to the smaller Cc discussion.

The most important point is that the document shepard and AD have
instructed that a draft-ietf-rtgwg-cl-requirement-11 version be
created and that it undergo another WGLC due to the nature of the
changes.  Details are in the summary appended to the end.

Some of your comments are resolved by avoiding conflicts with ITU-T
defined terminology as described below.

The SecDir and GenArt review comments will also be reflected.

Thank you for your help in substantially improving this document.

Curtis


In message <[e-mail]>
Lou Berger writes:
> 
> Hi Curtis,
>  
> Thank you for the detailed response. I have included specific responses
> in-line below.  Before we go there, I'd like to jump ahead to one point
> that is (indirectly) pointed to in a number of spots.

It would have helped if this had come up in WGLC.  If we need to
change the document enough that we need to go back to that step, then
maybe we need to notify MPLS and CCAMP of the RTGWG WGLC.

*CV* Perhaps with this repeated WGLC we should notify MPLS and CCAMP.

> I asked:
> >> So where does one go to get an intro/definition of the IETF work on
> >> composite links?  Is it G.800, the framework draft, the use cases
> >> draft, this draft? ...
>  
> Your response is:
> > I'd say "Use Cases".  I'm in favor of de-emphasizing G.800.
>  
> Okay, then I think a bunch of detailed comments can be replaced by using
> references. More on this below. The issue with the use case document
> being the authoritative definition for CL (and presumably NPO?) is that

It would be best if the authoritative definitions where in
draft-ietf-rtgwg-cl-requirement (this document).  I would not like to
see draft-ietf-rtgwg-cl-use-cases have to be a normative reference
over the definitions of Composite Link and Performance Objective.

My preference is to embed the authoritative definitions for all of the
composite link work in this document.  This also removes the circular
references.

*CV* Consensus on the later thread was to do this - to put the
authoritative definitions in draft-ietf-rtgwg-cl-requirement and also
to avoid conflicts with ITU-T terminology.

> I think you currently have circular references.
> draft-ietf-rtgwg-cl-use-cases starts out:
>  
>    Composite link requirements are specified in
>    [I-D.ietf-rtgwg-cl-requirement].  A composite link framework is
>    defined in [I-D.ietf-rtgwg-cl-framework].
>  
> BTW ietf-rtgwg-cl-framework adds
>  
>    Classic multipath, including Ethernet Link Aggregation has been
>    widely used in today's MPLS networks [RFC4385][RFC4928].  Classic
>    multipath using non-Ethernet links are often advertised using MPLS
>    Link bundling.  A link bundle [RFC4201] bundles a group of
>    homogeneous links as a TE link to make IGP-TE information exchange
>    and RSVP-TE signaling more scalable.  A composite link allows
>    bundling non-homogenous links together as a single logical link.  The
>    motivations for using a composite link are descried in
>    [I-D.ietf-rtgwg-cl-requirement] and [I-D.ietf-rtgwg-cl-use-cases].
>  
>    A composite link is a single logical link in MPLS network that
>    contains multiple parallel component links between two MPLS LSR.
>    Unlike a link bundle [RFC4201], the component links in a composite
>    link can have different properties such as cost, capacity, delay, or
>    jitter.
>  
> I think answering this question (of which document has the CL
> definition) and ensuring the documents line up accordingly would be
> really helpful.  -- I'll assume it's the use case draft for my comments
> below.

I'd rather we put the authoritative definitions directly in the
requirements document.  See block of text very far below.

*CV* See prior comment.

> At the nit level, the defining document should be listed as a Normative
> Reference by this (and other CL) documents.

For that reason I would rather move the definition.  I would also like
to get it right in the first step of this process and have Use Cases
follow as strictly an informtional document about current practices,
then have the framework follow that.

*CV* See comment two up from here.

> On 6/20/2013 3:51 PM, Curtis Villamizar wrote:
> > In message <[e-mail]>
> > Lou Berger writes:
> >  
> >> Hello,
> >>  
> >> I have been selected as the Routing Directorate reviewer for this draft.
> >> The Routing Directorate seeks to review all routing or routing-related
> >> drafts as they pass through IETF last call and IESG review, and
> >> sometimes on special request. The purpose of the review is to provide
> >> assistance to the Routing ADs. For more information about the Routing
> >> Directorate, please see http://www.ietf.org/iesg/directorate/routing.html
> >>  
> >> Although these comments are primarily for the use of the Routing ADs, it
> >> would be helpful if you could consider them along with any other IETF
> >> Last Call comments that you receive, and strive to resolve them through
> >> discussion or by updating the draft.
> >>  
> >> Document: draft-ietf-rtgwg-cl-requirement-10
> >> Reviewer: Lou Berger
> >> Review Date: 6/19/2013
> >> IETF LC End Date: 2013-06-19
> >> Intended Status: Informational
> > 
> > Hi Lou,
> > 
> >> Summary:
> >>         I have significant concerns about this document and recommend
> >> that the Routing ADs discuss these issues further with the authors.
> > 
> > OK.  Lets go through them.
> > 
> >> Comments:
> >>  
> >> 	My comments really fall into three categories: (a) understanding
> >> document objectives, (b) comments/questions on technical details, and
> >> (c) readability/editorial.  As this document is informational and does
> >> not represent IETF consensus, the AD together with the document Shepherd
> >> and WG may reasonably choose to publish the document as is.
> > 
> > Does not reflect IETF consensus?
> > 
>  
> Sure, see http://tools.ietf.org/html/rfc5741, informational documents
> can represent IETF consensus, but don't need to; it depends on LC and is
> represented in the boilerplate.  The datatracker represents this and
> says "Consensus: says unknown". I translated this into no request for
> IETF consensus was to be made -- I accept that this may be a mistake on
> may part. (Perhaps it was wishful thinking ;-)
>  
> The document Shepherd should be able to answer this.
>  
> Alia?

Awaiting response from Alia.

*CV* Didn't get reponse from Alia on this point.  I will ping her and
 point this out.

> >> Major Issues:
> >>  
> >> - At the highest level the objective/purpose of the document is unclear
> >> to me.  It states:
> >>  
> >>    The purpose of this document is to describe why network operators
> >>    require certain functions in order to solve certain business problems
> >>    (Section 2).
> > 
> > Forward reference for brevity in "1.  Introduction".  Is that bad?
>  
> Brevity is great, but what's that quote "As brief as possible (but no
> briefer)"...

The quote is Einstien.  Simple as possible and no simpler.

*CV* Einstein is spelled wrong.  Oops.

> I genuinely am confused as to the purpose of the document so it's my
> conclusion that the Intro is too brief. Other conclusions are of course
> possible.

You are right.  The Introduction is too short (and some sentences are
poorly worded).  See further comments below regarding your suggested
replacement for the first sentence.

*CV* See summary in the thread concatonated below.  This and the next
set of changes that you suggest are explicitly called for in the
summary.

> > Section 1 is brief and forward references sections 2, 4, 5, and
> > appendix A.
> > 
> >> Thankfully (for hopefully obvious reasons), section 2 covers
> >> "Assumptions" and not "business problems". Section 4 does provide what
> >> is termed "Network Operator Functional Requirements", which relies on
> >> Y.1541 defined Network Performance Objectives.  While this section title
> >> appears to be defining provider requirements, the document also appears
> >> to be placing requirements on CL solutions.  To add to the confusion, it
> >> sometimes also seems to also be placing requirements on the "network
> >> layer" that supplies component links for the CL, and even on the
> >> performance of a vendor implementation(DR#7)!
> > 
> > It seems the objection is to the phrase "business problems".  If we
> > change the first sentence in section 1, would that solve it?
> > 
> >  OLD
> > 
> >    The purpose of this document is to describe why network operators
> >    require certain functions in order to solve certain business
> >    problems (Section 2).
> > 
> >  NEW
> > 
> >    The purpose of this document is to describe why network operators
> >    require certain functions and to clearly enumerate a set of
> >    requirements related to the use of MPLS based Composite Links
> >    (see Section 2).
>  
>  
> Well, I see a couple of phrases that confuse me in the context of the
> document.  Certainly the phrase "... to solve certain business problems
> (Section 2)" caused me to be a bit concerned that an IETF document was
> going to discuss business issues; it also led me to believe that section
> 2 was going to discuss these problems. As I said, there are no
> "business problems" covered in section 2, so aligning the Intro with the
> section contents is a fine improvement.

I didn't add the "business problems" text but I think the intent was
to say that the requirements reflect serious provider needs.

> The other phrase that confuses me is "... describe why network operators
> require certain functions ... (See Section 2)".  Section 2 is titled
> Assumptions and only references network operators to say that "...in
> general network operators do not rely on the DSCP...". So I think your
> revised text isn't really aligned with the section.

OK.

> Finally, you now say the draft documents requirements on the "use of
> MPLS based Composite Links".  So this reads like the document places
> requirements on the user, i.e., the operator.  Do you perhaps mean "on
> the protocols and mechanisms used to provide MPLS based Composite Links"?

Yes.

> Taken all together, perhaps the following better matches the/your intent.
>     The purpose of this document is to clearly enumerate a set of
>     requirements related to the protocols and mechanisms that
>     provide MPLS based Composite Links. MPLS based Composite Links
>     are defined in [I-D.ietf-rtgwg-cl-use-cases].  Section 2 describes
>     MPLS based Composite Links assumptions.

Yes.  This is a much better sentence than the existing first
sentence.  I think you are also right that the rest of the paragraph
lacks clarity as well.

> > Then in section 2, change "The services supported include ..." to
> > "Services which may be supported over MPLS based Composite Links
> > include ...".
>  
> That is helpful.  Do you perhaps mean:
> "Any MPLS-based service may be supported over MPLS based Composite Links
> including, for example, ..."

It is at least IP and MPLS services.  IP services can also be carried
with certain limitations.  The document discusses measuring and
compensating IP and LDP traffic.  Both IP and LDP are not traffic
engineered (and the requirement to NOT add TE to LDP is also stated).

*CV* Your later suggestion is to move the Assumptions section to the
Use Case document.  That is called for in the summary below.

> >> I think clearly stating the document's objective/purpose and where the
> >> stated requirements are to be applied/satisfied, is one of major issues
> >> that should be corrected.
> >>  
> >> Once this is done, it may turn out that there are derivative
> >> comments/changes that are needed in the body of the draft.  For example
> >> clarifying what it means to "meet the NPO" and who needs to "meet" it.
> > 
> > The reason the term NPO is used it because of objections to using the
> > term SLA.  A Service Level Agreement (SLA) is interpreted as
> > contractual.  A Network Performance Objection (NPO) may simply be an
> > interal goal that an organiztion would like to meet.  The NPO may or
> > may not be based on Y.1540 and Y.1541.  The term NPO is defined in
> > Y.1541 and the use of the term in this docuement is consistent with
> > that definition.
> > 
> > The use cases document has a long paragraph defining SLA, SLS, and
> > NPO.  Those definitions are in an appendix intended to be illustrative
> > and entitled "More Details on Existing Network Operator Practices and
> > Protocol Usage".
>  
> The use case document also says:
>    In this
>    document we use the term Network Performance Objective (NPO) as
>    defined in section 5 of [ITU-T.Y.1541] since the SLA and SLS measures
>    have network operator and service specific implications.

Yes.  If we continued to reference the Use Case document, we would
have to fix that.  If we put the definition directly in this document,
then we could not take that definition verbatim given your
interpretation that using the term NPO requires meeting all
requirements for NPO defined in Y.1540 and Y.1541, which was
definitely not the intent.

See comments below where s/NPO/performance objectives/ is mentioned
(search for "3a.").

*CV* There appears to be no objections to s/NPO/performance objectives/.

> > The definition here is simply:
> > 
> >    Network Performance Objective (NPO):  Numerical values for
> >        performance measures, principally availability, latency, and
> >        delay variation.  See [I-D.ietf-rtgwg-cl-use-cases] for more
> >        details.
> > 
> > I can expand on this and say "NPO may or may not be based on ITU
> > specifications."  I prefer not to get too wordy when all we are trying
> > to say "like SLA or SLS (in diffserv), but NPO is really a better
> > term".
> > 
>  
> Why not just say:
>    Network Performance Objective (NPO): as defined by
>    [I-D.ietf-rtgwg-cl-use-cases].

We would have to change the definition in Use Cases.  We want to use
the term "performance objective" term rather than SLA as used in
Diffserv, and one was conveniently defined in Y.1541, but make using
any other part of Y.1540 or Y.1541 completely oprional.  It would be
best to drop all mention of Y.1540 or Y.1541 and avoid any further
confusion.

See comments below where s/NPO/performance objectives/ is mentioned
(search for "3a.").

> > As to what it means by "meet the NPO", it means "falls within the
> > numerical values for measures, principally availability, latency, and
> > delay variation".  I'm not sure why that is not obvious.
>  
> Try using this expansion in the identified requirements and see if you
> think it makes sense in all cases.  Also, recall that Table 1 in section
> 5 of [ITU-T.Y.1541] provides absolute time values based on traffic "QoS
> class" and that a number of values are also specified as "U", which is
> stated as meaning U", performance with respect to
> that parameter may, at times, be arbitrarily poor.

Again, where s/NPO/performance objectives/ and performance objectives
need not be based on Y.1540 or Y.1541.  The service provider (SP)
defines their performance objectives and "meeting the performance
objectives" means falling within the parameters of the SP performance
objectives regardless of how they came about defining them.

> For example:
>  
>    FR#1  The solution SHALL provide a means to summarize some routing
>          advertisements regarding the characteristics of a composite
>          link such that the routing protocol converges within the
>          timeframe needed to [meet the network performance objective]
>          fall within the numerical values for measures, principally
>          availability, latency, and delay variation
>  
>  
>    FR#2  The solution SHALL ensure that all possible restoration
>          operations happen within the timeframe needed to [meet the NPO]
>          fall within the numerical values for measures, principally
>          availability, latency, and delay variation
>  
> I really think more is needed to make these requirements meaningful in
> the design of any solution.  For example, I could see something along
> the lines of the following as being meaningful:
>  
> (Note this text is for discussion not inclusion in the draft.)
>    FR#2  The solution SHALL enable restoration operations to occur
>          within the timeframe need to provide an Upper bound on the
>          packet loss probability (NPO IPLR) of 1x10^-3 in a
>          network with a diameter of XYZ.
>  
> Do you think I'm wrong?

Yes.  Absolutely.  There is no intent to mandate use of Y.1540 or
Y.1541.  Apparently that was not clear and we need to make it clear.

Re FR#1.  SP have existing convergence performance objectives.  Those
objectives are *never* based on Y.1541 IPLR.  The point of the first
sentence of FR#1 is to not make changes to the IGP that will make
convergence worse.  The point of the second sentence is that we want
to summarize the CL, but also provider more detailed information about
the component links, or preferably groups of compont links.

For example:  Link has 2TB, delay is <= 12 ms, etc.  One group of
component links has 1 TB, delay <= 12 ms.  Another group of component
links has 1 TB, delay <= 10 msec.  This could be the case with two
slightly different fiber paths.

*CV* Again, s/NPO/performance objectives/ should clear this up.

> On an aside, how do FR#1 and DR#6 really differ?

FR#1 expose info on component but don't affect convergence
FR#2 don't make restoration time any worse
FR#3 for the "simple link" case (not across the network) don't require
     additional protocol
FR#4 minimally disruptive migration if new protocols are defined
FR#5 no (or minimal) oscillation - stability - bounded frequency of
     path change
FR#6 don't break network management

So these are base functional requirements.  FR#1 is key.  Expose
information about component links or groups of component links.

The derived requirements are supposed to provide more practical detail
on how there requirements can be met.

*CV* Please let me know if you would like me to split FR#1 in two.
Ths simplest way to provide a clarification is to create an a) and b)
sublist with the two points: a) expose info on component, b) but don't
affect convergence.  It would be better to just make the "expose info
on component" into FR#0 and then explicitly state that this must be
done in such a way that other requirements are met, particularly
FR#1-FR#6.

> DR#6 seems to be the
> better formulated to me.

DR#6 could refer back to FR#1.  DR#6 is where we imply that we need to
retain a mechanism to at least advertise groups of component links.
For example, this is so we don't have 80 TLVs with one component link
each if a 8TB link is made up of 80 x 100 Gb/s component links.

*CV* Do we need to be more clear that the FR# are functional goals
independent of the assumptions about how IGP and MPLS works as-is and
that the DR# set are more specific requirements that are derived from
those functional goals.

> > As to the "who", it is the provider but the protocols and
> > implementation have to provide the means for the provider to do so.
>  
> I think this point is now understood, assuming there's no substantive
> disagreement on the revised intro.

Your change to the first sentence is accepted.

> > As to the scope of the NPO: if the NPO is defined over a link, the
> > link needs to meet it; if it is defined as edge-to-edge within an
> > internal infrastructure, it must be met edge-to-edge; if it is defined
> > for a specific customer end-to-end, then it must be met end-to-end
> > (and may correlate strongly with a specific SLA).
>  
> Humm, the mechanisms needed to satisfy timing requirements between
> adjacent nodes can be radically different (simpler) than between network
> paths.  I'm not sure NPO-based requirements can be meaningful without
> some design targets/constraints.  At least ITU-T.Y.1541 provides some
> absolute time values.

This is why we need s/NPO/performance objective/.  We used NPO just to
avoid using SLA which implies a contract.

> > At what point does clarifying what is meant by an NPO and what it
> > means to meet an NPO just overly wordy and stating the obvious?
>  
> I think this is covered above.

Do you then agree that we should remove all reference to NPO and to
ITU-T.Y.1540 and ITU-T.Y.1541?

If so, then we have settled the definitions issue.  Again see text
regarding s/NPO/performance objective/ below (search for "3a.").

> >> - The scope of the composite links referenced in this document is not
> >> clear.  The document seems to be using G.800 terminology and concepts,
> >> yet the G.800 related summary is relegated to an Appendix.  If this CL
> >> is being driven/constrained by the G.800 CL, then this relationship
> >> should be made explicit and in the (early) body of the document, or by
> >> reference to another document that defines CL as such.  (In other words
> >> Appendix A should early in the document, perhaps in section 1.)
> > 
> > We give a definition for composite link and component link.  ITU gives
> > a definition for the same two terms.  We have give a summary of a few
> > ITU-T G.800 definitions in Appendix A.  I would be perfectly happy
> > getting rid of Appendix A and simply being more clear in our
> > definitions section.
> > 
> >  OLD
> > 
> >    ITU-T G.800 Based Composite and Component Link Definitions:
> >        Section 6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite and
> >        component links as summarized in Appendix A.  The following
> >        definitions for composite and component links are derived from
> >        and intended to be consistent with the cited ITU-T G.800
> >        terminology.
> > 
> >  NEW
> > 
> >    ITU-T G.800 Based Composite and Component Link Definitions: Section
> >        6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite and
> >        component links.  The following definitions for composite and
> >        component links are derived from and intended to be consistent
> >        with the cited ITU-T G.800 terminology, but differ slightly.
> > 
> > We can then point out that we are specifically excluding inverse
> > multiplexing from our definition of Composite Link and that we are not
> > using the ITU definition.
> > 
> > I would like to see us use the definition of multipath in
> > draft-ietf-mpls-multipath-use but this work started out using the term
> > composite link so we may be stuck with it.  Some key definitions in
> > draft-ietf-mpls-multipath-use are:
> > 
> >    Multipath
> >        The term multipath includes all techniques in which
> > 
> >        1.  Traffic can take more than one path from one node to a
> >            destination.
> > 
> >        2.  Individual packets take one path only.  Packets are not
> >            subdivided and reassembled at the receiving end.
> > 
> >        3.  Packets are not resequenced at the receiving end.
> > 
> >        4.  The paths may be:
> > 
> >            a.  parallel links between two nodes, or
> > 
> >            b.  may be specific paths across a network to a destination
> >                node, or
> > 
> >            c.  may be links or paths to an intermediate node used to
> >                reach a common destination.
> > 
> >    Composite Link
> >        The term Composite Link had been a registered trademark of Avici
> >        Systems, but was abandoned in 2007.  The term composite link is
> >        now defined by the ITU in [ITU-T.G.800].  The ITU definition
> >        includes multipath as defined here, plus inverse multiplexing
> >        which is explicitly excluded from the definition of multipath.
> > 
> >    Inverse Multiplexing
> >        Inverse multiplexing either transmits whole packets and
> >        resequences the packets at the receiving end or subdivides
> >        packets and reassembles the packets at the receiving end.
> >        Inverse multiplexing requires that all packets be handled by a
> >        common egress packet processing element and is therefore not
> >        useful for very high bandwidth applications.
> > 
> > We can also add the following:
> > 
> >    Classic Multipath
> >        Classic multipath refers to the most common current practice in
> >        implementation and deployment of multipath where multipath
> >        consist only of homogeneous links and in cases such as Ethernet
> >        Link Aggregation are entirely transparent to upper layer
> >        control planes.
> > 
> > These IMHO are more clear definitions.
> > 
> > If you prefer these, I'd be glad to put them in and then indicate that
> > inverse multiplexing is the only form of composite link that is out of
> > scope for this document.
>  
> Given that you stated that the use case document is the one that
> provides the definition/intro to IETF CLs, I'd just refer to that
> document and drop all references to G.800, including the Appendix, in
> this document.  Also use the following definitions:
>  
>        Composite Link:  As defined in [I-D.ietf-rtgwg-cl-use-cases].
>  
>        Component Link:  As defined in [I-D.ietf-rtgwg-cl-use-cases].
>  
> BTW I like your ideas WRT multipath, but I suspect that that ship has
> sailed.

Please explain your "ship has sailed" comment.

I am fine with the substitution above except that we now have a
normative refernce to the Use Cases document which I would rather
avoid.

I was under the (apparently very mistaken) impression that the
existing definition was sufficient with the reference to Use Cases
being intended to provide additional information.

If the definitions in draft-ietf-rtgwg-cl-requirement of "Composite
Link", "Component Link", and "Performance Objective" are not
sufficient, then we need to make them sufficient.  If that means that
we need to go back to the WG step, then we will have to do so.

*CV* Apparently based on the followup conversation, the ship just
returned to dock and will sail again shortly.  See the email appended
below.

> >> - On a more detailed level, it seems that this document really just has
> >> MPLS (composite link) over MPLS (component links) use cases/solutions in
> >> mind.  (Yes there is one line about other technology examples, but that
> >> is in just one spot and the MPLS is mentioned throughout.)  If this is
> >> correct, then this should be made explicit in the early part of the
> >> document.  If not, then the MPLS related assumptions/limitations need to
> >> removed.
> > 
> > That is not true.  The definition of component link should make this
> > clear.
> > 
> >        Component Link:  A point-to-point physical link (including one or
> >            more link layer) or a logical link that preserves ordering in
> >            the steady state.
> > 
> > If also gives examples which include Ethernet, PPP, SONET or OTN over
> > a physical link.  It also states "A set of link layers supported over
> > pseudowire is a logical link that appears to the client to be a
> > physical link."
> > 
> > In all cases it is MPLS over something, but not necissarily MPLS over
> > MPLS.  I don't see a basis for the statement "it seems that this
> > document really just has MPLS (composite link) over MPLS (component
> > links) use cases/solutions in mind".
> > 
> > Could you please point out any "MPLS related assumptions/limitations"
> > that are related to MPLS being used as a server layer rather than
> > using a link layer as a server layer.  I don't see any.  The existance
> > of section 4.2.  Component Links Provided by Lower Layer Networks does
> > not state, nor does it in any way imply that only MPLS lower layers
> > are being considered.  The one descriptive paragraph even gives
> > circuit switched or MPLS-TP as examples, further indicating that MPLS
> > over MPLS is not the sole focus.  This section gives four requirements
> > related to passing information from the sever layer up to the client
> > layer that apply to the case of a server layer that is more complex
> > than a simple physical link plus link layer.
>  
> Some of my comment was certainly biased by other drafts, including the
> framework draft which more clearly shows this.  In terms of this
> document, I don't see how you expect arbitrary technologies used to
> provide component links to meet requirements FR#7 and FR#8.  Do you
> really expect them to be met by non-MPLS (or perhaps GMPLS) controlled
> technologies?   Did you think about making these objectives, or limited
> to MPLS/GMPLS controlled networks, rather than a requirement?

There is a lot of discussion in the framework draft as it now stands
of the problematic cases.  Most of the problematic cases are unique to
MPLS over MPLS and do not occur in the MPLS over simple link case.

*CV* This is just a point of clarification as to why so much of the
framework focuses on the MPLS over MPLS case.

> Also, I read FR#11 as the labeling of the component not composite link,
> otherwise why mention "labeled traffic flow".

We need to reword FR#11.  The point is to get identified flows or
groups of flows assigned to components.  The existing hashing
techniques move around groups of flows where a group is those that
hash to the same value.  To make the dynamic techniques work,
measurement of the traffic contribution of a group avoids trial and
error movement of groups of flows in response to traffic imbalance.

 OLD:

   FR#11  The solution SHALL measure traffic on a labeled traffic flow
          and dynamically select the component link on which to place
          this flow in order to balance the load so that no component
          link in the composite link between a pair of nodes is
          overloaded.

 NEW:

   FR#11  The solution SHALL measure traffic flows or groups of
          traffic flows
          and dynamically select the component link on which to place
          this traffic in order to balance the load so that no component
          link in the composite link between a pair of nodes is
          overloaded.

A group of flows can be identified by label stack, by IP headers, or
cheating a bit and looking below the label stack at the IP headers.
PWE3 CW acknowledges that that form of cheating is widespread and
rectifies it for PW.  Entropy Label makes it no longer necessary to do
that form of cheating (once sufficiently deployed).

*CV* Please let me know if the new wording is OK.

> >> Minor Issues:
> >>  
> >> - Abstract
> >>  
> >> This whole section seems to be some early text that hasn't been
> >> revisited as the document progressed.  I'd suggest throwing it out and
> >> just picking up (with slight modification) the Abstract from the
> >> cl-framework document which seems pretty good.
> > 
> > Alia asked for an edit to the existing abstract.  With the change that
> > Alia requested the abstract is now:
> > 
> >    There is often a need to provide large aggregates of bandwidth that
> >    are best provided using parallel links between routers or carrying
> >    traffic over multiple MPLS LSP.  In core networks there is often no
> >    alternative since the aggregate capacities of core networks today far
> >    exceed the capacity of a single physical link or single packet
> >    processing element.
> > 
> >    The presence of parallel links, with each link potentially comprised
> >    of multiple layers has resulted in additional requirements.  Certain
> >    services may benefit from being restricted to a subset of the
> >    component links or a specific component link, where component link
> >    characteristics, such as latency, differ.  Certain services require
> >    that an LSP be treated as atomic and avoid reordering.  Other
> >    services will continue to require only that reordering not occur
> >    within a microflow as is current practice.
> > 
> > The point of the second paragraph is that some LSPs have specific
> > requirements (such as bounded delay) and some (most) do not.
> > Flexibility in accommodating both is the requirement.  Composite Links
> > supporting non-homogenous Component Links is the basis for the
> > solution.  That is why the framework mentions "group of homogenous or
> > non-homogenous links".
> > 
> > The fact that Alia just commented on and we just changed the abstract
> > is indication that it has been read and is not just left over text
> > from ancient times that no longer applies.  Could you please be more
> > specific about your objections to this text.
>  
> It's also an indication that it needed some work.  Given that you say
> the use-case is the source for IETF CL definition and that the abstract
> provides so much more information than the Intro, I'd suggest replacing
> with something really simple, perhaps along the lines of:
>  
>    This document provides a set of requirements for MPLS composite
>    links.
>  
>    Composite link is a formalization of multipath techniques currently
>    in use in IP and MPLS networks and a set of extensions to existing
>    multipath techniques.

Agree that Alia's last minute comment indicates the abstract could be
better.

I like your substitution and would not mind moving the existing text
in the abstact to the Intro.

*CV* Replacing the abstract as you suggest is called for in the
summary in the appended email below.

> >> - 1. Introduction
> >>  
> >> So where does one go to get an intro/definition of the IETF work on
> >> composite links?  Is it G.800, the framework draft, the use cases draft,
> >> this draft? If the last, then this section needs a serious revision.  If
> >> not, then this document should reference that definition and address the
> >> major issues identified above.
> > 
> > I'd say "Use Cases".  I'm in favor of de-emphasizing G.800.
>  
> (Covered above).

Given the two choice you gave, Use Cases was the better choice, but as
stated above an even better way to handle this is to put a correct set
of defintions in draft-ietf-rtgwg-cl-requirement (this document) and
not refer to G.800 at all and refer to Use Cases only for further
information on current practices (the intent).

*CV* This is settled.  See appended email below.

> > Please be clear.  What major issues?
>  
> "Major Issues" as identified in the RtgDir review template and above.
>  
> > Do you means the definitions of the well known terms CL and NPO?  
>  
> Yes.

My preference is to put definitive definitions into
draft-ietf-rtgwg-cl-requirement (this document).  I proposed
definitions on this thread but I think this change would be enough to
require another round trip through a WGLC and so on back to this
point.  If we are in agreement that putting definitive definitions
into this document is the best option, then I recommend we go with the
ones cited above (prior to your "ship sailed" comment).

*CV* This is settled.  See appended email below.

> > If
> > so, how would you like them changed (see above).
> As covered above, reference use case document and ensure that the terms
> are appropriately defined there.

I don't think that is the best option but if others feel this is the
best way to go, then I'll go along.

The problem that I anticipate is that the Use Cases document, being
one which describes current practices of the two dominant router
implementations and may other but not all, we could end up with push
back on that document.  Therefore I would prefer it to be ancilliary
information only and not listed as normative anywhere.

> > Do you mean your belief that this document only applies to MPLS over
> > MPLS?  In that case, you seem to be objecting to the intent of the
> > document even when your interpretation of the intent is contradicted
> > by many statements made in the document.  If so, how can we make it
> > even more clear that a component link can also be a physical link,
> > circuit switched, pseudowire, besides mentioning that in the
> > definition of compnent link and in numerous other places.
>  
> Covered above.

OK.

> >> - 2 Assumptions
> >> I suspect this section will need to be updated to address the document
> >> scope comments above.
> > 
> > AFAIK there is no change to the document scope.
>  
> Given CL is defined in use cases, it seems more appropriate to me that
> Assumptions be presented as part of the definition.  In other words,
> assuming the definition of CL is in the use case document, I think this
> it makes most sense for text to be in/moved to the use cases document.

At this point you have suggested a replacement abstract (above).  I
have agreed to that and suggested that the existing abstract be moved
into the introduction.  This is a lot of change, but I'm OK so far.

You are now suggesting here that the existing "Assumptions" section be
moved to the Use Cases document.  I don't feel that is necessary, but
I'm OK with it.  I would prefer that the "Assumptions" section stay
where it is.

*CV* All of this is called for in the summary in the email appended
below.

> >> - Definition of NPO.
> >> It seems to me that the authoritative definition of NPO is Y.1541 with
> >> Y.1540 providing additional detail.  I think these documents would be a
> >> much more reasonable reference for NPO than the CL use case draft as is
> >> currently used.
> > 
> > Would it be OK to add this text.
> > 
> >   Network operators have a contractual Service Level Agreement (SLA)
> >   with customers for services that are comprised of numerical values
> >   for performance measures, principally availability, latency, delay
> >   variation.  Additionally, network operators may have Service Level
> >   Specification (SLS) that is for internal use by the operator.  In
> >   this document we use the term Network Performance Objective (NPO)
> >   which may span multiple networks and may be looser than network
> >   operator SLA or SLS objectives.  The use of the term NPO in this
> >   document is not entirely consistent with its definition in Y.1540
> >   and Y.1541 in that it is not restricted to UNI to UNI.  There is no
> >   intent to imply that a provider must base their SLA, SLS, or NPO on
> >   Y.1540 and Y.1541.
> > 
> > If we stay consistent with Y.1540 and Y.1541 then I think we have to
> > s/NPO/SLS or NPO/ and we did want a single term.
>  
> Hereto, given the position that the definition of CL is in the use case
> document, I think the original reference makes sense. (Although I'd
> probably just say "Network Performance Objective (NPO): as defined in
> [I-D.ietf-rtgwg-cl-use-cases].")

We can push this document through by moving these definitions out and
returning to the problem later, but I would rather address the problem
in this document.

*CV* No one liked that option and therefore it is off the table.  See
appended email below.

> >> - Many of the requirements refer to "meeting" or "not violating" NPO
> >> without given any specifics as to which NPO parameter/parameters is/are
> >> relevant. For example Class 5 QoS has unbounded upper bounds on Network
> >> performance parameter.  Also NPO is defined UNI to UNI (end to end). I
> >> suggest adding a level of detail identifying the parameters and how the
> >> fixed values are to be applied relative to "meeting" or "not violating" NPO.
> > 
> > Please see comments above regarding use of the term NPO.
>  
> dito on the response ;-)
>  
> > 
> > We wanted one term.  Diffserv uses SLA, even though there may be no
> > agreement with an outside party.  If we can't use SLS because it is
> > specific to one provider only, can't use NPO because in Y.154[01] it
> > applies only to UNI to UNI, then we need another term.
> > 
> > This reminds me of the host and router vs ES and IS arguments back in
> > the 1990s and the issue of using the term subnet to mean what it meant
> > in IETF since the early 1980s, ignoring what ITU had defined it to
> > mean.  Eventually we dumped the use of Logical IP Subnetwork (LIS) as
> > we had to do in RFC1577 and could just say "subnet" again.
> > 
> > Perhaps the best course of action is s/NPO/performance objective/ and
> > then define performance objective without any reference to Y.1540 and
> > Y.1541, but perhaps referring to SLA as defined in RFC2475.
>  
> Independent of the terminology conundrum, you still have to figure out
> how to provide requirements that are meaningful in designing new
> protocols/mechanisms.  I know this is a very difficult point, but
> without do so, it's hard to see how any solution can be evaluated
> against a requirement.

This document is not trying to mandate Y.1540 and Y.1541 nor is it
trying to define the set of performance objectives that a provider
must adhere to.

*CV* This is settled.  See appended email below.

> BTW I really like the (relative terms) approach taken by DR#6 and DR#7
> as they avoid any of the complications inherent in a specific-value
> performance parameter approach (such as seen in Table 1 of Y.1541)

That is why we should get rid of all reference to Y.1540 and Y.1541.
I never wanted those references in there in the first place just to
pick up a definition of a term and try to otherwise distance ourselves
from it.  Your interpretation that we are advocating using Y.1541
parameters, is so far a unique interpretation, but it is better to
avoid any potential confusion and completely eliminate references to
Y.1540 and Y.1541.

*CV* Save comment as above.

> >> - It might be useful to somehow separate data plane requirements from
> >> control plane requirements (Management/OAM requirements are already
> >> separated.)
> > 
> > Too many of the requirements involve both the control plane and the
> > data plane.  Where we are talking solely about physical links and
> > local decisions can be made, then no control plane signaling is
> > required.  For some requirements, extensions to signaling would be
> > required for the MPLS over MPLS case.
> > 
> > A good example is path symetry.  In the framework, it is pointed out
> > that the most general case of path symetry on a per client LSP basis
> > (FR#22 and FR#23) is close to infeasible where there are multiple
> > client/server layer relationships.  FR#22 and FR#23 works for classic
> > link bundling where each component link takes a single component
> > bidirectional path, but would require a lot of extension to work for
> > MPLS over MPLS over MPLS ad nauseum.
> > 
> > Whether the solution is done in the data plane or control plane or
> > using a little contribution from each is a matter for the framework
> > document.  The requirement is simply that an objective be met.
> > 
>  
> okay. Accepted & point closed.
>  
> >> Nits:
> >>  
> >> I have a bunch of more nit level comments, but I suspect much will
> >> change if/when the above are addressed, so I'm going to defer the
> >> majority of them until the above issues are resolved.
> > 
> > I would like one more response from you before making any changes to
> > the document.
> > 
>  
> I agree, it makes sense to resolve the higher level comments before
> jumping on changes (and then on to nits.)

Obviously we still need at least one more iteration (maybe more).

> >> - It looks like you are referencing an old version of g.800.  The
> >> current is G.800 (02/2012). Also the G.800 "Cases" are identified in the
> >> draft not in G.800.
> > 
> > I'm in favor of removing reference to G.800 except as a point of
> > clarification to say that they also define Composite Link and do so
> > somewhat differently.
> > 
>  
> I think I go a step further in my comments above and say drop it all
> together (and let it be covered in the CL-defining use case draft).

We need to resolve that before going forward.  In either case we would
be dropping references to G.800, Y.1540, and Y.1541.  We then have two
options:

  1.  Reference use cases and fix up the definitions later.

  2.  Fix up the definitions here.

I think that in either case the changes that you are calling for are
enough to require a quick pass back through WGLC, AD, IESG, IETF LC,
and back to this point in the process.

*CV* We will make changes as agreed to and repeat WGLC as per summary
in email appended below.

> >> Lou
> > 
> > There is a lot of discussion above, mostly around definitions and
> > assumptions about document scope.
> > 
> > In summary:
> > 
> >   1.  Remove the phrase "business problems" as described above.
> >       Please let me know if the OLD/NEW change on this is sufficient.
> > 
> >   2.  In section 2, change "The services supported include ..." to
> >       "Services which may be supported over MPLS based Composite Links
> >       include ...".
> > 
> >   3.  Resovle issue of defintion of NPO.
> > 
> >       3a.  My preference is s/NPO/performance objective/.  Remove
> >            definition of NPO.  Replace with:
> > 
> >       Performance Objective: Numerical values for performance
> >          measures, principally availability, latency, and delay
> >          variation.  Performance objectives may be related to Service
> >          Level Agreements (SLA) as defined in RFC2475 or may be
> >          strictly internal.  Performance objectives may span links,
> >          edge-to-edge, or end-to-end.  Performance objectives may span
> >          one provider or may span multiple providers.
> > 
> >       3b.  Alternate is use the paragraph clarifying that we don't
> >       	   quite mean NPO as defined in Y.154[01].  I'd rather just
> >       	   get rid of the term NPO.
> > 
> >   4.  Replace definitions for "Composite Link" and "Component Link"
> >       with those from draft-ietf-mpls-multipath-use and include
> >       "Multipath" and "Inverse Multiplexing", plus "Classic Multipath"
> >       as suggested above.
> > 
> >   5.  State that "Composite Link is also defined in ITU G.800 but the
> >       usage here specifically excludes inverse multiplexing and is
> >       more similar to multipath but with extensions to support groups
> >       of homogeneous or non-homogeneous links.  Remove all other
> >       references to G.800.
> > 
> >   6.  Resolve wording on abstract.  Please be more specific about your
> >       objections to this text.
> > 
> >   7.  There is no change to document scope.  Please indicate why you
> >       think the document scope is not stated correctly.
> > 
> >   8.  Don't try to separate data plane vs control plane.  In too many
> >       cases that is more applicable to solution space than it is to
> >       requirements.
> > 
> > Please at least comment on only the summary before I make changes.
> > 
>  
> I think I responded to all the detail points.  Let me know if I missed
> anything.
>  
> Thanks again for the detailed response.
> Lou
>  
> > Curtis

I think you did respond to all points.  I think I understand all of
your objections and your suggestions.  The major point remaining may
be whether we put definitions in Use Cases or put sufficient
definition in this document as originally intended.

I'm not entirely sure that you will agree with all of my points as it
didn't seem clear to me in this response from you that you understood
that our intent was to neither encourage or discourage the use of
Y.1540 and Y.1541 but to just make use of a common term which turned
out to be a mistake on our part.  We should define "Performance
Objective" as above in "3a" (or similar if you have better text).

Curtis


*CV* The rest of this email is the summary email that is referred to
above.  Other than deleting a lot of the extra email headers that we
don't need, the email is as Alia sent it.


From: Alia Atlas <[e-mail]>
To: "[e-mail]" <[e-mail]>,
        Responsible AD
	<[e-mail]>,
        Authors
	<[e-mail]>,
        Trouble Maker
	<[e-mail]>,
        Scott Mansfield <[e-mail]>,
        "BRUNGARD, DEBORAH A" <[e-mail]>,
        John E Drake <[e-mail]>,
        "Alvaro Retana (aretana)" <[e-mail]>, Tom Yu <[e-mail]>
Subject: RE: issues with draft-ietf-rtgwg-cl-requirement and ITU-T docs
Thread-Topic: issues with draft-ietf-rtgwg-cl-requirement and ITU-T docs
Thread-Index: AQHOc1H8qNjpqG+1MkmzUYioNv8rlJlJvsvw
Date: Thu, 27 Jun 2013 16:25:32 +0000
Message-ID: <[e-mail]>
References: Your message of "Tue, 25 Jun 2013 13:17:29 EDT."
             <[e-mail]>
 <[e-mail]>
In-Reply-To: <[e-mail]>
Accept-Language: en-US
Content-Language: en-US

Curtis,

Quick top-posted clarification:

Stewart modified 3 as follows:
   "I do not think that it is necessary to roll out new draft names, just
   put a note in the draft (to be deleted by the RFC editor) explaining
   the name change.

   You may want to make a similar note somewhere in the body of the text
   noting the ambiguity and defining your new term."

That is what I'd recommend - rather than rewriting and issuing it as a 00.

Alia


-----Original Message-----
From: Curtis Villamizar [mailto:[e-mail]] 
Sent: Thursday, June 27, 2013 12:19 PM
To: Alia Atlas; Responsible AD; Authors; Trouble Maker; Scott Mansfield; BRUNGARD, DEBORAH A; John E Drake; Alvaro Retana (aretana); Tom Yu; [e-mail]
Subject: Re: issues with draft-ietf-rtgwg-cl-requirement and ITU-T docs


Alia, Stewart,

Top posting a summary and the plan of action based on the final
response from Alia (document shepard and WG chair).  Also adding
everyone to the Cc that got added at one point or another.  Also added
Tom, who sent the SecDir comments.

In this case top posting summary replies was helpful.  Here are the
set of responses.

 Alia Atlas:

   Curtis

   I like Option 3.  I think the work has changed over the years.  This
   is our last chance to change it and if we're taking it back into the
   Working Group, we may as well.

   Alia

 Andy Malis:

   Alia,

   Like Curtis, I prefer option 1 with the changes he proposed. It's
   clear that industry use of Composite Link precedes G.800.

   However, that said, if that's going to be impossible to get through
   the IESG (Responsible AD, can you comment on that?) then option 3 is
   my second choice.

   I've added to the cc list the IETF's liaison managers to ITU-T, as
   their opinions will also be very relevant.

   Cheers,
   Andy

 Lou Berger:

   Curtis,

   Thanks for capturing the implications of our discussion so succinctly.
   I also appreciate that you are the first in the IETF to finally
   recognize the proper spelling of my name!

   Lou

 Alia Atlas:

   Andy,

   I am fine with either Option 1 or 3.  I just wouldn't rule out 3 due
   to timing.

   Alia

 Stewart Bryant:

   All,

   Unless it is a showstopper, I prefer 3.

   1 & 2 are potentially confusing if there is a conflict of
   definitions. I would prefer that we not use political capital on the
   ITU over this.

   4 just ducks the issues which will bite us later.

   If we go with 3, can I suggest that one of the chairs writes a note to
   the WG explaining what the plan is.

   I do not think that it is necessary to roll out new draft names, just
   put a note in the draft (to be deleted by the RFC editor) explaining
   the name change.

   You may want to make a similar note somewhere in the body of the text
   noting the ambiguity and defining your new term.

   Stewart

 Deborah Brungard:

   As liaison, I'll go with what you decide.

   As ccamp co-chair doing the heated years of discussion with ITU on
   ASON, I'd advise to avoid ITU terms and definitions unless that really
   is the intention for the work. ITU still believes they invented the
   concept of multi domain control planes so we are very careful when
   referencing ASON.

   Deborah

 John Drake:

   3.

   Yours Irrespectively,

   John

 Lucy Yong:

   In favor of 3.

   Lucy

 Ning So:

   I am OK with either Option 1 or 3.

   Best regards,

   Ning So

 Alia Atlas:

   So - it sounds generally like we should bring the draft back to the WG
   and do 3 as Stewart suggests.

   If that's ok with the authors, could we get a new version out and I
   can send email to the WG.

   Alia

As Alia is document shepard, the next step should be based is Alia's
final call.  There was support for options 1 and 3, with strong
arguments for 3 rather than 1.

Alia call is to go back the the WG with option 3 which is:

>   3.  Use the term "Advanced Multipath" and define that to mean
>       "Classic Multipath" plus the requirements and extensions defined
>       in the set of documents.  Rename the documents to -amp- rather
>       than -cl- and change titles.
>       s/Composite Link/Advanced Multipath/.

Therefore I will do the following:

  1.  Repond to the thread started as part of the RtgDir review and
      include this message if no one on the Cc objects.

  2.  Edit draft-ietf-rtgwg-cl-requirement and produce a 00 version
      with the following changes:

      a.  Rename the document to draft-ietf-rtgwg-amp-requirement-00
          and with the title "Requirements for Advanced Multipath in
          MPLS Networks".  s/Composite Link/Advanced Multipath/ and
          make any other changes to reflect the change in terminology.

      b.  Remove the deinitions of "Composite Link", "Component Link",
          and "Network Performance Options".  Replace with definitions
          consistent with the discussion below.

      c.  Make the following changes to the Abstract, Introduction,
          and Assumptions section, based on Last Call comments.

          i.    Replace abstract with wording based on RtgDir
                comments from Lou.

          ii.   Move the existing abstract to the Introduction.  Make
                the Last Call change to that wording suggested by
                Alia.

          iii.  Clean up the existing wording in the Introduction
                based on RtgDir comments from Lou.

          iv.   Move the Assumptions section to the Use Cases
                document, as suggested in RtgDir comments from Lou.

      d.  Make changes to the document to expand all acronyms and
          include some in the definitions.  Stewart suggested this
          earlier as a Last Call comment.  The same suggestion was
          made as a side comment in the SecDir comments from Tom.

      e.  Replace the Security section as per comments in the SecDir
          thread, unless Tom objects to the new wording.

      f.  Make wording more clear in specific requirements,
          particularly some of the FR requirements based on RtgDir
          comments from Lou.

  3.  It will then be up to Alia and Alvaro a WG chairs to decide when
      to repeat the WGLC so I will wait for the WGLC.

It is helpful to note that none of the comment impact the requirements
themselves, except improving the clarity of wording.  The changes are
mostly in the Abstract, Introduction, and Assumptions sections.

Alia, Alvaro, Stewart - please confirm that the document rename
  from draft-ietf-rtgwg-cl-requirement-10
  to draft-ietf-rtgwg-amp-requirement-00 
is to be done.  If so, something has to be done to get the submission
tool to accept the new document name and to record the transition.

Curtis


In message <[e-mail]>
Curtis Villamizar writes:
> 
>  
> Alia, Stewart,
>  
> I've copied the co-authors and Lou on the Cc.
>  
> Lou and I spoke on the phone.  Lou brought up some important points
> regarding our use of ITU-T defined terms.  Here are my suggestion for
> how to go forward.
>  
> With the document as-is we may be handing ITU-T a hand grenade that
> they can pull the pin on at any time and lob back at us.
>  
> IETF rightfully reacted strongly when ITU-T appropriated and misused
> the term MPLS in their definition of T-MPLS.  We are now citing ITU-T
> documents yet redefining the terms Composite Link, Component Link, and
> NPO defined in those documents.
>  
> Y.1541 has a very strict definition of NPO including absurdly long
> prescribed values for delay based on DSCP value.  That is not a
> definition we want to use and therefore we MUST NOT use the term NPO.
> The solution here is clear and simple s/NPO/Performance Objective/ and
> put in out own definition of "Performance Objective".
>  
> The case is less straightforward for the terms Composite Link and
> Component Link.
>  
> If ITU-T objects, we then would have to explain that we are using the
> term "Composite Link" which was an accepted technical term before
> ITU-T started using it, but not using their definition.  We can argue
> that this is not like MPLS because there was no registered tradement
> or general use of the technical term MPLS before IETF defined it,
> whereas the term "Composite Link" was already in general use as a
> generic term before ITU-T started using it.  This is similar to the
> arguments surrounding IETF and ITU use of the term "subnet" or
> "subnetwork" which means different things to IETF and ITU and today
> can be disambiguated by saying "We mean IP subnet in this document
> when we say subnet".  Similarly in the future, "We mean IP/MPLS
> Composite Link when we say Composite Link" would disambiguate the IETF
> and ITU-T terms (and Avici use FWIW).
>  
> What Lou means by "this ship has sailed" is it is way too late to
> rename the set of documents to something like "Requirements for
> Advanced Multipath in MPLS Networks", "Advanced Multipath Use Cases
> and Design Considerations", and "Advanced Multipath Framework in MPLS
> Networks".  This would be the cleanest course of action with regard to
> not upsetting anyone at ITU-T.
>  
> My suggestion is that we bring the issue to the WG of definition of
> key terms, potential confusion resulting from our references to G.800,
> Y.1540, and Y.1541, and desire not to conflict with ITU-T terms and
> see what option is favored by the WG.
>  
> Options are:
>  
>   1.  Use the term "Composite Link" and redefine it.
>       (potentially politically problematic, but tractable)
>  
>   2.  Use the term "Composite Link" as-is from ITU-T.
>       (potentially technically problematic)
>  
>   3.  Use the term "Advanced Multipath" and define that to mean
>       "Classic Multipath" plus the requirements and extensions defined
>       in the set of documents.  Rename the documents to -amp- rather
>       than -cl- and change titles.
>       s/Composite Link/Advanced Multipath/.
>  
>   4.  Punt for now, reference Use Cases and put definitions there for
>      "Composite Link" using one of the above approaches.
>  
> Component Link need not change because it is already used in RFC4201
> (Link Bundling) which preceded G.800 by two years.  (Published 2005,
> first draft 2001 vs published 2007, first draft 2005).
>  
> The use of the term Composite Link also predated G.800 (for example
> the Avici registered trademark was issued in 2004 and the trademark
> may have been used as early as 1998 before being registered, see
> http://www.trademarkia.com/composite-links-78363042.html or
> https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&ved=0CDIQFjAB&url=http%3A%2F%2Fwww.businesswire.com%2Fnews%2Fhome%2F20050607005203%2Fen%2FAvici-Systems-TSR-Composite-Links-Deliver-Top&ei=Q7jJUeHoGZPH0AGU4IC4CQ&usg=AFQjCNEwEcQlIa7AhWySJ6yK9C7zFSCi9g&bvm=bv.48293060,d.dmQ
> , or just google "composite link avici" ).
>  
> BTW- If you google Composite Link you will find it used in biology for
> quite some time and in things like plastic chain link fencing long
> before any use in networking.  :-)
>  
> I like option 4 the least and don't care for option 2.  I'm fine with
> option 3, but acknowledge that Lou's "this ship has sailed" comment is
> right.  Therefore we have option 1.  If we can't do option 1 for some
> reason, I would favor the tough road of option 3.
>  
> IMHO we can use the terms Composite Link, and Component Link and have
> strong ground to stand on that both terms were either used by IETF or
> generic before ITU-T began using them.  We should not mention G.800
> except to note that they use the same term and their definition
> differs slightly.  We should not use the term NPO at all and should
> delete references to Y.1540 and Y.1541.
>  
> If we go with option 1, the definitions can remain unchanged.  We
> would drop some text on top and add a note at the end of the
> definitions sections.
>  
>  DELETE:
>  
>    ITU-T G.800 Based Composite and Component Link Definitions:
>        Section 6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite and
>        component links as summarized in Appendix A.  The following
>        definitions for composite and component links are derived from
>        and intended to be consistent with the cited ITU-T G.800
>        terminology.
>  
>  ADD (to end of section 3 Definitions):
>  
>    Note: The term Composite Link had been a registered trademark of
>    Avici Systems since 2004, but was abandoned in 2007.  In Avici's
>    use of the term, Composite Link applied solely to IP/MPLS networks.
>    The term Composite Link was later defined by the ITU [ITU-T.G.800].
>    The ITU definition includes multipath as defined here, plus inverse
>    multiplexing which is explicitly excluded from the definition of
>    Composite Link used here.  The term Component Link is used in
>    RFC4201 and also defined in G.800.  The term Component Link as
>    defined here differs slightly from RFC4201 and G.800.  Both terms,
>    "Composite Link" and "Component Link" predated any ITU-T use of
>    these terms in G.800 and are considered to have been widely used
>    technical terms prior to G.800.  To be perfectly clear, the G.800
>    definitions and descriptions of Composite Link and Component Link
>    may not perfectly agree with this document.
>  
> Also delete all other references to G.800 including all of Appendix A.
>  
> Lou has also called for a new abstract and changes to the introduction
> section.  I'll propose changes to those sections later.
>  
> If this is OK with everyone, I will edit the response below to Lou and
> then send diffs.  If Alia or Stewart agree with this approach then one
> of them can then recommend that we take this back to the WG and we can
> have a repeat WGLC and IETF LC emphasizing the changes.
>  
> I'm sending this prior to sending the response below to the much wider
> Cc (after getting responses to this email and then editing).
>  
> Curtis
>  
>  
> ******************** DRAFT RESPONSE ********************
> * 
> * ------- Forwarded Message
> *  
> * Message-Id: <[e-mail]>
> * To: Lou Berger <[e-mail]>
> * cc: [e-mail]
> * Reply-To: [e-mail]
> * From: Curtis Villamizar <[e-mail]>
> * Subject: PREVIEW: Re: RtgDir review: draft-ietf-rtgwg-cl-requirement-10
> * In-reply-to: Your message of "Mon, 24 Jun 2013 19:14:38 EDT."
> *              <[e-mail]>
> * Date: Mon, 24 Jun 2013 23:21:46 -0400
> *  
> *  
> * Lou,
> *  
> * This is a preview.  Cc is empty.  We can discuss this on the phone
> * tomorrow.  If we don't hook up, I'll just send it.
> *  
> * Please don't reply to this email.  Reply after the Cc is added.
> *  
> * If we talk I'll edit the response to reflect any agreement we come to.
> *  
> * Besides technical points, one thing I would like to understand is,
> * given that we are making a lot of last minute change, where does this
> * put us in the process and how would you like this to go forward.
> *  
> * Curtis
> *  
> *  
> * In message <[e-mail]>
> * Lou Berger writes:
> * > 
> * > Hi Curtis,
> * >  
> * > Thank you for the detailed response. I have included specific responses
> * > in-line below.  Before we go there, I'd like to jump ahead to one point
> * > that is (indirectly) pointed to in a number of spots.
> *  
> * It would have helped if this had come up in WGLC.  If we need to
> * change the document enough that we need to go back to that step, then
> * maybe we need to notify MPLS and CCAMP of the RTGWG WGLC.
> *  
> * > I asked:
> * > >> So where does one go to get an intro/definition of the IETF work on
> * > >> composite links?  Is it G.800, the framework draft, the use cases
> * > >> draft, this draft? ...
> * >  
> * > Your response is:
> * > > I'd say "Use Cases".  I'm in favor of de-emphasizing G.800.
> * >  
> * > Okay, then I think a bunch of detailed comments can be replaced by using
> * > references. More on this below. The issue with the use case document
> * > being the authoritative definition for CL (and presumably NPO?) is that
> *  
> * It would be best if the authoritative definitions where in
> * draft-ietf-rtgwg-cl-requirement (this document).  I would not like to
> * see draft-ietf-rtgwg-cl-use-cases have to be a normative reference
> * over the definitions of Composite Link and Performance Objective.
> *  
> * My preference is to embed the authoritative definitions for all of the
> * composite link work in this document.  This also removes the circular
> * references.
> *  
> * > I think you currently have circular references.
> * > draft-ietf-rtgwg-cl-use-cases starts out:
> * >  
> * >    Composite link requirements are specified in
> * >    [I-D.ietf-rtgwg-cl-requirement].  A composite link framework is
> * >    defined in [I-D.ietf-rtgwg-cl-framework].
> * >  
> * > BTW ietf-rtgwg-cl-framework adds
> * >  
> * >    Classic multipath, including Ethernet Link Aggregation has been
> * >    widely used in today's MPLS networks [RFC4385][RFC4928].  Classic
> * >    multipath using non-Ethernet links are often advertised using MPLS
> * >    Link bundling.  A link bundle [RFC4201] bundles a group of
> * >    homogeneous links as a TE link to make IGP-TE information exchange
> * >    and RSVP-TE signaling more scalable.  A composite link allows
> * >    bundling non-homogenous links together as a single logical link.  The
> * >    motivations for using a composite link are descried in
> * >    [I-D.ietf-rtgwg-cl-requirement] and [I-D.ietf-rtgwg-cl-use-cases].
> * >  
> * >    A composite link is a single logical link in MPLS network that
> * >    contains multiple parallel component links between two MPLS LSR.
> * >    Unlike a link bundle [RFC4201], the component links in a composite
> * >    link can have different properties such as cost, capacity, delay, or
> * >    jitter.
> * >  
> * > I think answering this question (of which document has the CL
> * > definition) and ensuring the documents line up accordingly would be
> * > really helpful.  -- I'll assume it's the use case draft for my comments
> * > below.
> *  
> * I'd rather we put the authoritative definitions directly in the
> * requirements document.  See block of text very far below.
> *  
> * > At the nit level, the defining document should be listed as a Normative
> * > Reference by this (and other CL) documents.
> *  
> * For that reason I would rather move the definition.  I would also like
> * to get it right in the first step of this process and have Use Cases
> * follow as strictly an informtional document about current practices,
> * then have the framework follow that.
> *  
> * > On 6/20/2013 3:51 PM, Curtis Villamizar wrote:
> * > > In message <[e-mail]>
> * > > Lou Berger writes:
> * > >  
> * > >> Hello,
> * > >>  
> * > >> I have been selected as the Routing Directorate reviewer for this draft.
> * > >> The Routing Directorate seeks to review all routing or routing-related
> * > >> drafts as they pass through IETF last call and IESG review, and
> * > >> sometimes on special request. The purpose of the review is to provide
> * > >> assistance to the Routing ADs. For more information about the Routing
> * > >> Directorate, please see http://www.ietf.org/iesg/directorate/routing.html
> * > >>  
> * > >> Although these comments are primarily for the use of the Routing ADs, it
> * > >> would be helpful if you could consider them along with any other IETF
> * > >> Last Call comments that you receive, and strive to resolve them through
> * > >> discussion or by updating the draft.
> * > >>  
> * > >> Document: draft-ietf-rtgwg-cl-requirement-10
> * > >> Reviewer: Lou Berger
> * > >> Review Date: 6/19/2013
> * > >> IETF LC End Date: 2013-06-19
> * > >> Intended Status: Informational
> * > > 
> * > > Hi Lou,
> * > > 
> * > >> Summary:
> * > >>         I have significant concerns about this document and recommend
> * > >> that the Routing ADs discuss these issues further with the authors.
> * > > 
> * > > OK.  Lets go through them.
> * > > 
> * > >> Comments:
> * > >>  
> * > >> 	My comments really fall into three categories: (a) understanding
> * > >> document objectives, (b) comments/questions on technical details, and
> * > >> (c) readability/editorial.  As this document is informational and does
> * > >> not represent IETF consensus, the AD together with the document Shepherd
> * > >> and WG may reasonably choose to publish the document as is.
> * > > 
> * > > Does not reflect IETF consensus?
> * > > 
> * >  
> * > Sure, see http://tools.ietf.org/html/rfc5741, informational documents
> * > can represent IETF consensus, but don't need to; it depends on LC and is
> * > represented in the boilerplate.  The datatracker represents this and
> * > says "Consensus: says unknown". I translated this into no request for
> * > IETF consensus was to be made -- I accept that this may be a mistake on
> * > may part. (Perhaps it was wishful thinking ;-)
> * >  
> * > The document Shepherd should be able to answer this.
> * >  
> * > Alia?
> *  
> * Awaiting response from Alia.
> *  
> * > >> Major Issues:
> * > >>  
> * > >> - At the highest level the objective/purpose of the document is unclear
> * > >> to me.  It states:
> * > >>  
> * > >>    The purpose of this document is to describe why network operators
> * > >>    require certain functions in order to solve certain business problems
> * > >>    (Section 2).
> * > > 
> * > > Forward reference for brevity in "1.  Introduction".  Is that bad?
> * >  
> * > Brevity is great, but what's that quote "As brief as possible (but no
> * > briefer)"...
> *  
> * The quote is Einstien.  Simple as possible and no simpler.
> *  
> * > I genuinely am confused as to the purpose of the document so it's my
> * > conclusion that the Intro is too brief. Other conclusions are of course
> * > possible.
> *  
> * You are right.  The Introduction is too short (and some sentences are
> * poorly worded).  See further comments below regarding your suggested
> * replacement for the first sentence.
> *  
> * > > Section 1 is brief and forward references sections 2, 4, 5, and
> * > > appendix A.
> * > > 
> * > >> Thankfully (for hopefully obvious reasons), section 2 covers
> * > >> "Assumptions" and not "business problems". Section 4 does provide what
> * > >> is termed "Network Operator Functional Requirements", which relies on
> * > >> Y.1541 defined Network Performance Objectives.  While this section title
> * > >> appears to be defining provider requirements, the document also appears
> * > >> to be placing requirements on CL solutions.  To add to the confusion, it
> * > >> sometimes also seems to also be placing requirements on the "network
> * > >> layer" that supplies component links for the CL, and even on the
> * > >> performance of a vendor implementation(DR#7)!
> * > > 
> * > > It seems the objection is to the phrase "business problems".  If we
> * > > change the first sentence in section 1, would that solve it?
> * > > 
> * > >  OLD
> * > > 
> * > >    The purpose of this document is to describe why network operators
> * > >    require certain functions in order to solve certain business
> * > >    problems (Section 2).
> * > > 
> * > >  NEW
> * > > 
> * > >    The purpose of this document is to describe why network operators
> * > >    require certain functions and to clearly enumerate a set of
> * > >    requirements related to the use of MPLS based Composite Links
> * > >    (see Section 2).
> * >  
> * >  
> * > Well, I see a couple of phrases that confuse me in the context of the
> * > document.  Certainly the phrase "... to solve certain business problems
> * > (Section 2)" caused me to be a bit concerned that an IETF document was
> * > going to discuss business issues; it also led me to believe that section
> * > 2 was going to discuss these problems. As I said, there are no
> * > "business problems" covered in section 2, so aligning the Intro with the
> * > section contents is a fine improvement.
> *  
> * I didn't add the "business problems" text but I think the intent was
> * to say that the requirements reflect serious provider needs.
> *  
> * > The other phrase that confuses me is "... describe why network operators
> * > require certain functions ... (See Section 2)".  Section 2 is titled
> * > Assumptions and only references network operators to say that "...in
> * > general network operators do not rely on the DSCP...". So I think your
> * > revised text isn't really aligned with the section.
> *  
> * OK.
> *  
> * > Finally, you now say the draft documents requirements on the "use of
> * > MPLS based Composite Links".  So this reads like the document places
> * > requirements on the user, i.e., the operator.  Do you perhaps mean "on
> * > the protocols and mechanisms used to provide MPLS based Composite Links"?
> *  
> * Yes.
> *  
> * > Taken all together, perhaps the following better matches the/your intent.
> * >     The purpose of this document is to clearly enumerate a set of
> * >     requirements related to the protocols and mechanisms that
> * >     provide MPLS based Composite Links. MPLS based Composite Links
> * >     are defined in [I-D.ietf-rtgwg-cl-use-cases].  Section 2 describes
> * >     MPLS based Composite Links assumptions.
> *  
> * Yes.  This is a much better sentence than the existing first
> * sentence.  I think you are also right that the rest of the paragraph
> * lacks clarity as well.
> *  
> * > > Then in section 2, change "The services supported include ..." to
> * > > "Services which may be supported over MPLS based Composite Links
> * > > include ...".
> * >  
> * > That is helpful.  Do you perhaps mean:
> * > "Any MPLS-based service may be supported over MPLS based Composite Links
> * > including, for example, ..."
> *  
> * It is at least IP and MPLS services.  IP services can also be carried
> * with certain limitations.  The document discusses measuring and
> * compensating IP and LDP traffic.  Both IP and LDP are not traffic
> * engineered (and the requirement to NOT add TE to LDP is also stated).
> *  
> * > >> I think clearly stating the document's objective/purpose and where the
> * > >> stated requirements are to be applied/satisfied, is one of major issues
> * > >> that should be corrected.
> * > >>  
> * > >> Once this is done, it may turn out that there are derivative
> * > >> comments/changes that are needed in the body of the draft.  For example
> * > >> clarifying what it means to "meet the NPO" and who needs to "meet" it.
> * > > 
> * > > The reason the term NPO is used it because of objections to using the
> * > > term SLA.  A Service Level Agreement (SLA) is interpreted as
> * > > contractual.  A Network Performance Objection (NPO) may simply be an
> * > > interal goal that an organiztion would like to meet.  The NPO may or
> * > > may not be based on Y.1540 and Y.1541.  The term NPO is defined in
> * > > Y.1541 and the use of the term in this docuement is consistent with
> * > > that definition.
> * > > 
> * > > The use cases document has a long paragraph defining SLA, SLS, and
> * > > NPO.  Those definitions are in an appendix intended to be illustrative
> * > > and entitled "More Details on Existing Network Operator Practices and
> * > > Protocol Usage".
> * >  
> * > The use case document also says:
> * >    In this
> * >    document we use the term Network Performance Objective (NPO) as
> * >    defined in section 5 of [ITU-T.Y.1541] since the SLA and SLS measures
> * >    have network operator and service specific implications.
> *  
> * Yes.  If we continued to reference the Use Case document, we would
> * have to fix that.  If we put the definition directly in this document,
> * then we could not take that definition verbatim given your
> * interpretation that using the term NPO requires meeting all
> * requirements for NPO defined in Y.1540 and Y.1541, which was
> * definitely not the intent.
> *  
> * See comments below where s/NPO/performance objectives/ is mentioned
> * (search for "3a.").
> *  
> * > > The definition here is simply:
> * > > 
> * > >    Network Performance Objective (NPO):  Numerical values for
> * > >        performance measures, principally availability, latency, and
> * > >        delay variation.  See [I-D.ietf-rtgwg-cl-use-cases] for more
> * > >        details.
> * > > 
> * > > I can expand on this and say "NPO may or may not be based on ITU
> * > > specifications."  I prefer not to get too wordy when all we are trying
> * > > to say "like SLA or SLS (in diffserv), but NPO is really a better
> * > > term".
> * > > 
> * >  
> * > Why not just say:
> * >    Network Performance Objective (NPO): as defined by
> * >    [I-D.ietf-rtgwg-cl-use-cases].
> *  
> * We would have to change the definition in Use Cases.  We want to use
> * the term "performance objective" term rather than SLA as used in
> * Diffserv, and one was conveniently defined in Y.1541, but make using
> * any other part of Y.1540 or Y.1541 completely oprional.  It would be
> * best to drop all mention of Y.1540 or Y.1541 and avoid any further
> * confusion.
> *  
> * See comments below where s/NPO/performance objectives/ is mentioned
> * (search for "3a.").
> *  
> * > > As to what it means by "meet the NPO", it means "falls within the
> * > > numerical values for measures, principally availability, latency, and
> * > > delay variation".  I'm not sure why that is not obvious.
> * >  
> * > Try using this expansion in the identified requirements and see if you
> * > think it makes sense in all cases.  Also, recall that Table 1 in section
> * > 5 of [ITU-T.Y.1541] provides absolute time values based on traffic "QoS
> * > class" and that a number of values are also specified as "U", which is
> * > stated as meaning U", performance with respect to
> * > that parameter may, at times, be arbitrarily poor.
> *  
> * Again, where s/NPO/performance objectives/ and performance objectives
> * need not be based on Y.1540 or Y.1541.  The service provider (SP)
> * defines their performance objectives and "meeting the performance
> * objectives" means falling within the parameters of the SP performance
> * objectives regardless of how they came about defining them.
> *  
> * > For example:
> * >  
> * >    FR#1  The solution SHALL provide a means to summarize some routing
> * >          advertisements regarding the characteristics of a composite
> * >          link such that the routing protocol converges within the
> * >          timeframe needed to [meet the network performance objective]
> * >          fall within the numerical values for measures, principally
> * >          availability, latency, and delay variation
> * >  
> * >  
> * >    FR#2  The solution SHALL ensure that all possible restoration
> * >          operations happen within the timeframe needed to [meet the NPO]
> * >          fall within the numerical values for measures, principally
> * >          availability, latency, and delay variation
> * >  
> * > I really think more is needed to make these requirements meaningful in
> * > the design of any solution.  For example, I could see something along
> * > the lines of the following as being meaningful:
> * >  
> * > (Note this text is for discussion not inclusion in the draft.)
> * >    FR#2  The solution SHALL enable restoration operations to occur
> * >          within the timeframe need to provide an Upper bound on the
> * >          packet loss probability (NPO IPLR) of 1x10^-3 in a
> * >          network with a diameter of XYZ.
> * >  
> * > Do you think I'm wrong?
> *  
> * Yes.  Absolutely.  There is no intent to mandate use of Y.1540 or
> * Y.1541.  Apparently that was not clear and we need to make it clear.
> *  
> * Re FR#1.  SP have existing convergence performance objectives.  Those
> * objectives are *never* based on Y.1541 IPLR.  The point of the first
> * sentence of FR#1 is to not make changes to the IGP that will make
> * convergence worse.  The point of the second sentence is that we want
> * to summarize the CL, but also provider more detailed information about
> * the component links, or preferably groups of compont links.
> *  
> * For example:  Link has 2TB, delay is <= 12 ms, etc.  One group of
> * component links has 1 TB, delay <= 12 ms.  Another group of component
> * links has 1 TB, delay <= 10 msec.  This could be the case with two
> * slightly different fiber paths.
> *  
> * > On an aside, how do FR#1 and DR#6 really differ?
> *  
> * FR#1 expose info on component but don't affect convergence
> * FR#2 don't make restoration time any worse
> * FR#3 for the "simple link" case (not across the network) don't require
> *      additional protocol
> * FR#4 minimally disruptive migration if new protocols are defined
> * FR#5 no (or minimal) oscillation - stability - bounded frequency of
> *      path change
> * FR#6 don't break network management
> *  
> * So these are base functional requirements.  FR#1 is key.  Expose
> * information about component links or groups of component links.
> *  
> * The derived requirements are supposed to provide more practical detail
> * on how there requirements can be met.
> *  
> * > DR#6 seems to be the
> * > better formulated to me.
> *  
> * DR#6 could refer back to FR#1.  DR#6 is where we imply that we need to
> * retain a mechanism to at least advertise groups of component links.
> * For example, this is so we don't have 80 TLVs with one component link
> * each if a 8TB link is made up of 80 x 100 Gb/s component links.
> *  
> * > > As to the "who", it is the provider but the protocols and
> * > > implementation have to provide the means for the provider to do so.
> * >  
> * > I think this point is now understood, assuming there's no substantive
> * > disagreement on the revised intro.
> *  
> * Your change to the first sentence is accepted.
> *  
> * > > As to the scope of the NPO: if the NPO is defined over a link, the
> * > > link needs to meet it; if it is defined as edge-to-edge within an
> * > > internal infrastructure, it must be met edge-to-edge; if it is defined
> * > > for a specific customer end-to-end, then it must be met end-to-end
> * > > (and may correlate strongly with a specific SLA).
> * >  
> * > Humm, the mechanisms needed to satisfy timing requirements between
> * > adjacent nodes can be radically different (simpler) than between network
> * > paths.  I'm not sure NPO-based requirements can be meaningful without
> * > some design targets/constraints.  At least ITU-T.Y.1541 provides some
> * > absolute time values.
> *  
> * This is why we need s/NPO/performance objective/.  We used NPO just to
> * avoid using SLA which implies a contract.
> *  
> * > > At what point does clarifying what is meant by an NPO and what it
> * > > means to meet an NPO just overly wordy and stating the obvious?
> * >  
> * > I think this is covered above.
> *  
> * Do you then agree that we should remove all reference to NPO and to
> * ITU-T.Y.1540 and ITU-T.Y.1541?
> *  
> * If so, then we have settled the definitions issue.  Again see text
> * regarding s/NPO/performance objective/ below (search for "3a.").
> *  
> * > >> - The scope of the composite links referenced in this document is not
> * > >> clear.  The document seems to be using G.800 terminology and concepts,
> * > >> yet the G.800 related summary is relegated to an Appendix.  If this CL
> * > >> is being driven/constrained by the G.800 CL, then this relationship
> * > >> should be made explicit and in the (early) body of the document, or by
> * > >> reference to another document that defines CL as such.  (In other words
> * > >> Appendix A should early in the document, perhaps in section 1.)
> * > > 
> * > > We give a definition for composite link and component link.  ITU gives
> * > > a definition for the same two terms.  We have give a summary of a few
> * > > ITU-T G.800 definitions in Appendix A.  I would be perfectly happy
> * > > getting rid of Appendix A and simply being more clear in our
> * > > definitions section.
> * > > 
> * > >  OLD
> * > > 
> * > >    ITU-T G.800 Based Composite and Component Link Definitions:
> * > >        Section 6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite and
> * > >        component links as summarized in Appendix A.  The following
> * > >        definitions for composite and component links are derived from
> * > >        and intended to be consistent with the cited ITU-T G.800
> * > >        terminology.
> * > > 
> * > >  NEW
> * > > 
> * > >    ITU-T G.800 Based Composite and Component Link Definitions: Section
> * > >        6.9.2 of ITU-T-G.800 [ITU-T.G.800] defines composite and
> * > >        component links.  The following definitions for composite and
> * > >        component links are derived from and intended to be consistent
> * > >        with the cited ITU-T G.800 terminology, but differ slightly.
> * > > 
> * > > We can then point out that we are specifically excluding inverse
> * > > multiplexing from our definition of Composite Link and that we are not
> * > > using the ITU definition.
> * > > 
> * > > I would like to see us use the definition of multipath in
> * > > draft-ietf-mpls-multipath-use but this work started out using the term
> * > > composite link so we may be stuck with it.  Some key definitions in
> * > > draft-ietf-mpls-multipath-use are:
> * > > 
> * > >    Multipath
> * > >        The term multipath includes all techniques in which
> * > > 
> * > >        1.  Traffic can take more than one path from one node to a
> * > >            destination.
> * > > 
> * > >        2.  Individual packets take one path only.  Packets are not
> * > >            subdivided and reassembled at the receiving end.
> * > > 
> * > >        3.  Packets are not resequenced at the receiving end.
> * > > 
> * > >        4.  The paths may be:
> * > > 
> * > >            a.  parallel links between two nodes, or
> * > > 
> * > >            b.  may be specific paths across a network to a destination
> * > >                node, or
> * > > 
> * > >            c.  may be links or paths to an intermediate node used to
> * > >                reach a common destination.
> * > > 
> * > >    Composite Link
> * > >        The term Composite Link had been a registered trademark of Avici
> * > >        Systems, but was abandoned in 2007.  The term composite link is
> * > >        now defined by the ITU in [ITU-T.G.800].  The ITU definition
> * > >        includes multipath as defined here, plus inverse multiplexing
> * > >        which is explicitly excluded from the definition of multipath.
> * > > 
> * > >    Inverse Multiplexing
> * > >        Inverse multiplexing either transmits whole packets and
> * > >        resequences the packets at the receiving end or subdivides
> * > >        packets and reassembles the packets at the receiving end.
> * > >        Inverse multiplexing requires that all packets be handled by a
> * > >        common egress packet processing element and is therefore not
> * > >        useful for very high bandwidth applications.
> * > > 
> * > > We can also add the following:
> * > > 
> * > >    Classic Multipath
> * > >        Classic multipath refers to the most common current practice in
> * > >        implementation and deployment of multipath where multipath
> * > >        consist only of homogeneous links and in cases such as Ethernet
> * > >        Link Aggregation are entirely transparent to upper layer
> * > >        control planes.
> * > > 
> * > > These IMHO are more clear definitions.
> * > > 
> * > > If you prefer these, I'd be glad to put them in and then indicate that
> * > > inverse multiplexing is the only form of composite link that is out of
> * > > scope for this document.
> * >  
> * > Given that you stated that the use case document is the one that
> * > provides the definition/intro to IETF CLs, I'd just refer to that
> * > document and drop all references to G.800, including the Appendix, in
> * > this document.  Also use the following definitions:
> * >  
> * >        Composite Link:  As defined in [I-D.ietf-rtgwg-cl-use-cases].
> * >  
> * >        Component Link:  As defined in [I-D.ietf-rtgwg-cl-use-cases].
> * >  
> * > BTW I like your ideas WRT multipath, but I suspect that that ship has
> * > sailed.
> *  
> * Please explain your "ship has sailed" comment.
> *  
> * I am fine with the substitution above except that we now have a
> * normative refernce to the Use Cases document which I would rather
> * avoid.
> *  
> * I was under the (apparently very mistaken) impression that the
> * existing definition was sufficient with the reference to Use Cases
> * being intended to provide additional information.
> *  
> * If the definitions in draft-ietf-rtgwg-cl-requirement of "Composite
> * Link", "Component Link", and "Performance Objective" are not
> * sufficient, then we need to make them sufficient.  If that means that
> * we need to go back to the WG step, then we will have to do so.
> *  
> * > >> - On a more detailed level, it seems that this document really just has
> * > >> MPLS (composite link) over MPLS (component links) use cases/solutions in
> * > >> mind.  (Yes there is one line about other technology examples, but that
> * > >> is in just one spot and the MPLS is mentioned throughout.)  If this is
> * > >> correct, then this should be made explicit in the early part of the
> * > >> document.  If not, then the MPLS related assumptions/limitations need to
> * > >> removed.
> * > > 
> * > > That is not true.  The definition of component link should make this
> * > > clear.
> * > > 
> * > >        Component Link:  A point-to-point physical link (including one or
> * > >            more link layer) or a logical link that preserves ordering in
> * > >            the steady state.
> * > > 
> * > > If also gives examples which include Ethernet, PPP, SONET or OTN over
> * > > a physical link.  It also states "A set of link layers supported over
> * > > pseudowire is a logical link that appears to the client to be a
> * > > physical link."
> * > > 
> * > > In all cases it is MPLS over something, but not necissarily MPLS over
> * > > MPLS.  I don't see a basis for the statement "it seems that this
> * > > document really just has MPLS (composite link) over MPLS (component
> * > > links) use cases/solutions in mind".
> * > > 
> * > > Could you please point out any "MPLS related assumptions/limitations"
> * > > that are related to MPLS being used as a server layer rather than
> * > > using a link layer as a server layer.  I don't see any.  The existance
> * > > of section 4.2.  Component Links Provided by Lower Layer Networks does
> * > > not state, nor does it in any way imply that only MPLS lower layers
> * > > are being considered.  The one descriptive paragraph even gives
> * > > circuit switched or MPLS-TP as examples, further indicating that MPLS
> * > > over MPLS is not the sole focus.  This section gives four requirements
> * > > related to passing information from the sever layer up to the client
> * > > layer that apply to the case of a server layer that is more complex
> * > > than a simple physical link plus link layer.
> * >  
> * > Some of my comment was certainly biased by other drafts, including the
> * > framework draft which more clearly shows this.  In terms of this
> * > document, I don't see how you expect arbitrary technologies used to
> * > provide component links to meet requirements FR#7 and FR#8.  Do you
> * > really expect them to be met by non-MPLS (or perhaps GMPLS) controlled
> * > technologies?   Did you think about making these objectives, or limited
> * > to MPLS/GMPLS controlled networks, rather than a requirement?
> *  
> * There is a lot of discussion in the framework draft as it now stands
> * of the problematic cases.  Most of the problematic cases are unique to
> * MPLS over MPLS and do not occur in the MPLS over simple link case.
> *  
> * > Also, I read FR#11 as the labeling of the component not composite link,
> * > otherwise why mention "labeled traffic flow".
> *  
> * We need to reword FR#11.  The point is to get identified flows or
> * groups of flows assigned to components.  The existing hashing
> * techniques move around groups of flows where a group is those that
> * hash to the same value.  To make the dynamic techniques work,
> * measurement of the traffic contribution of a group avoids trial and
> * error movement of groups of flows in response to traffic imbalance.
> *  
> *  OLD:
> *  
> *    FR#11  The solution SHALL measure traffic on a labeled traffic flow
> *           and dynamically select the component link on which to place
> *           this flow in order to balance the load so that no component
> *           link in the composite link between a pair of nodes is
> *           overloaded.
> *  
> *  NEW:
> *  
> *    FR#11  The solution SHALL measure traffic flows or groups of
> *           traffic flows
> *           and dynamically select the component link on which to place
> *           this traffic in order to balance the load so that no component
> *           link in the composite link between a pair of nodes is
> *           overloaded.
> *  
> * A group of flows can be identified by label stack, by IP headers, or
> * cheating a bit and looking below the label stack at the IP headers.
> * PWE3 CW acknowledges that that form of cheating is widespread and
> * rectifies it for PW.  Entropy Label makes it no longer necessary to do
> * that form of cheating (once sufficiently deployed).
> *  
> * > >> Minor Issues:
> * > >>  
> * > >> - Abstract
> * > >>  
> * > >> This whole section seems to be some early text that hasn't been
> * > >> revisited as the document progressed.  I'd suggest throwing it out and
> * > >> just picking up (with slight modification) the Abstract from the
> * > >> cl-framework document which seems pretty good.
> * > > 
> * > > Alia asked for an edit to the existing abstract.  With the change that
> * > > Alia requested the abstract is now:
> * > > 
> * > >    There is often a need to provide large aggregates of bandwidth that
> * > >    are best provided using parallel links between routers or carrying
> * > >    traffic over multiple MPLS LSP.  In core networks there is often no
> * > >    alternative since the aggregate capacities of core networks today far
> * > >    exceed the capacity of a single physical link or single packet
> * > >    processing element.
> * > > 
> * > >    The presence of parallel links, with each link potentially comprised
> * > >    of multiple layers has resulted in additional requirements.  Certain
> * > >    services may benefit from being restricted to a subset of the
> * > >    component links or a specific component link, where component link
> * > >    characteristics, such as latency, differ.  Certain services require
> * > >    that an LSP be treated as atomic and avoid reordering.  Other
> * > >    services will continue to require only that reordering not occur
> * > >    within a microflow as is current practice.
> * > > 
> * > > The point of the second paragraph is that some LSPs have specific
> * > > requirements (such as bounded delay) and some (most) do not.
> * > > Flexibility in accommodating both is the requirement.  Composite Links
> * > > supporting non-homogenous Component Links is the basis for the
> * > > solution.  That is why the framework mentions "group of homogenous or
> * > > non-homogenous links".
> * > > 
> * > > The fact that Alia just commented on and we just changed the abstract
> * > > is indication that it has been read and is not just left over text
> * > > from ancient times that no longer applies.  Could you please be more
> * > > specific about your objections to this text.
> * >  
> * > It's also an indication that it needed some work.  Given that you say
> * > the use-case is the source for IETF CL definition and that the abstract
> * > provides so much more information than the Intro, I'd suggest replacing
> * > with something really simple, perhaps along the lines of:
> * >  
> * >    This document provides a set of requirements for MPLS composite
> * >    links.
> * >  
> * >    Composite link is a formalization of multipath techniques currently
> * >    in use in IP and MPLS networks and a set of extensions to existing
> * >    multipath techniques.
> *  
> * Agree that Alia's last minute comment indicates the abstract could be
> * better.
> *  
> * I like your substitution and would not mind moving the existing text
> * in the abstact to the Intro.
> *  
> * > >> - 1. Introduction
> * > >>  
> * > >> So where does one go to get an intro/definition of the IETF work on
> * > >> composite links?  Is it G.800, the framework draft, the use cases draft,
> * > >> this draft? If the last, then this section needs a serious revision.  If
> * > >> not, then this document should reference that definition and address the
> * > >> major issues identified above.
> * > > 
> * > > I'd say "Use Cases".  I'm in favor of de-emphasizing G.800.
> * >  
> * > (Covered above).
> *  
> * Given the two choice you gave, Use Cases was the better choice, but as
> * stated above an even better way to handle this is to put a correct set
> * of defintions in draft-ietf-rtgwg-cl-requirement (this document) and
> * not refer to G.800 at all and refer to Use Cases only for further
> * information on current practices (the intent).
> *  
> * > > Please be clear.  What major issues?
> * >  
> * > "Major Issues" as identified in the RtgDir review template and above.
> * >  
> * > > Do you means the definitions of the well known terms CL and NPO?  
> * >  
> * > Yes.
> *  
> * My preference is to put definitive definitions into
> * draft-ietf-rtgwg-cl-requirement (this document).  I proposed
> * definitions on this thread but I think this change would be enough to
> * require another round trip through a WGLC and so on back to this
> * point.  If we are in agreement that putting definitive definitions
> * into this document is the best option, then I recommend we go with the
> * ones cited above (prior to your "ship sailed" comment).
> *  
> * > > If
> * > > so, how would you like them changed (see above).
> * > As covered above, reference use case document and ensure that the terms
> * > are appropriately defined there.
> *  
> * I don't think that is the best option but if others feel this is the
> * best way to go, then I'll go along.
> *  
> * The problem that I anticipate is that the Use Cases document, being
> * one which describes current practices of the two dominant router
> * implementations and may other but not all, we could end up with push
> * back on that document.  Therefore I would prefer it to be ancilliary
> * information only and not listed as normative anywhere.
> *  
> * > > Do you mean your belief that this document only applies to MPLS over
> * > > MPLS?  In that case, you seem to be objecting to the intent of the
> * > > document even when your interpretation of the intent is contradicted
> * > > by many statements made in the document.  If so, how can we make it
> * > > even more clear that a component link can also be a physical link,
> * > > circuit switched, pseudowire, besides mentioning that in the
> * > > definition of compnent link and in numerous other places.
> * >  
> * > Covered above.
> *  
> * OK.
> *  
> * > >> - 2 Assumptions
> * > >> I suspect this section will need to be updated to address the document
> * > >> scope comments above.
> * > > 
> * > > AFAIK there is no change to the document scope.
> * >  
> * > Given CL is defined in use cases, it seems more appropriate to me that
> * > Assumptions be presented as part of the definition.  In other words,
> * > assuming the definition of CL is in the use case document, I think this
> * > it makes most sense for text to be in/moved to the use cases document.
> *  
> * At this point you have suggested a replacement abstract (above).  I
> * have agreed to that and suggested that the existing abstract be moved
> * into the introduction.  This is a lot of change, but I'm OK so far.
> *  
> * You are now suggesting here that the existing "Assumptions" section be
> * moved to the Use Cases document.  I don't feel that is necessary, but
> * I'm OK with it.  I would prefer that the "Assumptions" section stay
> * where it is.
> *  
> * > >> - Definition of NPO.
> * > >> It seems to me that the authoritative definition of NPO is Y.1541 with
> * > >> Y.1540 providing additional detail.  I think these documents would be a
> * > >> much more reasonable reference for NPO than the CL use case draft as is
> * > >> currently used.
> * > > 
> * > > Would it be OK to add this text.
> * > > 
> * > >   Network operators have a contractual Service Level Agreement (SLA)
> * > >   with customers for services that are comprised of numerical values
> * > >   for performance measures, principally availability, latency, delay
> * > >   variation.  Additionally, network operators may have Service Level
> * > >   Specification (SLS) that is for internal use by the operator.  In
> * > >   this document we use the term Network Performance Objective (NPO)
> * > >   which may span multiple networks and may be looser than network
> * > >   operator SLA or SLS objectives.  The use of the term NPO in this
> * > >   document is not entirely consistent with its definition in Y.1540
> * > >   and Y.1541 in that it is not restricted to UNI to UNI.  There is no
> * > >   intent to imply that a provider must base their SLA, SLS, or NPO on
> * > >   Y.1540 and Y.1541.
> * > > 
> * > > If we stay consistent with Y.1540 and Y.1541 then I think we have to
> * > > s/NPO/SLS or NPO/ and we did want a single term.
> * >  
> * > Hereto, given the position that the definition of CL is in the use case
> * > document, I think the original reference makes sense. (Although I'd
> * > probably just say "Network Performance Objective (NPO): as defined in
> * > [I-D.ietf-rtgwg-cl-use-cases].")
> *  
> * We can push this document through by moving these definitions out and
> * returning to the problem later, but I would rather address the problem
> * in this document.
> *  
> * > >> - Many of the requirements refer to "meeting" or "not violating" NPO
> * > >> without given any specifics as to which NPO parameter/parameters is/are
> * > >> relevant. For example Class 5 QoS has unbounded upper bounds on Network
> * > >> performance parameter.  Also NPO is defined UNI to UNI (end to end). I
> * > >> suggest adding a level of detail identifying the parameters and how the
> * > >> fixed values are to be applied relative to "meeting" or "not violating" NPO.
> * > > 
> * > > Please see comments above regarding use of the term NPO.
> * >  
> * > dito on the response ;-)
> * >  
> * > > 
> * > > We wanted one term.  Diffserv uses SLA, even though there may be no
> * > > agreement with an outside party.  If we can't use SLS because it is
> * > > specific to one provider only, can't use NPO because in Y.154[01] it
> * > > applies only to UNI to UNI, then we need another term.
> * > > 
> * > > This reminds me of the host and router vs ES and IS arguments back in
> * > > the 1990s and the issue of using the term subnet to mean what it meant
> * > > in IETF since the early 1980s, ignoring what ITU had defined it to
> * > > mean.  Eventually we dumped the use of Logical IP Subnetwork (LIS) as
> * > > we had to do in RFC1577 and could just say "subnet" again.
> * > > 
> * > > Perhaps the best course of action is s/NPO/performance objective/ and
> * > > then define performance objective without any reference to Y.1540 and
> * > > Y.1541, but perhaps referring to SLA as defined in RFC2475.
> * >  
> * > Independent of the terminology conundrum, you still have to figure out
> * > how to provide requirements that are meaningful in designing new
> * > protocols/mechanisms.  I know this is a very difficult point, but
> * > without do so, it's hard to see how any solution can be evaluated
> * > against a requirement.
> *  
> * This document is not trying to mandate Y.1540 and Y.1541 nor is it
> * trying to define the set of performance objectives that a provider
> * must adhere to.
> *  
> * > BTW I really like the (relative terms) approach taken by DR#6 and DR#7
> * > as they avoid any of the complications inherent in a specific-value
> * > performance parameter approach (such as seen in Table 1 of Y.1541)
> *  
> * That is why we should get rid of all reference to Y.1540 and Y.1541.
> * I never wanted those references in there in the first place just to
> * pick up a definition of a term and try to otherwise distance ourselves
> * from it.  Your interpretation that we are advocating using Y.1541
> * parameters, is so far a unique interpretation, but it is better to
> * avoid any potential confusion and completely eliminate references to
> * Y.1540 and Y.1541.
> *  
> * > >> - It might be useful to somehow separate data plane requirements from
> * > >> control plane requirements (Management/OAM requirements are already
> * > >> separated.)
> * > > 
> * > > Too many of the requirements involve both the control plane and the
> * > > data plane.  Where we are talking solely about physical links and
> * > > local decisions can be made, then no control plane signaling is
> * > > required.  For some requirements, extensions to signaling would be
> * > > required for the MPLS over MPLS case.
> * > > 
> * > > A good example is path symetry.  In the framework, it is pointed out
> * > > that the most general case of path symetry on a per client LSP basis
> * > > (FR#22 and FR#23) is close to infeasible where there are multiple
> * > > client/server layer relationships.  FR#22 and FR#23 works for classic
> * > > link bundling where each component link takes a single component
> * > > bidirectional path, but would require a lot of extension to work for
> * > > MPLS over MPLS over MPLS ad nauseum.
> * > > 
> * > > Whether the solution is done in the data plane or control plane or
> * > > using a little contribution from each is a matter for the framework
> * > > document.  The requirement is simply that an objective be met.
> * > > 
> * >  
> * > okay. Accepted & point closed.
> * >  
> * > >> Nits:
> * > >>  
> * > >> I have a bunch of more nit level comments, but I suspect much will
> * > >> change if/when the above are addressed, so I'm going to defer the
> * > >> majority of them until the above issues are resolved.
> * > > 
> * > > I would like one more response from you before making any changes to
> * > > the document.
> * > > 
> * >  
> * > I agree, it makes sense to resolve the higher level comments before
> * > jumping on changes (and then on to nits.)
> *  
> * Obviously we still need at least one more iteration (maybe more).
> *  
> * > >> - It looks like you are referencing an old version of g.800.  The
> * > >> current is G.800 (02/2012). Also the G.800 "Cases" are identified in the
> * > >> draft not in G.800.
> * > > 
> * > > I'm in favor of removing reference to G.800 except as a point of
> * > > clarification to say that they also define Composite Link and do so
> * > > somewhat differently.
> * > > 
> * >  
> * > I think I go a step further in my comments above and say drop it all
> * > together (and let it be covered in the CL-defining use case draft).
> *  
> * We need to resolve that before going forward.  In either case we would
> * be dropping references to G.800, Y.1540, and Y.1541.  We then have two
> * options:
> *  
> *   1.  Reference use cases and fix up the definitions later.
> *  
> *   2.  Fix up the definitions here.
> *  
> * I think that in either case the changes that you are calling for are
> * enough to require a quick pass back through WGLC, AD, IESG, IETF LC,
> * and back to this point in the process.
> *  
> * > >> Lou
> * > > 
> * > > There is a lot of discussion above, mostly around definitions and
> * > > assumptions about document scope.
> * > > 
> * > > In summary:
> * > > 
> * > >   1.  Remove the phrase "business problems" as described above.
> * > >       Please let me know if the OLD/NEW change on this is sufficient.
> * > > 
> * > >   2.  In section 2, change "The services supported include ..." to
> * > >       "Services which may be supported over MPLS based Composite Links
> * > >       include ...".
> * > > 
> * > >   3.  Resovle issue of defintion of NPO.
> * > > 
> * > >       3a.  My preference is s/NPO/performance objective/.  Remove
> * > >            definition of NPO.  Replace with:
> * > > 
> * > >       Performance Objective: Numerical values for performance
> * > >          measures, principally availability, latency, and delay
> * > >          variation.  Performance objectives may be related to Service
> * > >          Level Agreements (SLA) as defined in RFC2475 or may be
> * > >          strictly internal.  Performance objectives may span links,
> * > >          edge-to-edge, or end-to-end.  Performance objectives may span
> * > >          one provider or may span multiple providers.
> * > > 
> * > >       3b.  Alternate is use the paragraph clarifying that we don't
> * > >       	   quite mean NPO as defined in Y.154[01].  I'd rather just
> * > >       	   get rid of the term NPO.
> * > > 
> * > >   4.  Replace definitions for "Composite Link" and "Component Link"
> * > >       with those from draft-ietf-mpls-multipath-use and include
> * > >       "Multipath" and "Inverse Multiplexing", plus "Classic Multipath"
> * > >       as suggested above.
> * > > 
> * > >   5.  State that "Composite Link is also defined in ITU G.800 but the
> * > >       usage here specifically excludes inverse multiplexing and is
> * > >       more similar to multipath but with extensions to support groups
> * > >       of homogeneous or non-homogeneous links.  Remove all other
> * > >       references to G.800.
> * > > 
> * > >   6.  Resolve wording on abstract.  Please be more specific about your
> * > >       objections to this text.
> * > > 
> * > >   7.  There is no change to document scope.  Please indicate why you
> * > >       think the document scope is not stated correctly.
> * > > 
> * > >   8.  Don't try to separate data plane vs control plane.  In too many
> * > >       cases that is more applicable to solution space than it is to
> * > >       requirements.
> * > > 
> * > > Please at least comment on only the summary before I make changes.
> * > > 
> * >  
> * > I think I responded to all the detail points.  Let me know if I missed
> * > anything.
> * >  
> * > Thanks again for the detailed response.
> * > Lou
> * >  
> * > > Curtis
> *  
> * I think you did respond to all points.  I think I understand all of
> * your objections and your suggestions.  The major point remaining may
> * be whether we put definitions in Use Cases or put sufficient
> * definition in this document as originally intended.
> *  
> * I'm not entirely sure that you will agree with all of my points as it
> * didn't seem clear to me in this response from you that you understood
> * that our intent was to neither encourage or discourage the use of
> * Y.1540 and Y.1541 but to just make use of a common term which turned
> * out to be a mistake on our part.  We should define "Performance
> * Objective" as above in "3a" (or similar if you have better text).
> *  
> * Curtis
> *  
> * ------- End of Forwarded Message