Re: [Detnet] data plane framework editorial pass

Ethan,

See
https://www.ietf.org/about/groups/iesg/statements/normative-informative-references/
for more information.

Cheers,
Andy

On Thu, Oct 3, 2019 at 2:28 PM Andrew G. Malis <agmalis@gmail.com> wrote:

> Ethan,
>
> Informative RFCs can have normative references. The rule is that if you
> need the reference to implement or understand the RFC, then it's a
> normative reference. If the reference just adds additional background
> information, then it's informative.
>
> Cheers,
> Andy
>
>
> On Thu, Oct 3, 2019 at 12:52 PM Grossman, Ethan A. <eagros@dolby.com>
> wrote:
>
>> Hi Balazs,
>>
>> Thanks for your responses. Given that this draft is Informational, should
>> references classified as Normative be moved to the Informative References
>> section, and the Normative References section removed?
>>
>> Thanks,
>>
>> Ethan.
>>
>>
>>
>> *From:* Balázs Varga A <balazs.a.varga@ericsson.com>
>> *Sent:* Thursday, October 3, 2019 6:59 AM
>> *To:* Grossman, Ethan A. <eagros@dolby.com>; detnet@ietf.org
>> *Subject:* RE: data plane framework editorial pass
>>
>>
>>
>> Hi Ethan,
>>
>>
>>
>> Many thanks for your detailed review and great suggestions. I have
>> accepted most of them “as is”. See reactions inline.
>>
>> I will update the draft on Github and create a pull request during the
>> next week to make changes visible.
>>
>> https://github.com/detnet-wg/data-plane-drafts
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_detnet-2Dwg_data-2Dplane-2Ddrafts&d=DwMFAw&c=lI8Zb6TzM3d1tX4iEu7bpg&r=ZcHC6wX_gDwPDcfMaFNZiQ&m=ooBa0YGoSfUxbAgLXk8YD9UZkK4Qeri0-G20uc_zLtc&s=Kjy_R6-zJrZe5ijdAhLrXpYSQB4T341hC7-WUGwVStk&e=>
>>
>>
>>
>> Any further comments from the group is highly welcome.
>>
>>
>>
>> Cheers
>>
>> Bala’zs
>>
>>
>>
>> *From:* detnet <detnet-bounces@ietf.org> *On Behalf Of *Grossman, Ethan
>> A.
>> *Sent:* Tuesday, October 1, 2019 4:38 AM
>> *To:* detnet@ietf.org
>> *Subject:* [Detnet] data plane framework editorial pass
>>
>>
>>
>> Hi All, (particularly Balazs as editor of the draft),
>>
>> I hate to be the guy who waits until WG LC to actually read the draft
>> carefully and then comes with a bunch of comments, but here I am; hopefully
>> I will get into gear sooner with the other data plane drafts, but as
>> document shepherd I have to read carefully, and I figured as long as I was
>> noticing things I would write them down.
>>
>> So as a result there are many comments below; thankfully they are mostly
>> grammatical, the sort that the IETF Editorial will surely make if we don’t
>> correct them, so I figure we should bite the bullet and do them now rather
>> than later. There are even a few actual “what does this mean” type of
>> questions. But on the whole the draft is in quite good shape and I plan to
>> start my “shepherd writeup” soon, given that I have gone over the draft
>> carefully now.
>>
>> The form I am using for my comments is like this:
>>
>> Issue type (e.g. wording, grammar, abbreviation, etc):
>>
>> “text in question”
>>
>> è “Ethan’s suggestion on how to fix it:”
>>
>> I hope this is helpful. If it isn’t, just let me know and for the
>> remaining drafts I’ll be less picky (or whatever your suggestion to me is).
>> And of course I may have gotten the wrong end of the stick on some of the
>> items, so please feel free to push back if I got the meaning wrong.
>>
>> Ethan (as data plane framework draft document shepherd).
>>
>> ---------------------------------------
>>
>> Wording:
>>
>> “Abstract
>>
>> This document provides an overall framework for the Deterministic
>> Networking data plane.  It covers concepts and considerations that are
>> generally common to any Deterministic Networking data plane specification.”
>>
>>    - Say DetNet instead of the more generic Deterministic Networking
>>    since it is specific to our architecture, not to all of deterministic
>>    networking in the world.
>>
>> *>Balazs> OK.*
>>
>> Wording:
>>
>> “1.  Introduction
>>
>>    Deterministic Networking (DetNet)”
>>
>>    - Similarly, say “DetNet (Deterministic Networking)” .
>>
>> *>Balazs> OK.*
>>
>> Wording:
>>
>> “The forwarding sub-layer is used to provide congestion protection (low
>> loss, assured latency, and limited out-of-order delivery) and leverages
>> Traffic Engineering mechanisms.”
>>
>>    - Improve clarity:
>>
>> The forwarding sub-layer leverages Traffic Engineering mechanisms to
>> provide congestion protection (low loss, assured latency, and limited
>> out-of-order delivery).
>>
>> *>Balazs> OK in principle. Slightly modified your proposal to catch
>> better the sense “… leverages Traffic Engineering mechanisms and provides
>> …”*
>>
>> Grammar:
>>
>> “service sub-layer”
>>
>>    - Service is a proper name, s/b capitalized.
>>
>> *>Balazs> Hmmm. In architecture draft we have used “service sub-layer”
>> and “forwarding sub-layer”. I would keep it.*
>>
>> Wording:
>>
>> “It also describes the forwarding sub-layer that is used to   eliminate
>> (or reduce) contention loss and provide bounded latency for   DetNet flows.”
>>
>>    - Redundant with above paragraph. s/b just “…the Forwarding
>>    sub-layer”. “assured latency” vs “bounded latency”...
>>
>> *>Balazs> OK. I will collapse the two sentences.*
>>
>> Meaning:
>>
>> “ Different traffic types, or application flows, can be mapped on top
>> of DetNet.”
>>
>>    - Are these two separate things? I gather they are? Maybe “Different
>>    traffic types and application flows can be mapped on top  of DetNet”. Below
>>    it then says
>>
>> “3.  DetNet Data Plane Overview
>>
>>    This document describes how application flows, or app-flows, are
>> carried over DetNet networks.”
>>
>> This sounds like they are synonyms? But an App Flow is not differentiated
>> by “traffic type” – we differentiate based on individual flow. So I think
>> we should remove “traffic types” above.
>>
>> *>Balazs> OK. Traffic types intended to refer to the encapsulation of the
>> App-flow (e.g., Ethernet or IP). I will fix this.*
>>
>> Wording:
>>
>>  “functions  related to the control plane”
>>
>>    - Are we standardizing on controller plane vs control plane, or are
>>    these different grammatical roles?
>>
>> *>Balazs> OK. I think it would be better simple to refer to the OAM
>> drafts.*
>>
>> Wording:
>>
>> “The forwarding sub-layer provides the quality underpin needed by the
>> DetNet flow...”
>>
>>    - Is underpin a real word? Sounds like we mean QoS?
>>
>> *>Balazs> OK. I will change to “… the QoS related functions needed …”*
>>
>> Abbreviation:
>>
>> “An example of   this is Packet Replication, Elimination, and Ordering
>> (PREOF)    functions see Section 4.3.”
>>
>>    - Not first use of PREOF, should not spell out here.
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “The method of instantiating each of the layers is specific to the
>> particular DetNet data plane method.  There may be more than one   approach
>> that is applicable to a given bearer network type.”
>>
>>    - I would combine these into a single sentence with comma, as: “The
>>    method of instantiating each of the layers is specific to the particular
>>    DetNet data plane method, and more than one approach may be applicable to a
>>    given bearer network type”.
>>
>> *>Balazs> OK. *
>>
>> Wording:
>>
>> “3.1.  Data Plane Characteristics
>>
>>    There are two major characteristics to the data plane:
>>
>>    1.  Data plane technology: *The DetNet data plane is provided by the
>> DetNet service and forwarding sub layers*.”
>>
>> à Please don’t tell me this again. Eliminate this sentence.
>>
>> *>Balazs> OK. :--)))*
>>
>> Wording:
>>
>> “Namely”
>>
>>    - Remove this word, already said “specifically” which is better
>>    anyway.
>>
>> *>Balazs> OK. *
>>
>> Abbreviations:
>>
>> “S-label and d-CW”
>>
>>    - Need to add these to Abbrev section.
>>
>> *>Balazs> OK. *
>>
>> Structure:
>>
>> “There are two major characteristics to the data plane:
>>
>>    1. Data plane technology:”
>>
>>
>>
>>    - I would say these should not be a 2-item numbered list, they should
>>    be a topic sentence listing both, followed by subsections, as is done for
>>    Encapsulation but not for Technology. Also one says Data Plane Technology,
>>    the other just Encapsulation – should be consistent – So we could have:
>>
>> 3.1.  Data Plane Characteristics
>>
>> There are two major characteristics to the data plane: the technology and
>> the encapsulation, as discussed below.
>>
>> 3.1.1 Technology
>>
>> …
>>
>> 3.1.2 Encapsulation
>>
>> …
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “The encapsulation of the DetNet flows allows them to be sent over a
>> data plane technology other than their native type.  Encapsulation is
>> essential if, for example, it is required to send Ethernet TSN stream    as
>> a DetNet Application over a data plane such as MPLS.”
>>
>>    - Consolidate: “The encapsulation of a DetNet flow allows it to be
>>    sent over a   data plane technology other than its native type. For
>>    example, an Ethernet TSN app flow can be sent as a DetNet app flow over
>>    MPLS...”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “meta-data”
>>
>>    - s/b metadata – not a hyphenated word.
>>
>> *>Balazs> OK. *
>>
>> Clarity:
>>
>> “The DetNet data plane can provide or carry meta-data:
>>
>>    1.  Flow-ID
>>
>>    2.  Sequence Number
>>
>> … Both of these metadata are required …”
>>
>>    - I initially read this as “they are both required” but then it turns
>>    out they are not. Rephrase:
>>
>> “The DetNet data plane supports a Flow-ID (for identification of the flow
>> or aggregate flow) and/or a Sequence Number (for PREOF) for each DetNet
>> flow. The DetNet Service sub-layer requires both; the DetNet forwarding
>> sub-layer requires only Flow-ID. Metadata can also be used for OAM
>> indications and instrumentation of DetNet data plane operation.”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “it is anticipated that more than one encapsulation   may be deployed for
>> example GRE”
>>
>>    - Need comma after “deployed,”
>>
>> *>Balazs> OK. *
>>
>> Wording:
>>
>> “[RFC7657] also provides useful background on the delivery differentiated
>> services (DiffServ)”
>>
>>    - I don’t think “the delivery” belongs there? Delete? (”…useful
>>    background on Differentiated Services (Diffserv)…” (following RFC7657
>>    format).
>>
>> *>Balazs> OK. I will rephrase. Reference to rfc7657 is important here, so
>> I will keep it.*
>>
>> Word meaning:
>>
>> “It is possible to include such information in a native IP   packet
>> explicitly, or implicitly.”
>>
>>    - I understand “explicitly” to mean there is additional metadata, vs
>>    “implicitly” to mean that it is inferred from the existing fields e.g.
>>    6-tuple. However above, it says “Some MPLS examples of implicit metadata
>>    include the sequence number    for use by the PREOF function” – to me that
>>    sounds odd. Can we please reconcile use of the words “explicit” vs
>>    “implicit” and/or explain the meaning of each. We also have
>>    explicit/implicit *paths* (for which I don’t see a clear definition
>>    from a simple google search):
>>
>> “MPLS provides the ability to forward traffic over implicit and
>> explicit paths to the point in the network where the next DetNet   service
>> sub-layer action needs to take place.”
>>
>> “Reservation and Allocation of resources:” (and similar lines below)
>>
>>    - Perhaps these paragraphs should be 3rd level headers, as opposed to
>>    one-sentence paragraphs with “:” at the end. This would also cause them to
>>    be included in the table of contents. Perhaps having a bullet list at the
>>    top would be useful? Then one could move to here the statement from below:
>>    “Several of these capabilities are expanded upon in more detail in sections
>>    below.”
>>
>> *>Balazs> Hmmm. I will reconsider these sentences. *
>>
>> Clarity:
>>
>> “This can eliminate packet contention and loss”
>>
>>    - I would explicitly say “This can eliminate packet contention and
>>    packet loss”.
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “This also can reduce jitter for the DetNet traffic”
>>
>>    - Eliminate superfluous “the”: “This also can reduce jitter for
>>    DetNet traffic”
>>
>> *>Balazs> OK. *
>>
>> Clarity:
>>
>> “DetNet flows are assumed to behave with respect to the reserved traffic
>> profile.  If other traffic shares the link resources, the use of (queuing,
>> policing, shaping) policies can be used to ensure that the allocation of
>> resources reserved for DetNet is met.   Queuing and shaping of DetNet
>> traffic could be required to ensure that DetNet traffic does not exceed its
>> reserved profile but this would impact the DetNet service characteristics.”
>>
>>    - I think this could be made clearer. Maybe:
>>
>> “DetNet flows are assumed to behave with respect to the reserved traffic
>> profile.  If other traffic shares the link resources, the use of policies
>> (such as queuing, policing, shaping) can be applied to the “other” traffic
>> to ensure that the allocation of resources reserved for DetNet is not
>> compromised.  Queuing and shaping of DetNet traffic to ensure that it does
>> not exceed its reserved profile (for example by dropping packets) may be
>> necessary to prevent fault conditions, but should not be used under normal
>> conditions as this could compromise the DetNet QoS.”
>>
>> *>Balazs> Hmmm. Agree some rewording is needed here. We intend to say
>> here two things: (1) resources allocated to a DetNet flow must be protected
>> from other traffic. (2) misbehaving DetNet flows must be detected and it
>> have to be ensured that there do not compromise DetNet QoS. I will
>> reformulate.*
>>
>> “Network coding”
>>
>>    - Proper name, Network Coding, s/b capitalized.
>>
>> *>Balazs> OK. *
>>
>> “packet by packet”
>>
>>    - s/b hyphenated “packet-by-packet”.
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “Since Detnet leverages many different forwarding sub-layers, those
>> technologies also support a number of tools to troubleshoot connectivity
>> for example, to support identification of misbehaving  flows.”
>>
>>    - Try:
>>
>>  “Detnet leverages many different forwarding sub-layers, each of which
>> supports various tools to troubleshoot connectivity, for example
>> identification of misbehaving flows.”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “At the service layer again there are existing mechanisms to troubleshoot
>> or monitor flows.  Many of these mechanisms exist for IP and MPLS
>> networks.  A client of a DetNet service can introduce any monitoring
>> applications which can detect and monitor delay and loss.”
>>
>>    - Try:
>>
>> “The DetNet Service layer can leverage existing mechanisms to
>> troubleshoot or monitor flows, such as those in use by IP and MPLS
>> networks.  At the Application layer a client of a DetNet service can use
>> existing techniques to detect and monitor delay and loss.”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “Recognize flow(s) for analytics:”
>>
>>    - Try
>>
>> “Flow recognition for analytics:
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “To a large degree this follows the logic in the previous section.
>> Analytics can be inherited from the two sub-layers.  At the DetNet service
>> edge packet and bit counters e.g. sent, received, dropped, and out of
>> sequence are maintained.”
>>
>>    - Try:
>>
>> “Network analytics can be inherited from the technologies of the Service
>> and Forwarding sub-layers.  At the DetNet service edge, packet and bit
>> counters (e.g. sent, received, dropped, and out-of-sequence) can be
>> maintained.”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “Correlate events with flows:”
>>
>>       The provider of a DetNet service may allow other capabilities to
>> monitor flows such as more detail loss statistics and time stamping of
>> events.  The details of these capabilities are currently out of scope for
>> this document.”
>>
>>    - Try:
>>
>> “Correlation of events with flows:”
>>
>>       The provider of a DetNet service may provide other capabilities to
>> monitor flows, such as more detailed loss statistics and time stamping of
>> events.  The details of these capabilities are currently out of scope for
>> this document.”
>>
>> *>Balazs> OK. *
>>
>> 3.6.1
>>
>> Grammar:
>>
>> “in the case of network congestion or some failures.”
>>
>>    - Try:
>>
>> “in the case of network congestion or equipment failure.”
>>
>> *>Balazs> OK in principle. Failures can be both equipment failure or
>> network link failure, so I will change to ““in the case of network
>> congestion or network failure.””*
>>
>> Abbreviation error:
>>
>> “combinations of PRF, PRE, and POF.”
>>
>>    - Try:
>>
>> “combinations of PRF, PEF, and POF.”
>>
>> *>Balazs> OK. *
>>
>> ‘“This example also illustrates 1:1 protection scheme meaning there is
>> traffic and path for each segment of the end to end path.”
>>
>>    - I do not understand what this means; “there is traffic and path”?
>>
>> *>Balazs> OK. “… meaning there is traffic over each segment …”*
>>
>> Grammar: (Ring Service Protection is a proper name, capitalized, “rings”
>> is not).
>>
>> ‘“Many of the same concepts apply however Rings”
>>
>>    - “Many of the same concepts apply, however rings”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “The DetNet data plane also allows for the aggregation of DetNet flows,
>> to improved scaling by reducing the state per hop...”
>>
>>    - Try:
>>
>> “The DetNet data plane also allows for the aggregation of DetNet flows,
>> which can improve scalability by reducing the per-hop state.”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “If bandwidth reservations are used, the sum of the reservations should
>> be the sum of all the individual reservations, in other words, the
>> reservations should not create an over subscription of bandwidth
>> reservation. If maximum delay bounds are used the system should ensure that
>> the aggregate does not exceed the delay bounds of the individual flows.”
>>
>>    - Try:
>>
>> “If bandwidth reservations are used, the sum of the reservations should
>> be the sum of all the individual reservations; in other words, the
>> reservations should not add up to an over-subscription of bandwidth
>> reservation. If maximum delay bounds are used, the system should ensure
>> that the aggregate does not exceed the delay bounds of the individual
>> flows.”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “Encapsulation can either be in the same service type or in a different
>> service type see Figure 3 for example.  When an encapsulation is used the
>> choice of reserving a maximum resource level and then tracking the services
>> in the aggregated service or adjusting the aggregated resources as the
>> services are added is implementation and technology specific.”
>>
>>    - Try:
>>
>> “Encapsulation can either be in the same service type or in a different
>> service type (see Figure 3, for example).  When an encapsulation is used,
>> the choice of reserving a maximum resource level and then tracking the
>> services in the aggregated service, versus adjusting the aggregated
>> resources as the services are added, is implementation- and
>> technology-specific.”
>>
>> In the following, I don’t know what “conditions where general
>> requirements are not satisfied” means, please clarify:
>>
>> “DetNet flows at edges must be able to handle rejection to an aggregation
>> group due to lack of resources as well as conditions where general
>> requirements are not satisfied.”
>>
>> *>Balazs> OK. I think here we need some reconstruction of the text.*
>>
>> Grammar:
>>
>> “For the data plane flows may be aggregated for treatment based on shared
>> characteristics such as 6-tuple.”
>>
>>    - Need comma after “For the data plane,”.
>>
>> *>Balazs> OK. I think here we need some reconstruction of the text.*
>>
>> Grammar:
>>
>> “MPLS aggregation similarly has data plane and controller plane aspects.
>> In the case of MPLS flows are often tunneled in a forwarding sub-layer and
>> reservation is associated with that MPLS tunnel.”
>>
>>    - Try (I am not certain this is the correct meaning?):
>>
>> “MPLS aggregation also has data plane and controller plane aspects.  MPLS
>> flows are often tunneled in a forwarding sub-layer, under the reservation
>> associated with that MPLS tunnel.”
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “3.6.3.  End-System Specific Considerations”
>>
>>    - Hyphenate “End-System-Specific”
>>
>> *>Balazs> OK. *
>>
>> Abbreviation: “DN” for DetNet as in “which are not provided by DN
>> functions”.
>>
>>    - “DN” is not defined as an abbreviation, but is also used in Fig 2
>>    and Fig 3. In any of these cases either spell out DetNet or define DN as an
>>    abbreviation. I would just replace these kind of instances of “DN” with
>>    “DetNet”.
>>
>> *>Balazs> OK. *
>>
>> Grammar:
>>
>> “For example, a DetNet MPLS domain the DN functions use the d-CWs,
>> S-Labels and F-Labels to provide DetNet services.”
>>
>>    - Try:
>>
>> “For example, in a DetNet MPLS domain the DetNet functions use the d-CWs,
>> S-Labels and F-Labels to provide DetNet services.”
>>
>> *>Balazs> OK. *
>>
>> Terminology: “edge system”
>>
>>    - Don’t we say “edge node”? I don’t see “edge system” anywhere else
>>    in this draft. If it is used elsewhere so we are going to use it, I would
>>    think it should be hyphenated “edge-system” like “end-system”. Or maybe
>>    “edge system” in this context should be “end-system”.
>>
>> *>Balazs> OK. “edge node” is better here.*
>>
>> Abbreviation: “TDM technologies”
>>
>>    - TDM not defined.
>>
>> *>Balazs> OK. *
>>
>> Something is wrong grammatically, run-on sentence at best, but I can’t
>> figure out how to fix it, please help:
>>
>> “Flow aggregation includes aggregation accomplished through the use of
>> hierarchical LSPs in MPLS and tunnels, in the case of IP, MPLS and TSN, all
>> of which aggregate multiple DetNet flows into a single new DetNet flow.”
>>
>>    - Maybe something like:
>>
>> “Flow aggregation includes aggregation accomplished through the use of
>> hierarchical LSPs in MPLS, and through IP or TSN tunnels. IP, MPLS and TSN
>> are each capable of aggregating multiple DetNet flows into a single new
>> DetNet flow.”
>>
>> *>Balazs> OK. I will rewrite this sentences. TSN is also not correctly
>> referred here.*
>>
>> Grammar:
>>
>> “Depending on the specific technology the assigned resources are updated
>> and distributed in the databases preventing over subscription.”
>>
>>    - Try:
>>
>>  “Depending on the specific technology, the assigned resources are
>> updated and distributed in the databases, preventing over-subscription.”
>>
>> *>Balazs> OK. *
>>
>> Question about the following:
>>
>> “While the DetNet IP data plane must support bidirectional DetNet flows,
>> there are no special bidirectional features with respect to the data plane
>> other than the need for the two directions of a co-
>>
>>    routed bidirectional flow to take the same path.  That is to say that
>> bidirectional DetNet flows are solely represented at the management and
>> control plane levels, without specific support or knowledge within the
>> DetNet data plane.”
>>
>>    - There was discussion about use cases that wanted symmetrical delays
>>    in both directions; is that guaranteed by taking “the same path”?
>>    Presumably it is not implied by “fate sharing”?
>>
>> *>Balazs> Hmmm. Same path not necessarily results in symmetrical delay.
>> Some technologies provides asymmetric link delays by nature. *
>>
>> Wording:
>>
>> “DetNet's use of PREOF may increase the complexity of using co-routing
>> bidirectional flows, since if PREOF is used, then the replication points in
>> one direction would have to match the elimination points in the other
>> direction, and vice versa, and the optimal points for these functions in
>> one direction may not match the optimal points in the other subsequent to
>> the network and traffic constraints.”
>>
>>    - Try:
>>
>> “DetNet's use of PREOF may increase the complexity of using co-routing
>> bidirectional flows, since if PREOF is used, then the replication points in
>> one direction would have to match the elimination points in the other
>> direction, and vice versa. In such cases the optimal points for these
>> functions in one direction may not match the optimal points in the other,
>> due to network and traffic constraints.”
>>
>> *>Balazs> OK. *
>>
>> ---------------------------------------
>>
>>
>> _______________________________________________
>> detnet mailing list
>> detnet@ietf.org
>> https://www.ietf.org/mailman/listinfo/detnet
>>
>