[Detnet] draft-ietf-detnet-architecture-07

[János Farkas <[e-mail]>]

Hi,

We have updated the architecture draft to resolve the extended WGLC 
comments. The current revision is v07: 
https://tools.ietf.org/html/draft-ietf-detnet-architecture-07.

All WGLC comments have been addressed; most of them can be found below. 
Some further nits have been also resolved.

The main changes:

1)
Based on the last discussion item below, a new sentence has been added 
to congestion protection in section 3.1, where it is first described:
"Note that congestion
protection provided via congestion detection and notification is
explicitly excluded from consideration in DetNet, as it serves a
different set of applications."

(Note that throttling/throttled has been already part of the text, there 
are three occurrences.)

2)
Based on the discussion below, the following introductory text has been 
added to section 4.1:
"DetNet functionality (Section 3) is implemented in two adjacent
layers in the protocol stack: the DetNet service layer and the DetNet
transport layer. The DetNet service layer provides DetNet service,
e.g., service protection, to higher layers in the protocol stack and
applications. The DetNet transport layer supports DetNet service in
the underlying network, e.g., by providing explicit routes and
congestion protection to DetNet flows."

Consequently, the definition of DetNet service layer has been updated to:
"The layer at which A DetNet service, e.g., service protection
is provided."

3)
Based on 
https://www.ietf.org/mail-archive/web/detnet/current/msg01577.html, the 
following sentence has been added to the OAM paragraph in section 4.1.1:
"Active and hybrid OAM
methods require additional bandwidth to perform fault management and
performance monitoring of the DetNet domain. OAM may, for instance,
generate special test probes or add OAM information into the data
packet."

4)
Based on the consensus reached at the DetNet session at IETF 102, 
Controller Plane Entity (CPE) has been replaced with Controller Plane 
Function (CPF).

5)
In order to clarify, the last sentence of the introduction has been 
updated to:
"DetNet can
accommodate various time synchronization techniques and profiles that
are defined elsewhere to address the needs of different market
segments."

6)
In order to clarify, the packet loss bullet in section 3.1 has been 
updated to:
"Packet loss ratio, under various assumptions as to the operational
states of the nodes and links."

7)
DetNet t-aware has been updated to:
"A DetNet transport-aware system. It knows about
some TSN functions (e.g., reservation), but not about service
protection."

8)
DetNet s-aware has been updated to:
"DetNet service-aware system. It supplies
sequence numbers, but doesn’t know about zero congestion loss."


Best regards,
János


On 7/19/2018 2:04 AM, Prof. Diego Dujovne wrote:
> +1
>
> El mié., 18 de jul. de 2018 19:53, Black, David <[e-mail] 
> <mailto:[e-mail]>> escribió:
>
>     +1, Thanks, --David
>
>     *From:*Lou Berger [mailto:[e-mail] <mailto:[e-mail]>]
>     *Sent:* Wednesday, July 18, 2018 6:52 PM
>     *To:* Norman Finn; Janos Farkas; Black, David
>     *Cc:* DetNet WG
>     *Subject:* Re: [Detnet] WG Last Call:
>     draft-ietf-detnet-architecture-05
>
>     This sounds sensible to me...
>
>     Lou
>
>     ------------------------------------------------------------------------
>
>     On July 18, 2018 6:47:01 PM Norman Finn
>     <[e-mail]
>     <mailto:[e-mail]>> wrote:
>
>         "Congestion loss protection" is better than "congestion
>         protection".  That's good.
>
>         It's a big change, and makes the document unnecessarily
>         wordy.  That's bad.
>
>         I would use "congestion protection" but explain somewhere that
>         we are not talking about source throttling.
>
>         And if adding that explanation is difficult or controversial,
>         I would stick with "congestion protection".
>
>         -- Norm
>
>         ------------------------------------------------------------------------
>
>         *From:*detnet [[e-mail]
>         <mailto:[e-mail]>] on behalf of Janos Farkas
>         [[e-mail] <mailto:[e-mail]>]
>         *Sent:* Wednesday, July 18, 2018 2:26 PM
>         *To:* Black, David
>         *Cc:* DetNet WG
>         *Subject:* Re: [Detnet] WG Last Call:
>         draft-ietf-detnet-architecture-05
>
>         Both “congestion protection” and “congestion loss protection”
>         work for me.
>
>         Note that the architecture document is quite populated with
>         the term “congestion protection”. I do not know the other
>         documents by heart.
>
>         So it would be a bit of a change..
>
>         I’d like to have the group’s opinion before making such a
>         change, e.g., any objections?
>
>         Thanks,
>
>         Janos
>
>         *From:*Black, David <[e-mail]
>         <mailto:[e-mail]>>
>         *Sent:* Wednesday, July 18, 2018 8:19 PM
>         *To:* Janos Farkas <[e-mail]
>         <mailto:[e-mail]>>
>         *Cc:* DetNet WG <[e-mail] <mailto:[e-mail]>>
>         *Subject:* RE: [Detnet] WG Last Call:
>         draft-ietf-detnet-architecture-05
>
>         Hi Janos,
>
>         That text works nicely.  In briefly talking to Lou, a related
>         suggestion surfaced: “congestion protection” -> “congestion
>         loss protection” where possible (may need to be abbreviated in
>         the Figure) .
>
>         This change would make it clearer that the primary focus is
>         protection from losses caused by congestion as opposed to
>         protection from all congestion. DetNet should be able to
>         tolerate some modest congestion (e.g., in nodes that don’t
>         have reserved DetNet resources) as long as delivery is
>         reliable and timely.
>
>         Thanks, --David
>
>         *From:*Janos Farkas [mailto:[e-mail]]
>         *Sent:* Wednesday, July 18, 2018 11:45 AM
>         *To:* Black, David
>         *Cc:* DetNet WG
>         *Subject:* RE: [Detnet] WG Last Call:
>         draft-ietf-detnet-architecture-05
>
>         Hi David,
>
>         This implies updating the definition of the DetNet service
>         layer, e.g., to:
>
>         DetNet service layer
>
>         The layer at which A DetNet Service, e.g., service protection
>         is provided.
>
>         The definition of the DetNet transport layer remains the same:
>
>         DetNet transport layer
>
>         The layer that optionally provides congestion protection for
>
>         DetNet flows over paths provided by the underlying network.
>
>         Regards,
>
>         Janos
>
>         *From:*Black, David <[e-mail]
>         <mailto:[e-mail]>>
>         *Sent:* Monday, July 16, 2018 3:21 PM
>         *To:* Janos Farkas <[e-mail]
>         <mailto:[e-mail]>>
>         *Cc:* DetNet WG <[e-mail] <mailto:[e-mail]>>
>         *Subject:* RE: [Detnet] WG Last Call:
>         draft-ietf-detnet-architecture-05
>
>         Don’t use “congestion protection” in the definitions for both
>         levels.
>
>         Thanks, --David
>
>         *From:*János Farkas [mailto:[e-mail]]
>         *Sent:* Sunday, July 15, 2018 10:43 PM
>         *To:* Black, David
>         *Cc:* DetNet WG
>         *Subject:* Re: [Detnet] WG Last Call:
>         draft-ietf-detnet-architecture-05
>
>         Thank you David!
>
>         I think we are set. :-)
>
>         You got a point, we should update
>         "DetNet service layer is the layer at which A DetNet Service,
>         such as congestion or service protection is provided."
>         to
>         "DetNet service layer is the layer at which A DetNet Service,
>         such as congestion protection and/or service protection is
>         provided."
>         in the definitions as well to make it clear.
>
>         Thanks,
>         Janos
>
>         On 7/16/2018 3:03 AM, Black, David wrote:
>
>             Moving the only two open items to the top …
>
>             -- Section 4
>
>             > We could add some introductory text to 4.1 or 4.1.1, for
>             instance:
>             > The functionality needed for DetNet (Section 3) are
>             implemented in the DetNet layer which comprises the DetNet
>             service layer and the DetNet transport layer.
>
>             > The DetNet service layer functions rely on flow
>             identification and sequencing. The DetNet transport layer
>             function may rely on flow identification.
>             >
>             > What do you think?
>
>             I prefer functional (what it does) characterization to
>             mechanistic (how it works) characterization in an
>             overview, e.g., excerpting from what I originally wrote:
>
>             ·The DetNet Service layer provides timely reliable
>             in-order DetNet flow delivery to applications that use DetNet.
>
>             ·The DetNet Transport layer insulates DetNet flows from
>             interference by network mechanisms, primarily queuing and
>             routing
>
>             I understand the principle.
>
>             What about just copying the two definitions here:
>             DetNet service layer is the layer at which A DetNet
>             Service, such as congestion or service protection is provided.
>             DetNet transport layer optionally provides congestion
>             protection for DetNet flows over paths provided by the
>             underlying network.
>
>             David> Yes, that works, although I don’t think
>             “congestion” was meant to be included in the first
>             definition …
>
>                Many real-time networks rely on physical rings or
>             chains of two-port
>
>                devices, with a relatively simple ring control protocol.
>
>             How does a “ring control protocol” control “chains”? 
>             Suggest: “ring control protocol” -> “control protocol”
>
>             Also, are paired chains between common endpoints intended,
>             or a more general notion of chains?
>
>             This text refers to typical factory deployment, which is
>             chain or ring..
>             It seems better to keep “ring control protocol” because
>             they are often very different from generic control
>             protocols applicable to mesh networks.
>
>             Would flipping chain and ring resolve your concern?
>
>                Many real-time networks rely on physical chains or
>             rings of two-port
>
>                devices, with a relatively simple ring control protocol.
>
>             */[David>] /*It feels like “ring control protocol” is an
>             important phrase to retain. In the original text, would it
>             be reasonable to add “(applies to both rings and chains)”
>             to the end of the sentence?
>
>             Well, I guess the point is that there are many specific
>             ring protocols, but we are looking for a generic solution.
>             What about deleting "chains" from the sentence?
>
>             David> Yes, if the sentence is all about rings, it’s clear.
>
>             Thanks, --David
>
>             *From:*János Farkas [mailto:[e-mail]]
>             *Sent:* Sunday, July 15, 2018 7:25 PM
>             *To:* Black, David
>             *Cc:* DetNet WG
>             *Subject:* Re: [Detnet] WG Last Call:
>             draft-ietf-detnet-architecture-05
>
>             Hi David,
>
>             Please see in-line.
>
>             Thank you!
>             Janos
>
>             On 7/6/2018 11:50 PM, Black, David wrote:
>
>                 -- Bridged path definition
>
>                 >  Or should we do something else?
>
>                 > There is only one occurrence of "bridged path" in the
>                 text in brackets.
>
>                 That sounds like a better suggestion – change that
>                 occurrence and remove the definition of “bridged path”
>                 from the draft.  That occurrence is in Section 4.1.1:
>
>                 Congestion protection
>
>                 The DetNet transport layer provides congestion protection.
>
>                 See Section 4.5.  The actual queuing and shaping
>                 mechanisms
>
>                 are typically provided by underlying subnet layers,
>                 these can
>
>                 be closely associated with the means of providing
>                 paths for
>
>                           DetNet flows (e.g., MPLS LSPs or bridged
>                 paths), the path and
>
>                 the congestion protection are conflated in this figure.
>
>                 Perhaps: (e.g., MPLS LSPs or bridged paths) -> (MPLS
>                 LSPs or other network paths)
>
>                 or just delete the entire parenthetical example.
>
>             OK. Let's just delete the entire parenthetical example.
>
>                 -- Section 4
>
>                 > We could add some introductory text to 4.1 or 4.1.1,
>                 for instance:
>                 > The functionality needed for DetNet (Section 3) are
>                 implemented in the DetNet layer which comprises the
>                 DetNet service layer and the DetNet transport layer.
>
>                 > The DetNet service layer functions rely on flow
>                 identification and sequencing. The DetNet transport
>                 layer function may rely on flow identification.
>                 >
>                 > What do you think?
>
>                 I prefer functional (what it does) characterization to
>                 mechanistic (how it works) characterization in an
>                 overview, e.g., excerpting from what I originally wrote:
>
>                   * The DetNet Service layer provides timely reliable
>                     in-order DetNet flow delivery to applications that
>                     use DetNet.
>                   * The DetNet Transport layer insulates DetNet flows
>                     from interference by network mechanisms, primarily
>                     queuing and routing
>
>             I understand the principle.
>
>             What about just copying the two definitions here:
>             DetNet service layer is the layer at which A DetNet
>             Service, such as congestion or service protection is provided.
>             DetNet transport layer optionally provides congestion
>             protection for DetNet flows over paths provided by the
>             underlying network.
>
>
>                 > Should we also rearrange, e.g., indent the text after
>                 Figure 2 such that
>                 > Packet sequencing, Duplicate elimination, Flow
>                 replication, Flow merging, Packet encoding, and Packet
>                 decoding get under Service layer
>                 > and
>                 > Congestion protection and Explicit routes get under
>                 Transport layer?
>
>                 No, I don’t think that’s necessary – Figure 2 provides
>                 sufficient structure.
>
>             OK.
>
>                 Plus a few more minor comments inline …
>
>                 Thanks, --David
>
>                 *From:*János Farkas [mailto:[e-mail]]
>                 *Sent:* Wednesday, July 4, 2018 2:15 PM
>                 *To:* Black, David
>                 *Cc:* DetNet WG
>                 *Subject:* Re: [Detnet] WG Last Call:
>                 draft-ietf-detnet-architecture-05
>
>                 Hi David,
>
>                 Thank you very much for your review and comments!
>
>                 Please see in-line.
>
>                 On 6/29/2018 4:10 PM, Black, David wrote:
>
>                     The -06 version of this draft is much improved
>                     over the -05 version – lots of diligent work by
>                     the authors is evident and appreciated.
>
>                     I have two minor content concerns and some smaller
>                     editorial items.
>
>                     ** Content
>
>                     -- Section 2.2
>
>                     The definition of “bridged path” looks like a
>                     definition of “bridged forwarding” – the
>                     definition needs to encompass the notion of a path
>                     that involves more than one bridge.
>
>                        bridged path
>
>                                A VLAN bridge uses the VLAN ID and the
>                     destination MAC
>
>                                address to select the outbound port
>                     hence the path for a
>
>                                frame.
>
>                 You are right, it looks like the emphasis is on bridge
>                 forwarding rather than path.
>                 However, the superposition of the outbound ports
>                 provides a path, which was tried to be captured by
>                 "hence the path".
>                 The idea was to keep it simple not dive into the
>                 details, which are not necessary in this document.
>
>                 Would it be good to update to:
>
>                    bridged path
>
>                            A VLAN bridge uses the VLAN ID and the
>                 destination MAC
>
>                            address to select the outbound port. The
>                 superposition
>
>                            of the outbound ports is the path for a frame.
>
>                 ?
>
>                 Or should we do something else?
>                 There is only one occurrence of "bridged path" in the
>                 text in brackets.
>
>                     -- Section 4
>
>                     Figure 2 in Section 4.1.1 introduces the Detnet
>                     service and transport layers.   The associated
>                     text quickly dives into explaining the
>                     functionality in those layers, but it would be
>                     helpful to start with an initial high level
>                     description of each of those layers that helps
>                     motivate their separation.  It appears that:
>
>                     - The DetNet Service layer provides timely
>                     reliable in-order DetNet flow delivery to
>                     applications that use DetNet.   A DetNet flow at
>                     the top of the DetNet Service layer may or may not
>                     be a compound flow at the bottom of that layer.
>
>                     - The DetNet Transport layer insulates DetNet
>                     flows (at the bottom of the DetNet Service layer)
>                     from interference by network mechanisms, primarily
>                     queuing and routing, where congestion is an
>                     indication of queueing interference.  A DetNet
>                     flow at the top of the DetNet Transport layer may
>                     or may not be a member flow that is part of a
>                     compound flow, depending on what is done in the
>                     DetNet Service layer.
>
>
>                 You are right, section 4 jumps into the details
>                 without much of an introduction. I guess we had a kind
>                 of chicken and egg problem here. Figure 4 provides a
>                 higher level view than Figure 2. However, Figure 4
>                 captures different aspects, dives into the details of
>                 data plane options. So, it seems better to keep the order.
>
>                 We could add some introductory text to 4.1 or 4.1.1,
>                 for instance:
>                 The functionality needed for DetNet (Section 3) are
>                 implemented in the DetNet layer which comprises the
>                 DetNet service layer and the DetNet transport layer.
>                 The DetNet service layer functions rely on flow
>                 identification and sequencing. The DetNet transport
>                 layer function may rely on flow identification.
>
>                 What do you think?
>
>                 Should we also rearrange, e.g., indent the text after
>                 Figure 2 such that
>                 Packet sequencing, Duplicate elimination, Flow
>                 replication, Flow merging, Packet encoding, and Packet
>                 decoding get under Service layer
>                 and
>                 Congestion protection and Explicit routes get under
>                 Transport layer?
>
>                     There’s a related paragraph of text between
>                     Figures 3 and 4 in Section 4.1.2, but it doesn’t
>                     provide useful intuition about which functionality
>                     belongs in which layer.
>
>                     ** Editorial
>
>                     -- Section 2.1
>
>                     “expected” seems off in the definition of reservation:
>
>                     reservation
>
>                     The set of resources allocated between a source
>                     and one or
>
>                     more destinations through transit nodes and subnets
>
>                     associated with a DetNet flow, to provide the
>                     expected DetNet
>
>                     Service.
>
>                     Suggest either “to provide the provisioned DetNet
>                     Service to that flow”
>
>                 I like this one, will update to this one.
>
>                     or “to provide the specified DetNet Service to
>                     that flow.”
>
>                     -- Section 3.1
>
>                        o  An upper bound on out-of-order packet
>                     delivery.  It is worth
>
>                           noting that some DetNet applications are
>                     unable to tolerate any
>
>                           out-of-order delivery.
>
>                     Is that a sequence bound or a time bound or both?
>                     I suspect that a sequence bound (maximum degree of
>                     reordering in a flow, independent of flow rate)
>                     was intended.  An analogous clarification would be
>                     helpful in section 3.2.2.1 .
>
>                 It is intentionally kept high level in the
>                 architecture draft. Details are to be provided by the
>                 flow information model draft. Please see my previous
>                 mail:
>                 https://mailarchive.ietf.org/arch/msg/detnet/AWG2eemg_AWsvF2FwtREyd_xtC8.
>
>                 */[David>] /*OK.
>
>                     -- Section 3.2.3
>
>                     I don’t understand “defined” in:
>
>                        Even the use of
>
>                        redundant paths through a network defined,
>                     e.g., by [RFC6372] do not
>
>                        eliminate the chances of packet loss.
>
>                     Was “e.g., as defined by [RFC6372],” intended?
>
>                 Yes, this was the intention, will update to this one.
>
>                        Many real-time networks rely on physical rings
>                     or chains of two-port
>
>                        devices, with a relatively simple ring control
>                     protocol.
>
>                     How does a “ring control protocol” control
>                     “chains”?  Suggest: “ring control protocol” ->
>                     “control protocol”
>
>                     Also, are paired chains between common endpoints
>                     intended, or a more general notion of chains?
>
>                 This text refers to typical factory deployment, which
>                 is chain or ring.
>                 It seems better to keep “ring control protocol”
>                 because they are often very different from generic
>                 control protocols applicable to mesh networks.
>
>                 Would flipping chain and ring resolve your concern?
>
>                    Many real-time networks rely on physical chains or
>                 rings of two-port
>
>                    devices, with a relatively simple ring control
>                 protocol.
>
>                     */[David>] /*It feels like “ring control protocol”
>                     is an important phrase to retain.   In the
>                     original text, would it be reasonable to add
>                     “(applies to both rings and chains)” to the end of
>                     the sentence?
>
>             Well, I guess the point is that there are many specific
>             ring protocols, but we are looking for a generic solution.
>             What about deleting "chains" from the sentence?
>
>                        Explicit routes can be established various ways,
>
>                     established various -> established in various
>
>                 OK.
>
>                        This is
>
>                        irrespective of the distribution method used,
>                     also applies to service
>
>                        protection over explicit routes.
>
>                     used, also -> used, and also
>
>                 OK.
>
>                     -- Figure 5 (Section 4.1.3)
>
>                     The UNI acronym is introduced without expansion or
>                     definition – please correct that, e.g., by
>                     defining UNI in Section 2.1.
>
>                 DetNet-UNI is there among the definitions. It seems
>                 enough to me.
>
>                 */[David>] /*That’s fine – I overlooked that
>                 definition, sorry.
>
>                     -- Figure 9 (Section 4.7.2)
>
>                     I think I understand what’s going on, but the use
>                     of “add/remove” in the figure should be explained
>                     in associated text.
>
>                 What about updating the sentence above Figure 9 to:
>                 A packet with corresponding Flow-IDs is illustrated in
>                 Figure 9, which also indicates where Flow-ID can be
>                 added or removed.
>
>                 */[David>] /*Sure, with one minor edit: “where
>                 Flow-ID” -> “where each Flow-ID”
>
>             OK.
>
>                     Thanks, --David
>
>
>                 Thank you!
>                 Janos
>
>                     *From:*detnet [mailto:[e-mail]] *On
>                     Behalf Of *János Farkas
>                     *Sent:* Thursday, June 28, 2018 9:09 PM
>                     *To:* Lou Berger; Stewart Bryant; [e-mail]
>                     <mailto:[e-mail]> >> DetNet WG
>                     *Subject:* Re: [Detnet] WG Last Call:
>                     draft-ietf-detnet-architecture-05
>
>                     Dear all,
>
>                     Off-line discussions among Lou, Stewart, and
>                     authors followed the discussions to properly
>                     address the WGLC comments, including the detailed
>                     comments.
>
>                     A new revision of the draft has been uploaded:
>                     draft-ietf-detnet-architecture-06.
>
>                     In addition to the changes already described in
>                     this thread, the following bigger changes have
>                     been made to the draft:
>
>
>                     *Section 2.1 Terms used in this document*
>
>                     Some definitions refined as suggested by the
>                     detailed comments
>
>                     New definitions have been added:
>
>                        "allocation
>                                Resources are dedicated to support a
>                     DetNet flow.  Depending
>                                on an implementation, the resource may
>                     be reused by non-
>                                DetNet flows when it is not used by the
>                     DetNet flow.
>
>
>                        PEF     A Packet Elimination Function (PEF)
>                     eliminates duplicate
>                                copies of packets to prevent excess
>                     packets flooding the
>                                network or duplicate packets being sent
>                     out of the DetNet
>                                domain.  PEF can be implemented by an
>                     edge node, a relay
>                                node, or an end system.
>
>                       PRF     A Packet Replication Function (PRF)
>                     replicates DetNet flow
>                                packets and forwards them to one or
>                     more next hops in the
>                                DetNet domain.  The number of packet
>                     copies sent to each next
>                                hop is a DetNet flow specific parameter
>                     at the node doing the
>                                replication. PRF can be implemented by
>                     an edge node, a relay
>                                node, or an end system.
>
>                        PREOF   Collective name for Packet Replication,
>                     Elimination, and
>                                Ordering Functions.
>
>                        POF     A Packet Ordering Function (POF)
>                     re-orders packets within a
>                                DetNet flow that are received out of
>                     order.  This function
>                                can be implemented by an edge node, a
>                     relay node, or an end
>                                system.
>
>                       DetNet service proxy
>                                Maps between App-flows and DetNet flows.
>
>                        bridged path
>                                A VLAN bridge uses the VLAN ID and the
>                     destination MAC
>                                address to select the outbound port
>                     hence the path for a
>                                frame."
>
>
>                     *Section 3.1 Primary goals defining the DetNet QoS*
>
>                     A new QoS aspect has been added:
>                        "o  An upper bound on out-of-order packet
>                     delivery.  It is worth
>                           noting that some DetNet applications are
>                     unable to tolerate any
>                           out-of-order delivery."
>
>
>                     The 3rd paragraph on loss on page 8 after the
>                     bullet list has been extended to:
>                        "After congestion, the most important
>                     contributions to packet loss are
>                        typically from random media errors and
>                     equipment failures. Service
>                        protection is the name for the mechanisms used
>                     by DetNet to address
>                        these losses.  The mechanisms employed are
>                     constrained by the
>                        requirement to meet the users' latency
>                     requirements.  Packet
>                        replication and elimination (Section 3.2.2) and
>                     packet encoding
>                        (Section 3.2.2.3) are described in this
>                     document to provide service
>                        protection; others may be found.  For instance,
>                     packet encoding can
>                        be used to provide service protection against
>                     random media errors,
>                        packet replication and elimination can be used
>                     to provide service
>                        protection against equipment failures. This
>                     mechanism distributes
>                        the contents of DetNet flows over multiple
>                     paths in time and/or
>                        space, so that the loss of some of the paths
>                     does need not cause the
>                        loss of any packets."
>
>
>                     *3.2.2.  Service Protection*
>
>                     Service protection is used as a more generic term.
>                     Introductory text added:
>                        "Service protection aims to mitigate or
>                     eliminate packet loss due to
>                        equipment failures, random media and/or memory
>                     faults.  These types
>                        of packet loss can be greatly reduced by
>                     spreading the data over
>                        multiple disjoint forwarding paths. Various
>                     service protection
>                        methods are described in [RFC6372], e.g., 1+1
>                     linear protection.
>                        This section describes the functional details
>                     of an additional method
>                        in Section 3.2.2.2, which can be implemented as
>                     described in
>                        Section 3.2.2.3 or as specified in
>                     [I-D.ietf-detnet-dp-sol-mpls] in
>                        order to provide 1+n hitless protection.  The
>                     appropriate service
>                        protection mechanism depends on the scenario
>                     and the requirements."
>
>
>                     New sub-section added:
>
>                     "3.2.2.1.  In-Order Delivery
>
>                        Out-of-order packet delivery can be a side
>                     effect of service
>                        protection.  Packets delivered out-of-order
>                     impact the amount of
>                        buffering needed at the destination to properly
>                     process the received
>                        data.  Such packets also influence the jitter
>                     of a flow.  The DetNet
>                        service includes maximum allowed misordering as
>                     a constraint.  Zero
>                        misordering would be a valid service constraint
>                     to reflect that the
>                        end system(s) of the flow cannot tolerate any
>                     out-of-order delivery.
>                        Service protection may provide a mechanism to
>                     support in-order
>                        delivery."
>
>
>                     *3.2.2.2. Packet Replication and Elimination*
>
>                     New bullet added as the last one:
>                        "o  The Packet Ordering Function (POF) uses the
>                     sequencing information
>                           to re-order a DetNet flow's packets that are
>                     received out of
>                           order."
>
>                     New sentence added after the bullet list:
>                     "The order in which a node applies the PEF and the
>                     PRF to a DetNet
>                     flow is implementation specific."
>
>                     2nd paragraph after the bullet list has been
>                     updated to:
>                     "Some service protection mechanisms rely on
>                     switching from one flow to
>                        another when a failure of a flow is detected. 
>                     Contrarily, packet
>                        replication and elimination combines the DetNet
>                     member flows sent
>                        along multiple different paths, and performs a
>                     packet-by-packet
>                        selection of which to discard, e.g., based on
>                     sequencing information."
>
>
>                     *3.2.3.  Explicit routes*
>
>                     Out-of-order aspect added to the first paragraph,
>                     which is about distributed routing:
>                     "Furthermore, out-of-order
>                     packet delivery can be a side effect of route
>                     changes."
>
>                     New sentence added to the 3rd paragraph:
>                     "Explicit routes can be established various
>                     ways, e.g., with RSVP-TE [RFC3209], with Segment
>                     Routing (SR)
>                     [I-D.ietf-spring-segment-routing], via a Software
>                     Defined Networking
>                     approach [RFC7426], with IS-IS [RFC7813], etc."
>
>                     New paragraph added:
>                        "Out-of-order packet delivery can be a side
>                     effect of distributing a
>                        single flow over multiple paths especially when
>                     there is a change
>                        from one path to another when combining the
>                     flow.  This is
>                        irrespective of the distribution method used,
>                     also applies to service
>                        protection over explicit routes.  As described
>                     in Section 3.2.2.1,
>                        out-of-order packets influence the jitter of a
>                     flow and impact the
>                        amount of buffering needed to process the data;
>                     therefore, DetNet
>                        service includes maximum allowed misordering as
>                     a constraint.  The
>                        use of explicit routes helps to provide
>                     in-order delivery because
>                        there is no immediate route change with the
>                     network topology, but the
>                        changes are plannable as they are between the
>                     different explicit
>                        routes."
>
>                     *
>                     4.1.1. Representative Protocol Stack Model*
>
>                     "Explicit routes" have been added to Figure 2 with
>                     the corresponding explanation:
>                      "Explicit routes
>                                The DetNet transport layer provides
>                     mechanisms to ensure that
>                                fixed paths are provided for DetNet
>                     flows.  These explicit
>                                paths avoid the impact of network
>                     convergence."
>
>
>                     Section 4.11 Connected islands vs. networks of v05
>                     has been deleted because it was just a leftover
>                     from early drafts on what DetNet WG should do;
>                     which are covered by the charter anyways.
>
>
>                     References have been cleaned up and brought
>                     up-to-date.
>
>
>                     Refinements have been implemented in the draft
>                     according to Lou's detailed comments. They are not
>                     listed here because they are minor changes.
>
>
>                     Best regards,
>                     Janos
>
>                     On 6/12/2018 2:48 PM, Lou Berger wrote:
>
>
>                         Balázs,
>
>                         Thanks for the response -- please see below.
>
>                         ----------
>                         On June 12, 2018 4:07:35 AM Balázs Varga A
>                         <[e-mail]>
>                         <mailto:[e-mail]> wrote:
>
>                             Hi Lou, Thanks for the comments. See
>                             reactions inline. Document update in
>                             progress. Cheers Bala'zs
>
>                             -----Original Message-----
>                             From: detnet <[e-mail]>
>                             <mailto:[e-mail]> On Behalf
>                             Of Lou Berger
>                             Sent: 2018. június 1. 22:42
>                             To: DetNet WG <[e-mail]>
>                             <mailto:[e-mail]>;
>                             [e-mail]
>                             <mailto:[e-mail]>
>
>                             Subject: [Detnet] Promised comments on
>                             draft-ietf-detnet-architecture
>
>                             Hi,
>
>                                  I have a number of high level
>                             comments on the document that I'd like to
>                              raise below.  I also have a number of
>                             more editorial/specific comments that  I'd
>                             like to review directly with the authors
>                             and then have them report back  on changes
>                             -- if any turn out to be more substantive
>                             discussions from the  author's
>                             perspective, I'll raise these on the list
>                             separately.
>
>                         ...
>
>
>                             - WRT Section 4.4.3, I think the inclusion
>                             of a distributed control plane in the
>                             "Network Plane" is inconsistent with other
>                             functional definitions and conflates where
>                             a function resides from the actual
>                             function and that whether control is
>                             implemented in a fully centralized, fully
>                             distributed or some hybrid form doesn't
>                             fundamentally change  the control
>                             function's role -- therefore I think the
>                             sections 4..4.2 and .3 should be revised
>                             accordingly
>                             [Balázs Varga A] Agree in principal. Let's
>                             discuss the details.
>
>                         Okay - I'll work with you off line and we can
>                         report back the results/proposed changes.
>
>                             - In several places it's not clear that
>                             DetNet service is always a L3 service
>                             which is controlled using L3 identifiers,
>                             i.e., IP addresses -- this is true even in
>                             the MPLS service case and the TSN over
>                             MPLS case. I think this is an important
>                             point to be clear on in the document.
>                             [Balázs Varga A] I am not sure. DetNet
>                             service is always provided over a L3
>                             network (IP or MPLS), that is fine.
>                             However the service itself can be L2 or
>                             L3. In case of TSN Ethernet frames are
>                             transported, so it is a L2 service. In
>                             case of IP end systems IP packets are
>                             transported so it is a L3 service.
>
>
>                         Humm - While I agree that DetNet is providing
>                         an (enhanced) L2VPN service, it is not itself
>                         providing control or service of L2 devices --
>                         this is TSN's job.  The fact that DetNet is
>                         all about behavior of L3 nodes (i.e., IP and
>                         PW/MPLS) and not L2 nodes (i.e., TSN bridges)
>                         is something the architecture should make
>                         unambiguous.
>
>                         Thanks,
>                         Lou
>
>                             Please let me know what you think.
>
>                             Cheers,
>
>                             Lou
>
>
>                             _______________________________________________
>
>                             detnet mailing list
>                             [e-mail] <mailto:[e-mail]>
>                             https://www.ietf.org/mailman/listinfo/detnet
>                             _______________________________________________
>
>                             detnet mailing list
>                             [e-mail] <mailto:[e-mail]>
>                             https://www.ietf.org/mailman/listinfo/detnet
>
>
>
>                     On 6/12/2018 6:27 PM, Stewart Bryant wrote:
>
>                         Dear Bala'zs
>
>                         Thank you your for your consideration of these
>                         points. I will just pick up a few that need
>                         some further thought:
>
>                             DetNet transit node
>
>                             A node operating at the DetNet transport
>                             layer, that utilizes
>
>                             link layer and/or network layer switching
>                             across multiple
>
>                             links and/or sub-networks to provide paths
>                             for DetNet service
>
>                             layer functions. Optionally provides
>                             congestion protection
>
>                             over those paths. An MPLS LSR is an
>                             example of a DetNet
>
>                             transit node.
>
>                             SB> In that example it would have to be a
>                             DetNet enable/aware LSR. An
>
>                             SB> ordinary LSR would not know anything
>                             about DetNet.
>
>                             [Balázs Varga A] No, A DetNet aware LSR
>                             would be a relay node (S-PE).
>
>
>                         I think the confusion is what "DetNet
>                         Transport Layer" means. This
>                         technology touches on Transport Layer in the
>                         L4 sense, and the
>                         Transport Network Layer as in the packet
>                         network that carries
>                         L3 packets.
>
>                         So I think that we need to clarify whether a
>                         DetNet transit node
>                         is an S-PE (i.e. a a transit node in the
>                         DetNet layer), or a P node
>                         (i.e. a transit node that is carrying DetNet
>                         packets but could be
>                         carrying any sort of MPLS packet)
>
>                             ============
>
>                             These three techniques can be applied
>                             independently, giving eight
>
>                             possible combinations, including none (no
>                             DetNet), although some
>
>                             combinations are of wider utility than
>                             others.  This separation keeps
>
>                             the protocol stack coherent and maximizes
>                             interoperability with
>
>                             existing and developing standards in this
>                             (IETF) and other Standards
>
>                             Development Organizations. Some examples
>                             of typical expected
>
>                             combinations:
>
>                             o  Explicit routes plus service protection
>                             are exactly the techniques
>
>                             employed by [HSR-PRP]. Explicit routes are
>                             achieved by limiting
>
>                             the physical topology of the network, and
>                             the sequentialization,
>
>                             replication, and duplicate elimination are
>                             facilitated by packet
>
>                             tags added at the front or the end of
>                             Ethernet frames.
>
>                             SB> ER can be done virtually as well as
>                             physically. RSVP is a type of
>
>                             SB> ER, as is Adj-SID based SR, and we can
>                             design other types.
>
>                             [Balázs Varga A] Agree, but these are
>                             examples. Statement is for HSR-PRP.
>
>                         So the question is whether we should expand
>                         the set of examples, particularly
>                         to more accessible ones.
>
>                         ============
>
>                             Packet replication and elimination
>
>                             > > > > > > > > > relay > > > > > > > >
>
>                             > /------------+ R node E +------------\ >
>
>                             > / v + ^               \ >
>
>                             end    R + v | ^                + E end
>
>                             system + v | ^ +   system
>
>                             > \ v + ^               / >
>
>                             > \------------+ R relay E +-----------/ >
>
>                             > > > > > > > > > node > > > > > > > >
>
>                             Figure 1
>
>                             Packet replication and elimination does
>                             not react to and correct
>
>                             failures; it is entirely passive. Thus,
>                             intermittent failures,
>
>                             SB> I think it copes with intermittent
>                             failures OK.
>
>                             [Balázs Varga A] Yes, PRF and PEF can
>                             eliminate the effect of such failures. But
>                             text
>
>                             intends to say that it is passive. It is
>                             not reacting to such failures. No change
>                             in text.
>
>
>                         You say that PREF does not correct failures. I
>                         would contend that is exactly
>                         what it does. Sure it does not react but it
>                         does correct, and it does
>                         address intermittent failures.
>
>                             ===========
>
>                             transported between the peer end systems.
>                             Therefore, the following
>
>                             possible types / formats of a DetNet flow
>                             are distinguished in this
>
>                             document:
>
>                             o  App-flow: native format of a DetNet
>                             flow.  It does not contain any
>
>                             DetNet related attributes.
>
>                             o  DetNet-t-flow: specific format of a
>                             DetNet flow. Only requires
>
>                             the congestion / latency features provided
>                             by the Detnet transport
>
>                             layer.
>
>                             o  DetNet-s-flow: specific format of a
>                             DetNet flow. Only requires
>
>                             the replication/elimination feature
>                             ensured by the DetNet service
>
>                             layer.
>
>                             o  DetNet-st-flow: specific format of a
>                             DetNet flow.  It requires
>
>                             both DetNet service layer and DetNet
>                             transport layer functions
>
>                             during forwarding.
>
>                             SB> I find the relisting of these types
>                             confusing. Wheren't they defined
>
>                             SB> in the section above?
>
>                             [Balázs Varga A] This is inline with the
>                             previous section " Grouping of end systems ".
>
>                         Surely if we have defined them we never need
>                         to do anything but name them in
>                         later sections. Redefinition is never a good
>                         idea because it often leads to
>                         conflicting definitions.
>
>                             ============
>
>                             [HSR-PRP]  IEC, "High availability
>                             seamless redundancy (HSR) is a
>
>                             further development of the PRP approach,
>                             although HSR
>
>                             functions primarily as a protocol for
>                             creating media
>
>                             redundancy while PRP, as described in the
>                             previous
>
>                             section, creates network redundancy.  PRP
>                             and HSR are both
>
>                             described in the IEC 62439 3 standard.",
>
>                             <http://webstore.iec.ch/webstore/webstore.nsf/
>
>                             artnum/046615!opendocument>.
>
>                             SB> Not available at the time of this
>                             review, but my recollection is
>
>                             SB> that this is not readily available
>                             without paying a large fee.
>
>
>                         If we decide that this is essential as a key
>                         reference, there needs to be
>                         some suitable text, as this will get raised a
>                         number of times between
>                         here an publication as first the directorates
>                         and then the ADs run
>                         into this.
>
>                             ===========
>
>                             [ISA95] ANSI/ISA, "Enterprise-Control
>                             System Integration Part 1:
>
>                             Models and Terminology", 2000,
>
>                             <https://www.isa.org/isa95/>.
>
>                             SB> Should that be 2000, or 2010.
>
>                             SB> Note that this seems to be a very
>                             expensive document to access.
>
>                         You did not comment on the correctness of the
>                         reference.
>
>
>                         Best Regards
>
>                         Stewart
>
>         _______________________________________________
>         detnet mailing list
>         [e-mail] <mailto:detnet%40ietf.org>
>         https://www.ietf.org/mailman/listinfo/detnet
>
>     _______________________________________________
>     detnet mailing list
>     [e-mail] <mailto:[e-mail]>
>     https://www.ietf.org/mailman/listinfo/detnet
>