Re: [Detnet] Initial DP split documents available

[Stewart Bryant <[e-mail]>]

Hi,

Sorry this is so late. For some reason I though the interim was in May
and I did look at my DETNET folder these last few days.

- Stewart

SB> Following up on the list discussion, I think that
SB> we probably need:
SB> - DP Arch
SB> - MPLS DP
SB> - IP DP
SB> All as separate documents.

SB> I stopped reviewing at Section 6.3 since there is huge
SB> uncertainty beyont that point.

SB> Given the number of detailed comments, I have made, I wonder the
SB> editors and I should set up a shared screen session to deal with
SB> as many as possible and present the results of our discussions
SB> to the WG.

SB> I see the OAM indication material did not get incorporated
SB> from draft-bryant-detnet-mpls-dp-00. I think it needs to
SB> be here even if we have a separate OAM draft.

SB> Similarly I think the flow agregation text from
SB> draft-bryant-detnet-mpls-dp-00 should be included.

SB> There needs to be some payload type discussion (section 7 of
SB> draft-bryant-detnet-mpls-dp-00. )  It does not need to be that
SB> text, but it does nedd to be thought through.

SB> Section 9 of draft-bryant-detnet-mpls-dp-00 was an attempt
SB> at talking about the elements of proceedure, and I think we
SB> need something of that type in this text.

SB> We need to jointly review draft-bryant-detnet-mpls-dp-00
SB> Section 10 and section 11 and move some material over.

Please look for SB in the trimmed text below.


============


1.  Introduction

    Deterministic Networking (DetNet) is a service that can be offered by
    a network to DetNet flows.  DetNet provides these flows extremely low
    packet loss rates and assured maximum end-to-end delivery latency.
SB> DetNet provides these flows with an extremely low
    packet loss rates and an assured maximum end-to-end delivery latency.

======

2.  Terminology

2.1.  Terms used in this document


SB> The thing I keep looking for is a term for the DN object
SB> to use in the same way that I would use the work pseudowire
SB> We really need an agreed term so that we can easily talk
SB> about it as an object.

=======


    Local-ID      A DetNet Edge and Relay node internal construct that
                  uniquely identifies a DetNet flow within a node and
                  never appear on-wire.  It may be used to select proper
                  forwarding and/or DetNet specific service function.

SB> If it does not appear on the wire this is really control plane
SB> and thus out of scope.

=========

    PREF          A Packet Replication and Elimination Function (PREF)
                  does the replication and elimination processing of
                  DetNet flow packets in edge or relay nodes.  The
                  replication function is essentially the existing 1+n
                  protection mechanism.  The elimination function reuses
                  and extends the existing duplicate detection mechanism
                  to operate over multiple (separate) DetNet member flows
                  of a DetNet compound flow.

SB> I really think that we need to split the functions:
SB> PR, EF and ROF (re-order) You can have any of them at a
SB> node, and so not need to have them all.

=========
    DetNet Control Word  A control word used for sequencing and
                  identifying duplaicate packets at the DetNet service
                  layer.

2.2.  Abbreviations


    PREF          Packet Replication and Elimination Function.
SB> So here I think we are going to need PRF, EF and ROF.

==========

       TSN    |<---------- End to End DetNet Service ------>|  TSN
      Service |           Transit           Transit | Service
  TSN  (AC)   |        |<-Tunnel->| |<-Tnl->|        |  (AC)  TSN
  End    |    V        V     1    V        V   2   V V   |    End
  System |    +--------+          +--------+ +--------+   |  System
  +---+  |    |   E1   |==========|   R1   |=======|   E2 |   |   +---+
  |   |--|----|._X_....|..DetNet..|.._ _...|..DF3..|...._X_.|---|---|   |
  |CE1|  |    |    \   |  Flow 1  |   X    |       |   / |   |   |CE2|
  |   |       |     \_.|...DF2....|._/ \_..|..DF4..|._/ |       |   |
  +---+       |        |==========|        |=======| |       +---+
      ^       +--------+          +--------+ +--------+       ^
      |        Edge Node          Relay Node       Edge Node        |
|                                                             |
      |<----- Emulated Time Sensitive Networking (TSN) Service ---->|


                     Figure 1: IEEE 802.1TSN over DetNet

    Figure 2 illustrates how end to end MPLS-based DetNet service is
    provided in a more detail.  In this case, the end systems are able to
    send and receive DetNet flows.  For example, an end system sends data
    encapsulated in MPLS.

SB> I don't think that this is a likely scenario. What is more
SB> likely is a PE model with the Edge nodes as PEs.

========

    An example MPLS DetNet network fragment and packet flow is
    illustrated in Figure 3 .

       1111   11111111  111111   222222   2222222     333
    CE1----EN1--------R1-------R2-------R3--------EN2----CE2
             \2          22222/                 3 /
              \2222222  /----+                 3 /
               +------R4------------------------+
                        333333333333333333333333

        Figure 3: Example Packet flow in DetNet Enabled MPLS Network

    Customer Equipment CE1 send a packet into the DetNet enabled MPLS
    network.  Edge Node EN1
SB> I think that this is where the MPLS encap likely happens.

========

    1.  DetNet function related scenarios:

        *  Congestion protection and latency control: usage of allocated
           resources (queuing, policing, shaping).

        *  Explicit routes: select/apply the flow specific path.

        *  Service protection: recognize DetNet compound and member flows
           for replication an elimination.

        Comment #1  I am not sure whether the correct architectural
           construct is flow or flow group.  Flow suggests that sharing/
           aggregation is not allowed but whether this is allowed or not
           is an application specific issue.

        Discussion:  Agree that a flow group would be a better
           characterization.

SB> Agree

        Comment #2  I think that there needs to be some clarification as
           to whether FG is is understood by the DN system exclusively or
           whether there is an expectation that it is understood by the
           underlay.

        Discussion:  Agree that more detail is needed here. DetNet aware
           nodes need to understand flow groups.  Underlay needs to be
           aware of flow groups at the resource allocation level.

SB> The model I was using for VPN+ seems right. The underlay
SB> has resources set asside with certain properties. The
SB> control plane maps the respources to the slices.
SB> A Detnet FG runs on a slice. There are of course two methods
SB> of describing the mapping in the DP, explicit (a label for the 
resource),
SB> and implicit (a lable for the path which is mapped to the resource).

===========

    2.  OAM function related scenarios:

  <snip>

    Each DetNet node (edge, relay and transit) use an internal/
    implementation specific local-ID of the DetNet-(compound)-flow in
    order to accomplish its role during transport. Recognizing the
    DetNet flow is more relaxed for edge and relay nodes, as they are
    fully aware of both the DetNet service and transport layers.  The
    primary DetNet role of intermediate transport nodes is limited to
    ensuring congestion protection and latency control for the above
    listed DetNet functions.

SB> A relay node can be fully aware as well. The way I had in mind
SB> was for the relay nodes to recognise and operate on the
SB> service label.


    The DetNet data plane allows for the aggregation of DetNet flows,
    e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet
    flows are aggregated, transit nodes may have limited ability to
    provide service on per-flow DetNet identifiers. Therefore,
    identifying each individual DetNet flow on a transit node may not be
    achieved in some network scenarios, but DetNet service can still be
    assured in these scenarios through resource allocation and control.

    Comment #3  You could introduce the concept of a flow group
       identified into the packet.  You may also include a flow id at a
       lower layer.

SB> So do we need heirachical service labels?
SB> Not something we needed in PW, and a new MPLS construct, but
SB> quite feasible.

    Discussion:  Agree on the identification properties. Adding a
       specific id into actual on-wire formats is not necessarily needed.

SB> I do not understand this point.

    On each DetNet node dealing with DetNet flows, an internal local-ID
    is assumed to determine what local operation a packet goes through.
    Therefore, local-IDs has to be unique on each edge and relay nodes.
    Local-ID is unambiguously bound to the DetNet flow.

SB> Maybe we think of different things when you say local-id.
SB> We need to discuss.
SB> Is a local ID a control plane concept (out of scope) or is
SB> it equivalent to a PW of VPN label.

4.2.  Packet replication and elimination considerations

    DetNet service layer introduces packet replication and elimination
    functionality (PREF) for use in DetNet edge and relay node and end
    system packet processing.  PREF MAY be enabled in a DetNet node and
    the required processing is only applied to packets with a positive
    flow identification at the DetNet service layer.  PREF utilizes a
    sequence number carried within a DetNet flow packets.

SB> Here we go back to the fundamental question of whether PREF
SB> is a single function. I think it is three components and thus
SB> using the term PREF in this way add confusion.

    At a DetNet node level the output of the PREF elimination function is
    always a single packet.

SB> No it is not. Look at Fig 3 R4. Do you mean the PREF EF is only
SB> one packet?


==========

4.3.  Packet reordering considerations

    DetNet service layer introduces also packet reordering functionality
SB>d/also/
    for use in DetNet edge and relay node and end system packet
    processing.  The reordering functionality MAY be enabled in a DetNet
    node.  The reordeing functionality relies on a presence of sequence
    numbers in a DetNet (compound) flows.  The reordering processing is
    only applied to packets with a positive flow identification at the
    DetNet service layer.
SB> What is "a positive flow identification"

5.  DetNet encapsulation

5.1.  End-system specific considerations

    Data-flows requiring DetNet service are generated and terminated on
    end-systems.  Encapsulation depends on application and its
    preferences.  In a DetNet (or even a TSN) domain the DN (TSN)
    functions use at most two flow parameters, namely Flow-ID and
    Sequence Number.  However, an application may exchange further flow
    related parameters (e.g., time-stamp), which are not considered by DN
    functions.

SB> So  Greg, Yaakov and I had a discussion at the Paris MPLS
SB> mtg abou this. We think that we will likely need the concept
SB> of strict flow re-order, but also a time-based function
SB> in which we only re-order "young" packets. This lead us to
SB> considering the s/n to be a timesstamp in some implimentations.

=============


    As a general rule, DetNet domains MUST be capable to forward any
SB> s/capable to forward/capable of forwarding/
    Data-flows and the DetNet domain MUST NOT intend to mandate end-
SB> d/intend to/
    system encapsulation format.

==============

5.2.  DetNet domain specific considerations

    From connection type perspective three scenarios are distinguished:
SB> s/From/From a/

    1.  Directly attached: end-system is directly connected to an edge
        node

    2.  Indirectly attached: end-system is behind a (L2-TSN / L3-DetNet)
        sub-net

    3.  DN integrated: end-system is part of the DetNet domain

    L3 end-systems may use any of these connection types, however L2 end-
    systems may use only the first two (directly or indirectly attached).



Korhonen & Varga        Expires October 25, 2018               [Page 11]

Internet-Draft           DetNet MPLS Data Plane               April 2018


    DetNet domain MUST allow communication between any end-systems of the
    same type (L2-L2, L3-L3), independent of their connection type and
    DetNet capability.

SB> I think you ought to build L2 only, L3 only and mixed mode.
SB> this may be important in terms of the cost of the relay nodes.

    However directly attached and indirectly attached
    end-systems have no knowledge about the DetNet domain and its
    encapsulation format at all.  See the figure below for L3 end-system
    scenarios.

                                                ____+----+
                        +----+        _____    /    | ES3|
                        | ES1|____   /     \__/ +----+___
                        +----+    \ / \
+                          |
                           ____     \ _/
             +----+     __/    \     +__    DetNet domain /
             | ES2|____/  L2/L3 |___/   \         __     __/
             +----+    \_______/         \_______/  \___/



                Figure 5: Connection types of L3 end-systems

5.2.1.  DetNet Bridging Service

    The simplest DetNet service is to provide bridging (i.e., tunneling
    for L2), where the connected hosts are in the same broadcast (BC)
    domain.  Forwarding over the DetNet domain is based on L2 (MAC)
    addresses (i.e. dst-MAC), so L2 headers MUST be kept. For both IP
    and MPLS PSN a DetNet specific tunnel encapsulation MUST be
    introduced.

SB> There is a simpler model that may also be of interest the
SB> PW model where it is the interface that is used to select the
SB> destination.
==============
============

                                   +------+
                                   |  X   |
                          +------+ +------+
                          |  X   | |  IP  |
                          +------+ +------+
              End+system  |  L2  | |  L2  |
+-----+======+-+======+--+======+-+======++
              DetNet tunnel                  |  IP  | | MPLS |
                                             +------+ +------+
                                             |  L2  | |  L2 |
                                             +------+ +------+

              Examples

                                             +------+ +------+
                                             |  X   | |  X |
                          +------+ +------+  +------+ +------+
                          |  X   | |  X   |  |  IP  | |  IP |
                          +------+ +------+  +------+ +------+
                          |  L2  | |  L2  |  |  L2  | |  L2 |
++======+-+======+--+======+-+======++
                          |  IP  | | MPLS |  |  IP  | | MPLS |
                          +------+ +------+  +------+ +------+
                          |  L2  | |  L2  |  |  L2  | |  L2 |
                          +------+ +------+  +------+ +------+


             Figure 6: Encapsulation format for DetNet Bridging

SB> That is not quite right. We need the d-CW in the layering.
SB> This may be explicity called out of symbolically included
SB> as a shim.

===========

5.2.2.1.  MPLS PSN

    In case of an MPLS PSN at the ingress/egress (i.e., PE nodes of
    DetNet domain) the IP packets are encapsulated in MPLS. The data-
    flow IP header MUST be preserved as-is.








Korhonen & Varga        Expires October 25, 2018               [Page 13]

Internet-Draft           DetNet MPLS Data Plane               April 2018


                      +------+ +------+
                      |  X   |                         | X   |
                      +------+ +------+
           End+system |  IP  |                         | IP  |
                 -----+------+-------+======+--- --+======+--
           DetNet                    | MPLS |          | MPLS |
                                     +------+ +------+
                                     |  L2  |          | L2  |
                                     +------+ +------+

SB> This really does need the shim since the stack on the right
SB> may be confused with ordinary IP over MPLS

    Figure 7: Encapsulation format for DetNet Routing in MPLS PSN for L3
                                 end-systems

5.3.  DetNet Inter-Working Function (DN-IWF)

5.3.1.  Networks with multiple technology segments

    There are network scenarios, where the DetNet domain contains
    multiple technology segments (IP, MPLS) and all those segments are
    under the same administrative control (see Figure 8). Furthermore,
    DetNet nodes may be interconnected via TSN segments.

    An important aspect of DetNet network design is placement of DetNet
    functions across the domain.  Designs based on segment-by-segment
    optimization can provide only suboptimal solutions.  In order to
    achieve global optimum Inter-Working Functions (DN-IWF) can be placed
    at segment border nodes, which stich together DetNet flows across
    connected segments.

    DN-IWF may ensure that flow attributes are correlated across segment
    borders.  For example, there are two DetNet functions which require
    Sequence Numbers: (1) Elimination: removes duplications from flows
    and (2) IOD: ensures in-order-delivery of packet in a flow.
    Stitching flows together and correlating attributes means for example
    that replication of packets can happen in one segment and elimination
    of duplicates in a different one.

SB> Do we really need this? It will add a lot of complexity and
SB> in practice such IWFs are rearely implemented. If one
SB> is needed it is likely to end up being proprietary.

==============
                                    ______
                          _____    /      \__
             ____        /     \__/          \___    ______
+----+   __/    +======+                        +==+ \     +----+
|src |__/  Seg1  )     |                        |  \  Seg3 \____| dst|
+----+  \_______+      \        Segment-2       | \+_____/    +----+
                  \======+__                    _+===/
                            \         __     __/
                             \_______/  \___/


                                           +------------+
              +------------------E----+    |            |
+----+       |                  |    +----R---+ |          +----+
|src |-------R              +---+             | E----------+ dst|
+----+       |              | E--------+          +----+
              +--------------R                 |
                             +-----------------+


       Figure 8: Optimal replication and elimination placement across
                         technology segments example

SB> So here we have separated the R and the E functions.

==========

=========

SB> This ought to be the core of this  document with most of the earlier
SB> text in a data-plane arch draft.

6.  MPLS-based DetNet data plane solution

6.1.  DetNet over MPLS Encapsulation Components


========

======

6.3.  DetNet control word

    A DetNet control word (d-CW) conforms to the Generic PW MPLS Control
    Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 11.
    The upper nibble of the d-CW MUST be set to zero (0). The effective
    sequence number bit length is between 0 and 28 bits, and configured
    either by a control plane or manually for each DetNet flow.  The
    sequence number is aligned to the right (least significant bits) and
    unused bits MUST be set to zero (0).  Each DetNet flow MUST have its
    own sequence number counter.  The sequence number is incremented by
    one for each new packet.

    The d-CW MUST always be present in a packet.  In a case the sequence
    number is not used (e.g., for DetNet-t-flows) the control plane or
    the manual configuration has to define zero (0) bit length seqeunce
    number and the value of the sequence number MUST be set to zero (0).

    [Editor's'note: TSN compatibility to be ensured (16 bits vs. 28
    bits).]

SB> If we want to we can put the 16 bits in the bottom of the 28 bits
SB> and configure the DN to be 16 bits by configuration.

7.4.  Bidirectional traffic

    [Editor's NOTE: delete this section and treat bi-dir flows as two
    uni-dir ???]

SB> That is what PWs do, but then we found this problematic in MPLS-TP
SB> we need to discuss.


8.  Time synchronization

    [Editor's NOTE: delete this section (scope time synchronization
    solution outside data plane).]

    Comment #10  SB> This section should point the reader to RFC8169
       (residence time in MPLS n/w.  We need to consider if we need to
       introduce the same concept in IP.

    Discussion:  Agree.  For IP we could reference to PTPv2 or v3 over
       UDP/IP, since it measures residence time among other things.

SB> We need a voice discussion, perhaps at an interim.