Re: [Detnet] Clarifications on RAW architecture

["Pascal Thubert (pthubert)" <[e-mail]>]

Dear Balázs, Lou, and Janos;

I made a pass to address some of the comments you made in mails before IETF   and during IETF.
The proposed changes are readable here: https://github.com/pthubert/raw_architecture/blob/master/raw-architecture.txt
And the commit is this; https://github.com/pthubert/raw_architecture/commit/cc668afa1bb527391e1051d7ae0734ab47493e27

For the sake of the ML, the main changes are below, addressing the case of non-centralized, and the fact that PAREO is not entirely done in the service layer (just like other things in DetNet).

Please let me know; comments in line in the xml in github, or a pull request, are both welcome.

Enjoy summer!

Pascal

In  Abstract:


    document defines an architecture element for the RAW data plane, in
    the form of an OODA loop, that optimizes the use of constrained
    spectrum and energy while maintaining the expected connectivity
-   properties.  The loop involves OAM, PCE, and PREOF extensions, and a
-   new component called the Path Selection Engine (PSE).
+   properties.  It also introduces a new Control plane Function to
+   prepare alternate paths to go around local failures.  The loop
+   involves OAM, PCE, and PREOF extensions, and a new component called
+   the Path Selection Engine (PSE).

…

    The establishment of a path is not in-scope for RAW.  It may be the
-   product of a centralized Controller Plane as described for DetNet.
+   product of a centralized Controller Plane Function like a Path
+   computation Element (PCE) [RFC4655] or a distributed routing
+   protocol.  For the most part, the remainder of the document mentions
+   centralized control and PCE, but conceptually, the same issues and
+   needs would arise for a distributed protocol that would attempt to
+   allocate constrained resources and optimize globally, and the
+   distributed approach is considered in scope too.
+
    As opposed to wired networks, the action of installing a path over a
…
        along a Track as well as the end-to-end packet delivery

-   2.  Optional Controller plane elements to reports the links
-       statistics to a Path computation Element (PCE) in a centralized
-       controller that computes and installs the Tracks and provides
-       meta data to Orient the routing decision
+   2.  Optional Controller plane elements that report the links
+       statistics to be used to compute and install the Tracks, and
+       provides meta data to Orient the routing decision, e.g., by a PCE
+       in a centralized controller



…


    back.  Reaching to the PCE can also be slow in regards to the speed
    of events that affect the forwarding operation at the radio layer.
+   In the same fashion, a distributed routing protocol may also take
+   time and consume excessive wireless resources to reconverge to a new
+   optimized state.




-   A RAW Network Plane may be strict or loose, depending on whether RAW
-   observes and takes actions on all hops or not.  For instance, the
-   packets between two wireless entities may be relayed over a wired
+   A RAW Network Plane may be strict (as illustrated in Figure 5 or
+   loose (as illustrated in Figure 6, depending on whether RAW observes
+   and takes actions on all hops or not.  For instance, the packets
+   between two wireless entities may be relayed over a wired
    infrastructure such as a Wi-Fi extended service set (ESS) or a 5G
    Core; in that case, RAW observes and controls the transmission over
    TSN, or neglected in the face of the wireless hops.

…

-   A Controller Plane Function (CPF) called the Path Computation Element
-   (PCE) [RFC4655] interacts with RAW Nodes over a Southbound API.  The
-   RAW Nodes are DetNet relays that are capable of additional diversity
-   mechanisms and measurement functions related to the radio interface,
-   in particular the PAREO diversity mechanisms.  RAW leverages a CPF
-   that operates inside the RAW Nodes (typically the Ingress Edge Nodes)
-   to dynamically adapt the path of the packets and optimizes the
-   resource usage.
+   A Controller Plane Function (CPF) such as a PCE interacts with RAW
+   Nodes over a Southbound API.  The RAW Nodes are DetNet relays that
+   are capable of additional diversity mechanisms and measurement
+   functions related to the radio interface, in particular the PAREO
+   diversity mechanisms.  RAW leverages a CPF that operates inside the
+   RAW Nodes (typically the Ingress Edge Nodes) to dynamically adapt the
+   path of the packets and optimizes the resource usage.
+
+

-   The PCE defines a complex Track between an Ingress End System and an
-   Egress End System, and indicates to the RAW Nodes where the PAREO
-   operations may be actioned in the Network Plane.  The Track may be
-   expressed loosely to enable traversing a non-RAW subnetwork.  In that
-   case, the expectation is that the non-RAW subnetwork can be neglected
-   in the RAW computation, that is, considered infinitely fast, reliable
-   and/or available in comparison with the links between RAW nodes.



+   The PCE defines a complex Track between an Ingress End System and an
+   Egress End System, and indicates to the RAW Nodes where the PAREO
+   operations may be actioned in the Network Plane.  The Track may be
+   strict, meaning that the DetNet forwarding sublayer operations are
+   enforced end-to-end The Track may be expressed loosely to enable
+   traversing a non-RAW subnetwork as in Figure 6.  In that case, RAW
+   can not leverage end-to-end DetNet and cannot provide latency
+   guarantees.  The non-RAW subnetwork is neglected in the RAW
+   computation, that is, considered jitterless, and infinitely reliable
+   and/or available in comparison with the links between RAW nodes, so
+   loss and jitter that is measured end-to-end is attributed to the RAW
+   hops (typically an access link).



In 3.3.  RAW vs. DetNet
-
-   RAW leverages the DetNet Forwarding sub-layer and requires the
-   support of in-situ OAM in DetNet Transit Nodes (see fig 3 of
-   [RFC8655] for the dynamic acquisition of link capacity and state to
-   maintain a strict RAW service, end-to-end, over a DetNet Network.
-
-   RAW extends the DetNet Stack (see fig 4 of [RFC8655]) with additional
-   functionality at the DetNet Service sub-layer for the PSE operation.
-   PREOF is extended with the PAREO capabilities (see Section 4.4) and
-   the RAW PAREO Actuator manages dynamically the PAREO operations.  The
-   RAW Service sub-layer also adds the OAM Propagator that (re)generates
-   the OAM information as it is formed and propagated In-Band or Out-of-
-   Band.  The RAW Service sub-layer may be present in DetNet Edge and
-   Relay Nodes, though the PAREO Actuator has no operation in the Egress
-   Edge Node.

+   RAW leverages the DetNet Forwarding sub-layer and requires the
+   support of in-situ OAM in DetNet Transit Nodes (see fig 3 of
+   [RFC8655] for the dynamic acquisition of link capacity and state to
+   maintain a strict RAW service, end-to-end, over a DetNet Network.

+   RAW extends the DetNet Stack (see fig 4 of [RFC8655]) with additional
+   functionality at the DetNet Service sub-layer for the PSE operation.
+   Layer-3 in general and DetNet in particular operates on abstractions
+   of the lower layers and through APIs to control those abstractions.
+   For instance, DetNet already leverages lower layers for time-
+   sensitive operations such as time synchronization and traffic
+   shapers.  Because the performances of the radio layers are subject to
+   rapid changes, so RAW needs more dynamic gauges and knobs.  To that
+   effect, the DetNet PREOF is extended with the PAREO capabilities (see
+   Section 4.4) and the RAW PAREO Actuator manages dynamically the PAREO
+   operations, which may be performed either within the Detnet sublayers
+   or at a lower layer, using a common radio abstraction and APIs in the
+   latter case.  In particular, PAREO needs the capability to push
+   reliability and timing hints like suggest X retries (min, max) within
+   a time window, or send unicast (one next hop) or multicast (for
+   overhearing).  The other way around RAW needs hints about the radio
+   conditions like L2 triggers (RSSI, LQI, ETX...) over all the wireless
+   hops.  This information is useful in the controller plane for both
+   the PCE and PSE.
+
+   The RAW Service sub-layer also adds the OAM Propagator that
+   (re)generates the OAM information as it is formed and propagated In-
+   Band or Out-of-Band.  The RAW Service sub-layer may be present in
+   DetNet Edge and Relay Nodes, though the PAREO Actuator has no
+   operation in the Egress Edge Node.



+   A minimal Forwarding sub-layer service is provided at all DetNet
+   Nodes to ensure that the OAM information flows.  Relay Nodes may or
+   may not support RAW services, and the Edge nodes do support RAW.
+   DetNet guarantees such as latency are provided end-to-end, and RAW
+   supports the DetNet Service to optimize the use of resources.
+



-Thubert & Papadopoulos   Expires 23 January 2023               [Page 23]
+

-   A minimal Forwarding sub-layer service is provided at all DetNet
-   Nodes to ensure that the OAM information flows.  Relay Nodes may or
-   may not support RAW services, and the Edge nodes do support RAW.
-   DetNet guarantees such as latency are provided end-to-end, and RAW
-   supports the DetNet Service to optimize the use of resources.



    2.  Controller plane elements to report the links statistics to a
-       Path computation Element (PCE) in a centralized controller that
-       computes and installs the Tracks and provides meta data to Orient
-       the routing decision, more in Section 4.2;
+       distributed or centralized control function such as a Path
+       Computation Element (PCE), that computes and installs the Tracks,
+       and provides meta data to Orient the routing decision, more in
+       Section 4.2;


From: Pascal Thubert (pthubert)
Sent: mardi 26 juillet 2022 14:02
To: 'Balázs Varga A' <[e-mail]>; [e-mail]
Cc: [e-mail]; [e-mail]
Subject: RE: Clarifications on RAW architecture

Hello Balázs

Great points, many thanks for all this. I added slides for your questions, I hope we’ll get discussion time on them. I’m adding below the response I make in the slides so we all can prepare for the discussion.


Q1, Mix of various OSI Layer functions

The definition of PAREO seems to be very confusing, as it contains a mix of Radio specific and DetNet specific functions. It is confusing as the referred functions work at different layers (e.g., HARQ is part of Radio at L1/L2 vs. PREOF is part of DetNet at L3) and have different "range" (radio acts on radio links vs. PREOF acts across several hops, maybe even end2end). This mix makes it unclear which OSI layer the RAW architecture belongs to. Could you please clarify?

Pascal > DetNet leverages lower layers, and RAW will augment that usage to hint about transmission suggestions. Lower Layers do what they like but if the API allows to pass hints, we’ll leverage that. In particular, we’ll need reliability and timing hints like suggest X retries (min, max), send unicast (one next hop) or multicast (overhearing). The other way around RAW will need hints about L2 conditions like L2 triggers (RSSI, LQI, ETX…) over all the wireless hops. This will be used by both PCE and PSE. Bottom line: to do its job, L3 works on abstractions of L2; in the (dynamic) case of wireless there’s more of it.


Q2, Also, the modeling of Radio components from deterministic networking perspective seems to be unclear and different from current work of radio related SDOs. The draft states that the concept is agnostic to the radio technology and agnostic to whether or not radio mesh is applied. Nonetheless, the model applied for the Radio layer seems to be unclear. Please note, e.g., that DetNet Study Item ongoing in 3GPP SA2 models the 5G System as a DetNet router.

Pascal > The case where the 5G network shows as one virtual switch is opaque to RAW as it is opaque to DetNet. I agree we can improve the discussion on interaction with lower layers in the dependency section following your suggestion. I hope we have more details on the mike…


Q3, Relationship of RAW and DetNet Layers

Related to the Q1, the relationship of RAW and DetNet Layers is unclear. Along the lines of your definition in "Section 3. The RAW Conceptual Model: ... The RAW Nodes are DetNet relays that are capable of additional diversity mechanisms and measurement functions related to the radio interface ..." whereas same section states that " ... the non-RAW subnetwork can be neglected in the RAW computation ...".


Pascal > I guess we’ll need to clarify. The non-RAW is when RAW is not end to end and latency cannot be guaranteed (loose RAW). Again, I hope there’s discussion on the mike.




Based on former discussions my interpretation was that, RAW is a wireless specific add-on function to DetNet. DetNet flows require resource allocation before forwarding. The "Track" is calculated by the PCE with considering the characteristics of wireless links and the "Track" is capable of providing the SLA even for worst case scenarios. The "Track" may include wireless segments e.g., between NodeR1-NodeE1, where multiple subTracks may exists (T11, T12). The task of RAW is to select "on Forwarding time scale", which sub-Track to use. The RAW functions monitor the subTracks using OAM.



Pascal > We’re in sync 😊

                     T11
               +*****>**********
               *                \         +---->-------+
               *  +***>*********E1->-+    |            |
    +---+      \ /    T12       |    +---R3->-+        |          +---+
    |src|--->--R1           +->-+             |        E3->--O----+dst|
    +---+       |           |                 E2-->----+          +---+
                +---->------R2                |
                            +----->-----------+

    R: replication function (PRF)
    E: elimination function (PEF)
    O: ordering function (POF)
   --: wired link
   **: wireless link


Pascal > Nice example when RAW plays at the access. Note that if one access is 5G and one is Wi-Fi, the elimination point might be much later, maybe at the destination. This is what we mean  by the ” non-RAW subnetwork can be neglected in the RAW computation “, RAW may not have visibility and control, so it cannot provide guarantees and will bill the access hop for any disturbance till the destination.

This serves the same use case as the below (already in the architecture). I see more complement than overlap, happy to include your drawing as will, indicating the case of a redundant WLAN or 5G access with a common egress. The drawing below is more generic.


                                         ...   ..
                      RAN 1  -----  ...      ..  ...
                   /              .    ..          ....
      +-------+  /              .            ..      ....    +------+
      |Ingress|-                .                     .....  |Egress|
      |  End  |------ RAN 2 -- .       Internet       ....---| End  |
      |System |-                ..                   .....   |System|
      +-------+  \               .               ......      +------+
                  \               ...   ...     .....
                      RAN n  --------  ...   .....

             <------------------> <-------------------->
                Observed by OAM       Opaque to OAM

            Figure 8: Observed Links in Radio Access Protection






Would You clarify? You may intend to add a Figure like below (or its modification or a different one) for clarification of the relationship to DetNet sub-layers. RAW is a shim layer between DetNet Service and Forwarding sub-layers.





     PCE                   DetNet Service

      ^                    sub-layer

      |                        +-----------------+

      |                        |+-----+   +-----+|

      |        +--------------->| OAM |   |PREOF||

      |        |               |+-----+   +-+-+-+|

      |        |               +------------|-*--+

RAW   v        |                            | *

+--------------v----------------------------|-*--+

|          +-------+                      +-+-+-+|

|          |Observe|                      | PSE ||

|          +-------+                      +--+--+|

+-------------^-^----------------------------|---+

              | |                            |

              | |              +-------------|---+

              | |              |+-----+   +--+--+|

              | +-------------->| OAM |   | TE  ||

              |                |+-----+   +--+--+|

              |                +-------------|---+

              |            DetNet Forwarding |

              |            sub-layer         |

              |                              |

              |                +-------------|---+

              |                |+-----+   +--+--+|

              +---------------->|State|   |RLink||

                               |+-----+   +--+--+|

                               +-------------|---+

                           Radio L1/L2       |

                                             v



Pascal > I think I see what you’re trying to do. It’s hard with just one picture.
To illustrate this we already have 2; one is the PSE interfaces as below:



               |

        packet | going

      down the | stack

    +==========v==========+=====================+=====================+

    |   (iOAM + iCTRL)    | (L2 Triggers, DLEP) |       (oOAM)        |

    +==========v==========+=====================+=====================+

    |     Learn from                                 Learn from       |

    |    packet tagging           Maintain           end-to-end       |

    +----------v----------+      Forwarding          OAM packets      |

    | Forwarding decision <        State        +---------^-----------|

    +----------v----------+                     |      Enrich or      |

    +    Retag Packet     |  Learn abstracted   >     Regenerate      |

    |    and Forward      | metrics about Links |     OAM packets     |

    +..........v..........+..........^..........+.........^.v.........+

    |                          Lower layers                           |

    +..........v.....................^....................^.v.........+

         frame | sent          Frame | L2 Ack        oOAM | | packet

          over | wireless        In  |                 In | | and out

               v                     |                    | v



                               Figure 9: PSE


And the other is the inclusion to DetNet sublayers (or not yet but soon or the controller plane function) :


    +------------------------------+ +--------------------------------+

    |                              | |                                |

   .....................................................................

    |                              | |                                |

    | +----------+  +------------+ | | .-.-.-.-.-.--.  .-.-.-.-.-.--. |

    | | PSE      |  | OAM        | | | | Distr. PSE |  | Distr. OAM | |

    | |          |  | Supervisor | | | |            |  | Supervisor | |

    | +----------+  +------------+ | | .-.-.-.-.-.--.  .-.-.-.-.-.--. |

    |                              | |    optional         optional   |

       RAW Control sub-layer

   .....................................................................

       DetNet Service sub-layer

    |                              | |                                |

    | +----------+  +------------+ | | +------------+  +------------+ |

    | | PAREO    |  |  OAM       | | | |  PAREO     |  |  OAM       | |

    | | Actuator |  |  Observer  | | | |  Actuator  |  |  Observer  | |

    | +----------+  +------------+ | | +------------+  +------------+ |

    |                              | |                                |

       DetNet Service sub-layer

   .....................................................................

       DetNet Forwarding sub-layer

    |                              | |                                |

    |               +------------+ | |                 +------------+ |

    |               |In-Situ OAM | | |                 |In-Situ OAM | |

    |               +------------+ | |                 +------------+ |

    |                              | |                                |

    +------------------------------+ +--------------------------------+



            End System or                       Relay

          Ingress Edge Node                     Node



                       Figure 4: RAW DetNet Services

Would there be a way to improve the existing and carry your ideas?

Many thanks again,


From: Balázs Varga A <[e-mail]<mailto:[e-mail]>>
Sent: lundi 25 juillet 2022 13:20
To: [e-mail]<mailto:[e-mail]>
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Clarifications on RAW architecture


Hi Authors,



the current version of the RAW architecture draft contains many interesting ideas/solutions. It also provides a great summary about the challenges of deterministic forwarding in radio networks. However, some conceptual basics may need some clarifications:



Q1, Mix of various OSI Layer functions

The definition of PAREO seems to be very confusing, as it contains a mix of Radio specific and DetNet specific functions. It is confusing as the referred functions work at different layers (e.g., HARQ is part of Radio at L1/L2 vs. PREOF is part of DetNet at L3) and have different "range" (radio acts on radio links vs. PREOF acts across several hops, maybe even end2end). This mix makes it unclear which OSI layer the RAW architecture belongs to. Could you please clarify?



Q2, Also, the modeling of Radio components from deterministic networking perspective seems to be unclear and different from current work of radio related SDOs. The draft states that the concept is agnostic to the radio technology and agnostic to whether or not radio mesh is applied. Nonetheless, the model applied for the Radio layer seems to be unclear. Please note, e.g., that DetNet Study Item ongoing in 3GPP SA2 models the 5G System as a DetNet router.



Q3, Relationship of RAW and DetNet Layers

Related to the Q1, the relationship of RAW and DetNet Layers is unclear. Along the lines of your definition in "Section 3. The RAW Conceptual Model: ... The RAW Nodes are DetNet relays that are capable of additional diversity mechanisms and measurement functions related to the radio interface ..." whereas same section states that " ... the non-RAW subnetwork can be neglected in the RAW computation ...".





Based on former discussions my interpretation was that, RAW is a wireless specific add-on function to DetNet. DetNet flows require resource allocation before forwarding. The "Track" is calculated by the PCE with considering the characteristics of wireless links and the "Track" is capable of providing the SLA even for worst case scenarios. The "Track" may include wireless segments e.g., between NodeR1-NodeE1, where multiple subTracks may exists (T11, T12). The task of RAW is to select "on Forwarding time scale", which sub-Track to use. The RAW functions monitor the subTracks using OAM.


                     T11
               +*****>**********
               *                \         +---->-------+
               *  +***>*********E1->-+    |            |
    +---+      \ /    T12       |    +---R3->-+        |          +---+
    |src|--->--R1           +->-+             |        E3->--O----+dst|
    +---+       |           |                 E2-->----+          +---+
                +---->------R2                |
                            +----->-----------+

    R: replication function (PRF)
    E: elimination function (PEF)
    O: ordering function (POF)
   --: wired link
   **: wireless link



Would You clarify? You may intend to add a Figure like below (or its modification or a different one) for clarification of the relationship to DetNet sub-layers. RAW is a shim layer between DetNet Service and Forwarding sub-layers.





     PCE                   DetNet Service

      ^                    sub-layer

      |                        +-----------------+

      |                        |+-----+   +-----+|

      |        +--------------->| OAM |   |PREOF||

      |        |               |+-----+   +-+-+-+|

      |        |               +------------|-*--+

RAW   v        |                            | *

+--------------v----------------------------|-*--+

|          +-------+                      +-+-+-+|

|          |Observe|                      | PSE ||

|          +-------+                      +--+--+|

+-------------^-^----------------------------|---+

              | |                            |

              | |              +-------------|---+

              | |              |+-----+   +--+--+|

              | +-------------->| OAM |   | TE  ||

              |                |+-----+   +--+--+|

              |                +-------------|---+

              |            DetNet Forwarding |

              |            sub-layer         |

              |                              |

              |                +-------------|---+

              |                |+-----+   +--+--+|

              +---------------->|State|   |RLink||

                               |+-----+   +--+--+|

                               +-------------|---+

                           Radio L1/L2       |

                                             v



I think these clarifications would improve the document.



Cheers

Bala'zs