Re: [Detnet] comments on draft-ietf-raw-architecture-16

[Lou Berger <[e-mail]>]

Pascal, (Janos)

Thank you for the much improved versions.  I think (as contributor) the 
document will be in good shape once the address the comments below.

I think Janos' comments are very helpful and  together with the drawings 
will answer one of my major open questions - how does a raw domain work 
with a non-raw detnet domain to support end to end detnet services.

A few terminology items to add to the comments below:

WRT Recovery graphs, I thought we were going to reuse one of the 
existing terms (i.e., protection group or recovery group) see, 
https://mailarchive.ietf.org/arch/msg/detnet/xj6bBURD66eB0ANhjqMd3anZ5vA/

WRT Lane - how does a lane differ from a 'protection path', i.e. do we 
really need this new term? I know I've asked this before, but I  do not 
see the difference based on the latest text.

At the nit level uplink and downlink can be dropped as terms as they are 
only used in their definition sections.

Thanks again,

Lou

On 11/6/2023 2:21 AM, Janos Farkas wrote:

> Hi Pascal, all,
>
> I am very sorry for not being able to respond earlier.
>
> First of all, many thanks for the updates and significant changes in 
> the draft!
>
> The draft has been improved a lot!
>
> I’ve read v16. I think it is better if I place my thoughts on v16 
> directly here than embedding in-line in our dialogue. These thoughts 
> actually include responses too. In addition, I’ve added responses 
> below too.
>
> text from the draft are marked with <v16>
>
> my thoughts are marked with <JF>
>
> <JF>:
>
> A couple of things are still confusing in the draft imo, e.g., some 
> self-inconsistencies. I’ve been trying to wrap my head around how to 
> comment actually. Perhaps it might be good to make sure that there is 
> consensus on high level principles before diving into details.
>
> Perhaps first what we are doing, what is RAW actually. I think there 
> are very good statements on this already in the draft, e.g.:
>
> <v16>:
>
> RAW improves the reliability of transmissions and the availability
>
> of the communication resources, but does not provide scheduling and
>
> shaping, so RAW itself does not provide guarantees such as latency
>
> for the application payload. Rather, it should be seen as a dynamic
>
> optimization of the use of redundancy to maintain it within certain
>
> boundaries. For instance, ARQ
>
> is operated by the lower layers and
>
> RAW will only abstract the concept and hint the lower layers on the
>
> desired outcome, as opposed to performing the retries at Layer-3.
>
> <JF>: note that I’ve intentionally removed the part I do not like and 
> will come back to it.
>
> <v16>:
>
> using a common radio abstraction and
>
> APIs
>
>  the capability
>
> to push reliability and timing hints like suggest X retries (min,
>
> max) within a time window, or send unicast (one next hop) or
>
> multicast (for overhearing). The other way around RAW needs hints
>
> about the radio conditions like L2 triggers
>
> <JF>: again, a few words left out intentionally for a sec.
>
> I like the text above. I think they reflect what I think I heard we 
> want: a clen layered model (not an integrated one)
>
> I guess I would illustrate it something like:
>
>                    .
>
>                    .
>
>      +-----------------------------+
>
>      |                             |
>
>      |  DetNet Service sub-layer   | (e.g., (PREOF)
>
>      |                             |
>
>      +-----------------------------+
>
>      |                             |
>
>      | DetNet Forwarding sub-layer |
>
>      |         _________           |
>
>      +--------< RAW API >----------+
>
>      |         ---------           |
>
>      |    Wireless/Radio Layer     | (e.g., (H)ARQ)
>
>      |                             |
>
>      +-----------------------------+
>
>                    .
>
> However, the draft still contains the term PAREO. I’m afraid I need to 
> keep repeating that the term/concept PAREO is a layer violation 
> itself. I guess PAREO comes from an integrated model. The figure above 
> clearly illustrates why PAREO is a layer violation. PAREO combines 
> (H)ARQ and PREOF. However, (H)ARQ resides in the Wireless/Radio Layer 
> whereas, PREOF is part of the DetNet Service sub-layer. They are very 
> different layers, and there is the DetNet Forwarding sub-layer in 
> between them. Therefore, the term and concept PAREO should be removed 
> completely from the document.
>
> So, to the question whether I suggest changes to Figure 4: Yes, remove 
> PARO at the minimum. Nonetheless, there may be further potential 
> changes falling out of the train of thoughts below.
>
> I do not have a replacement term yet, but perhaps we might get 
> somewhat closer via the thoughts below.
>
> So, I think one essence of RAW is the API.
>
> I think the draft contains further good text about the API, e.g.:
>
> <v16>:
>
> Conceptually, RAW is
>
> agnostic to the radio layer underneath
>
> <v16>:
>
> hint on
>
> the desired outcome, and the lower layer acts on those hints to
>
> provide the best approximation of that outcome,
>
> <v16:>:
>
> abstract
>
> interface, while they are really operated at the lower layers
>
> <JF>: Nonetheless, the draft sometimes uses “controls”, “manipulates” 
> and alike from what is done from the DetNet layer towards the 
> Wireless/Radio Layer.
>
> The draft also (imo correctly) includes statements like:
>
> <v16>:
>
> it is not expected that all lower layers
>
> support all those capabilities
>
> <JF>:
>
> I think that the RAW architecture should be crafted such that it 
> covers at least the wireless technologies in 
> draft-ietf-raw-technologies. They are quire different; provide 
> different ways / levels of exposure. So, I rather like the language 
> around “hint” than “control”.
>
> For instance,
>
> “RAW manipulates abstractions of the lower layer services to hint on
>
> the desired outcome,”
>
> could be replaced with
>
> “RAW provides hints to the lower layer services on the desired outcome,”
>
> <JF>:
>
> Back to what is RAW:
>
> In my understanding an essence of RAW is the capability of fast 
> reaction to rapidly changing wireless conditions on order to maintain 
> the desired QoS for the end to end service, esp., reliability.
>
> (I know I’m simplifying a couple of aspects a bit, but this is with 
> the attempt to make it easier to come to some similar page.)
>
> To this extent, the concept of recovery graph has been introduced. The 
> term protection path is widely used too. Perhaps, at a high level, it 
> is fair to say that the recovery graph is the superset of all possible 
> protection paths. (again, there may be some simplifications here to be 
> able to express some thoughts below)
>
> I found very good statements in the draft:
>
> <v16>:
>
> RAW
>
> itself operates in the DetNet Network Plane at a faster time scale
>
> <v16>:
>
> take the routing decision early
>
> enough along the possible paths to route around the failure. RAW
>
> defines a end-to-end control loop that dynamically controls the
>
> activation and deactivation of the feasible Protection Paths
>
> <v16>:
>
> An Operational Plane PLR that hosts the Decision function of
>
> which DetNet Paths to use for the future packets that will be
>
> routed within the recovery graph
>
> <v16>:
>
> The main action by the PLR is the swapping of the DetNet Path within
>
> the recovery graph for the future packets. The candidate DetNet
>
> Paths represent different energy and spectrum profiles, and provide
>
> protection against different failures.
>
> <v16>:
>
> RAW defines the Path Selection Engine (PLR) as a
>
> synchronous CPF that performs rapid local adjustments of the
>
> forwarding tables within the diversity that the asynchronous CPF has
>
> in store for the recovery graph
>
> <v16>:
>
> the system observed by the RAW OAM is the
>
> recovery graph
>
> <JF>:
>
> In my understanding, the key component being added by RAW to the 
> DetNet architecture is the PLR to perform the fast reactions to the 
> fast changes in wireless/radio conditions.
>
> In my understanding, the establishment of the forwarding paths, i.e., 
> the Protection Paths and ultimately the recovery graph belongs to the 
> DetNet Forwarding sub-layer. In my read, this is illustrated, e.g., in 
> Figure 2 in RFC 8655. The explicit routes (source routing whatsoever) 
> belong to the DetNet Forwarding sub-layer.
>
> In my understanding, PLR reacts based on the information received from 
> lower layers via the RAW API and based on OAM observing the recovery 
> graph.
>
> Which DetNet sub-layer the PLR then belongs to?
>
> Maybe not that easy to answer.
>
> Placing it in the DetNet Service sub-layer results in an additional 
> layer in between the PLR and the API the PLR operates upon. So, it may 
> not be the best approach.
>
> Placing it in the DetNet Forwarding sub-layer has the benefit that PLR 
> is adjacent / directly connected to the API it operates upon.
>
> With that, perhaps a simplified view on what RAW is, i.e., what are 
> the RAW add-ons / extensions to the DetNet architecture might be 
> roughly viewed as:
>
>                    .
>
>                    .
>
>      +-----------------------------+
>
>      |                             |
>
>      |  DetNet Service sub-layer   | (e.g., (PREOF)
>
>      |                             |
>
>      +-----------------------------+
>
>      |                             |
>
>      | DetNet Forwarding sub-layer |
>
>      |                             |
>
>      |           _____    _____    |
>
>      |          < PLR >--< OAM >   |
>
>      |           -+--     -----    |
>
>      |            | ^              |
>
>      |            | |              |
>
>      |         ___v_+_____         |
>
>      +--------< RAW API >----------+
>
>      |         ---------           |
>
>      |    Wireless/Radio Layer     | (e.g., (H)ARQ)
>
>      |                             |
>
>      +-----------------------------+
>
>                    .
>
> So, I’d suggest more changes to Figure 4.
>
> As for where the OAM is based upon which the PLR acts: Well, I guess 
> this part of the OAM really monitors the forwarding paths, i.e., the 
> protection paths, the protection graph. So, I do not see it alien to 
> be located at the DetNet Forwarding sub-layer.
>
> But, it all depends on the consensus of the WG.
>
> Some further comments:
>
> I’m not sure on the use of some terminology, e.g., 
> synchronous/asynchronous CPF, distributed (e.g., PLR).
>
> Synchronous/asynchronous often refer to time synchronization. In my 
> understanding, what the PLR does is actually a local decision. So, it 
> is rather some local CPF making decision on Protection Path compared 
> to the main CPF crafting the protection graph.
>
> As for distributed, my understanding is that in case of distributed, 
> multiple entities have to act together to reach some goal, e.g., RSVP 
> and OSPF are distributed and multiple nodes have to play together.
>
> I’m not sure I get, e.g., what “the PLR decision may be distributed” 
> means. As I understand the point is that it is a local decision based 
> on the local situation, independent of what’s going on at some remote 
> location.
>
> I am not sure about the “loose” approach. My concern is that we do not 
> know the guarantees of the loose segments, if any. Perhaps it would be 
> better not going into “loose”.
>
> I’d suggest to use “extend” instead of “enhance”.
>
> Some detailed  comments on v16:
>
> Section 1:
>
> <v16>:
>
> Deterministic Networking is an attempt to emulate the properties of
>
> a serial link over a switched fabric, by providing a bounded latency
>
> and eliminating congestion loss, even when co-existing with best effort
>
> traffic.
>
> <JF>:
>
> I’d suggest:
>
> Deterministic Networking is to provide bounded latency and eliminate
>
> congestion loss, even when co-existing with best effort traffic.
>
> as I never had the emulation of serial link in mind.
>
> <v16>:
>
> Bringing determinism in a packet network means eliminating the
>
> statistical effects
>
> <JF>:
>
> I’m afraid complete elimination is not possible, so I’d suggest 
> something like:
>
> Bringing determinism in a packet network means minimizing the
>
> statistical effects
>
> <v16>:
>
> This innovation was initially introduced on wired networks, with
>
> IEEE 802.1 Time Sensitive networking (TSN) - for Ethernet LANs - and
>
> IETF DetNet.
>
> <JF>:
>
> RFC 8578 DetNet Use Cases includes wireless.
>
> <v16>:
>
> With new scheduled
>
> radios such as TSCH and OFDMA
>
> <JF>
>
> Suggest to remove “new” as they are not new.
>
> <v16>:
>
> To achieve this, RAW
>
> leverages
>
> <JF>
>
> As it depends on the actual wireless technology etc., suggest to 
> replace with:
>
> To achieve this, RAW
>
> can leverage
>
> <v16>:
>
> As opposed to routing trees, Distance-Vector protocols can enable
>
> more than one feasible successors along non-equal-cost multipath
>
> forwarding graphs. This provide redundancy and allow to dynamically
>
> adapt the forwarding operation to the state of the links. But this
>
> protection is limited since only a subset of the nodes along the
>
> path will have an alternate feasible successor
>
> <JF>
>
> I’m afraid this is confusing. A distance vector protocol may also 
> build up a tree, see, e.g., RSTP. Sticking to the RSTP example, it is 
> true that RSTP has the Alternate Port concept for fast reaction to a 
> change, however, the concept has been then introduced to routing 
> protocols as well, see LFA.
>
> Section 2:
>
> <v16>:
>
> In an quantic analogy, a recovery graph is to a path what an atomic
>
> orbital is to a planetary orbit, in that the electron has a
>
> probability of presence within a known shape as opposed to a
>
> deterministic trajectory.
>
> <JF>:
>
> Consider removing the paragraph as it may be unsure how much it is 
> helpful to the reader in the given context.
>
> 2.1.7:
>
> Remove PAREO completely from the document for the reasons explained above.
>
> 2.2.1:
>
> Is this really flapping? Isn’t what the text explain is rather just 
> degradation? Isn’t flapping rather a link going down and coming back?
>
> 2.2.4:
>
> Should "in the direction from the source towards the destination" be 
> added?
>
> 2.3.1:
>
> Replace
>
> a packet from A to B
>
> experiences 2 paths,
>
> with
>
> B, a packet from A to B
>
> can experience 2 paths,
>
> as one particular packet only experiences one path
>
> 2.3.2
>
> I’m not sure about the need of “usage metadata;”. Perhaps remove?
>
> Figure 1 is without any mention in the text. Perhaps add explanation.
>
> <v16>:
>
> The vertices of that graph are DetNet Relay nodes that operate at
>
> the DetNet Service sub-layer and provide the PAREO functions.
>
> The topological edges of the graph are strict sequences of DetNet
>
> Transit nodes that operate at the DetNet Forwarding sub-layer.
>
> These bullet may need some updates depending on how the discussion goes.
>
> 2.5.1
>
> <v16>:
>
> an SLA (service level agreement) is a
>
> contract between a provider, the network, and a client, the
>
> application flow, about measurable metrics such as latency
>
> boundaries, consecutive losses, and packet delivery ratio (PDR).
>
> <JF>:
>
> In my understanding, an SLA is rather a contract between two parties 
> (not more).
>
> Consider updating to:
>
> an SLA (service level agreement) is a
>
> contract between a network provider and a client about measurable 
> metrics of a flow such as latency boundaries, consecutive losses, and 
> packet delivery ratio (PDR).
>
> 2.5.6:
>
> “Because a wireless lane may not be good enough to
>
> provide the required reliability, and even 2 parallel lanes may not
>
> be over a longer period of time, the RAW availability implies a
>
> journey that is a lot more complex than following a serial path.”
>
> Could be removed as not needed for the definition.
>
> 3.1.1:
>
> <v16>:
>
> elimination of single points of failure
>
> <JF>:
>
> Perhaps rather: elimination of each single point of failure
>
> <v16>:
>
> prompt detection of failures as they occur.
>
> <JF>:
>
> Perhaps rather: prompt detection and reaction to failures as they occur.
>
> 3.1.1.1:
>
> <v16>:
>
> IP Routers leverage routing protocols to compute alternate routes
>
> <JF>:
>
> replace “compute” with "reroute to" as it is not just compute
>
> 3.1.1.2:
>
> <v16>:
>
> Having a backup equipment has a limited value unless it can be
>
> reliably switched into use within the down-time parameters.
>
> <JF>:
>
> Some schemes, e.g., PREOF, do not require switchover.
>
> <v16>:
>
> delayed reestablishment of the second path
>
> <JF>:
>
> There is no reestablishment of 2nd path in case of 1+1 protection; 
> esp. in case of PREOF, where the basic idea is to send multiple copies 
> of the same packet over pre-established maximally disjoint paths all 
> the time.
>
> 3.1.2:
>
> <v16>:
>
> In data
>
> networks, this is rarely the case. Packet loss cannot never be fully
>
> avoided and the systems are built to resist to one loss, e.g., using
>
> redundancy with Retries (HARQ) or Packet Replication and Elimination
>
> (PRE), or, in a typical control loop, by linear interpolation from
>
> the previous measurements.
>
> <JF>
>
> Packet loss cannot be fully avoided and the systems are built to 
> resist only a limited number of losses e.g., using retransmissions 
> (HARQ), or, in a typical control loop, by linear interpolation from 
> the previous measurements.
>
> Note that the RLC mechanism exists too.
>
> 3.2:
>
> <v16>:
>
> Conceptually, RAW is
>
> agnostic to the radio layer underneath though the capability to
>
> schedule transmissions is assumed.
>
> <JF>:
>
> I suggest to remove “though the capability to
>
> schedule transmissions is assumed”
>
> or replace with
>
> though it is assumed that the radio layer underneath is capable of 
> schedule transmissions
>
> <v16>:
>
> Nevertheless,
>
> cross-layer optimizations may take place to ensure proper
>
> <JF>
>
> As we follow a layered approach (not an integrated one), it may be 
> clearer to have :
>
> cross-layer optimization may take place via the APIs provided between 
> RAW and the radio layer.
>
> 4.1:
>
> Suggest to remove “such as a Wi-Fi extended service set (ESS) or a 5G 
> Core;”
>
> For instance ,the 5G core is part of the 5G System. When it comes to 
> 5G, a better example would be a 5G System connected to a wireline domain.
>
> 4.3
>
> The RAW Management sub-layer
>
> functionality includes the PLR
>
> RAW aims to be faster than usual control plane. Management plane is 
> slower than control plane. Management plane does not seem to be a good 
> place for PLR, which intends to be very fast.
>
> 5.1
>
> Suggest to remove “In the wired world,”
>
> There could be a wireless subnet/segment along some of these paths.
>
> *From:* Pascal Thubert (pthubert) <pthubert=[e-mail]>
> *Sent:* Friday, August 11, 2023 3:03 PM
> *To:* Janos Farkas <[e-mail]>
> *Cc:* [e-mail]; [e-mail]; Adrian Farrel 
> ([e-mail]) <[e-mail]>; Greg Mirsky 
> <[e-mail]>; Lou Berger ([e-mail]) <[e-mail]>
> *Subject:* RE: Terminology: draft-ietf-raw-architecture-14.txt and 
> what's still nuclear
>
> Hello Janos
>
> Many thanks for your review!
>
> >  I think RAW, i.e., wireless aspects actually rather reside in the 
> Forwarding sub-layer (not in the Service sub-layer).
>
> > The Service sub-layer is pretty much IP in this case. Ideally, it is 
> hidden from the Service sub-layer as much as possible what kind of 
> forwarding is used in the Forwarding > sub-layer, e.g., whether it is 
> wireline or wireless. (Well, the Forwarding sub-layer may provide some 
> certain APIs towards the Service sub-layer if we want to.)
>
> We defined the sublayers by listing functions as hints as opposed to 
> mapping to IP or L2. Our intention (Norm and I) at the start was that 
> the DetNet operated at L3 but deterministic networking at large could 
> be widely done at either L2 or L3 (or a mix) and the abstractions and 
> service, from a distance, would be the same.
>
> RAW being DetNet it is L3, so neither wireless nor wired, though it is 
> meant to address gaps that are more common in wireless like quick 
> flaps, reliability degradation, and PHY speed variations. Wireless, 
> like wire, is only observed and requested through APIs. As Lou said, 
> some aspects like promiscuous and ARQ are more commonly observed on 
> wireless, but are not exclusive to wireless. We provide a L3 
> architecture that is sensitive to those elements but does not include 
> them since they are operated in another layer altogether.
>
> The RAW architecture is focused on a fast protection/recovery loop. 
> This is why the main operation (PAREO actuation and OAM 
> processing/compiling ) is in the DetNet (=> L3) service sublayer. But 
> not only:
>
> “
>
>     +------------------------------+ +--------------------------------+
>
>     |                              | |                                |
>
> .....................................................................
>
>     |                              | |                                |
>
>     | +--------------------------+ | | +----------------------------+ |
>
>     | |     aCPF                 | | | |         aCPF               | |
>
>     | +--------------------------+ | | +----------------------------+ |
>
>     | +----------+  +------------+ | | .-.-.-.-.-.--.  .-.-.-.-.-.--. |
>
>     | | PLR      |  | OAM        | | | | Distr. PLR |  | Distr. OAM | |
>
>     | |          |  | Supervisor | | | |            |  | Supervisor | |
>
>     | +----------+  +------------+ | | .-.-.-.-.-.--.  .-.-.-.-.-.--. |
>
>     |                              | |    optional         optional   |
>
>        RAW Management sub-layer
>
> .....................................................................
>
>        DetNet Service sub-layer
>
>     |                              | |                                |
>
>     | +----------+  +------------+ | | +------------+  +------------+ |
>
>     | | PAREO    |  |  OAM       | | | |  PAREO     |  |  OAM       | |
>
>     | | Actuator |  |  Observer  | | | |  Actuator  |  |  Observer  | |
>
>     | +----------+  +------------+ | | +------------+  +------------+ |
>
>     |                              | |                                |
>
>        DetNet Service sub-layer
>
> .....................................................................
>
>        DetNet Forwarding sub-layer
>
>     |                              | |                                |
>
>     |               +------------+ | |                 +------------+ |
>
>     |               |In-Situ OAM | | |                 |In-Situ OAM | |
>
>     |               +------------+ | |                 +------------+ |
>
>     |                              | |                                |
>
>     +------------------------------+ +--------------------------------+
>
>             End System or                       Relay
>
>           Ingress Edge Node                     Node
>
>           Figure 4: RAW functional posture within DetNet sublayers
>
> “
>
> Is there a change you would suggest to this picture?
>
> <JF>
>
> Yes, see above.
>
> > As I read, the 1st paragraph in section 4.3 is actually along these 
> lines
>
> https://www.ietf.org/archive/id/draft-ietf-raw-architecture-14.html#name-raw-and-detnet
>
> > “RAW leverages the DetNet Forwarding sub-layer”
>
> > “RAW enhances DetNet to improve the protection against link errors 
> such as transient flapping that are far more common in wireless 
> links.” – Well, this reads to me to be in the Forwarding sub-layer.
>
> Well, protection is service sub-layer actually (section 2.1 of RFC 
> 8655). Forwarding is packet handling, like find which buffer and which 
> protection path for this packet.
>
> <JF>
>
> This is my point: “Forwarding is packet handling, like find which 
> protection path for this packet.”
>
> So, you suggest Forwarding sub-layer too. 😉
>
> The PAREO actuator impacts the forwarding operation for future packets 
> but does not necessarily run on every packet, see the discussion with 
> David.
>
> <JF>
>
> Imo, PAREO should be completely removed from the document because it 
> is a layer violation, as I keep saying for a while. See more above.
>
> > However, I’m quire unsure of the beginning of the 2nd paragraph in 
> section 4.3:
>
> > “RAW extends the DetNet Stack (see fig 4 of 
> [https://www.ietf.org/archive/id/draft-ietf-raw-architecture-14.html#RFC8655]) 
> with additional functionality at the DetNet Service sub-layer for the 
> PLR operation.”
>
> >  and the location of PLR in Figure 4, where the PLR is not in the 
> data plane.
>
> You’re correct about the inconsistency. The PLR is a piece of the 
> controller that was exported to the forwarding devices. The discussion 
> with Greg placed that component in the management sublayer. The text 
> that you point out is a victim of the renaming and the component it 
> discusses is now called PAREO actuator. Proposed beginning of that 
> section:
>
> “
>
>    RAW leverages the DetNet Forwarding sub-layer and requires the
>
>    support of in-situ OAM in DetNet Transit Nodes (see fig 3 of
>
>    [RFC8655] for the dynamic acquisition of link capacity and state to
>
>    maintain a strict RAW service, end-to-end, over a DetNet Network.
>
>    RAW enhances DetNet to improve the protection against link errors
>
>    such as transient flapping that are far more common in wireless
>
>    links.  Nevertheless, the RAW methods are for the most part
>
>    applicable to wired links as well, e.g., when energy savings are
>
>    desirable and the available path diversity exceeds 1+1 linear
>
>    redundancy.
>
>    RAW adds a Management sub-layer that operates in the DetNet
>
>    Operational Plane.  The RAW Management sub-layer typically runs only
>
>    in the DetNet Ingress Edge Node or End System, though it may also run
>
>    in DetNet Relay Nodes when the RAW Control sub-layer is distributed
>
>    along the recovery graph.  The RAW Management sub-layer functionality
>
>    includes the PLR that decides the DetNet Path for the future packets
>
>    of a flows and controls the PAREO Actuators along the DetNet Path
>
>    through specific signaling, and the OAM Supervisor that triggers, and
>
>    learns from, OAM observations, and feeds the PLR for its next
>
>    decision.
>
> RAW extends the DetNet Stack (see fig 4 of [RFC8655]) with additional
>
> functionality at the DetNet Service sub-layer for the actuation of
>
>    the PLR decision by the PAREO Actuator.Layer-3 in general and
>
>    DetNet in particular operates on abstractions of the lower layers and
>
>    through APIs to control those abstractions.  For instance, DetNet
>
>    already leverages lower layers for time-sensitive operations such as
>
>    time synchronization and traffic shapers.  Because the performances
>
>    of the radio layers are subject to rapid changes, so RAW needs more
>
>    dynamic gauges and knobs.  To that effect, the DetNet PREOF is
>
>    extended with the PAREO capabilities (see Section 5.6) and the RAW
>
>    PAREO Actuator manages dynamically the PAREO operations, which may be
>
>    performed either within the DetNet sublayers or at a lower layer,
>
>    using a common radio abstraction and APIs in the latter case.  In
>
>    particular, PAREO needs the capability to push reliability and timing
>
>    hints like suggest X retries (min, max) within a time window, or send
>
>    unicast (one next hop) or multicast (for overhearing).  The other way
>
>    around RAW needs hints about the radio conditions like L2 triggers
>
>    (RSSI, LQI, ETX...) over all the wireless hops.  This information is
>
>    useful to both the aCPF and the PLR.
>
> “
>
> > I think PLR should be located in the Forwarding sub-layer, but I may 
> have misunderstood the intention.
>
> I think the operation you have in mind is the realization of the PLR 
> decision. We did not create a box for it. Should we? What name? Is 
> there really something new there vs. DetNet?
>
> > As far as I understood, the intention with the PLR is fast reaction 
> to actual wireless situation, i.e., where to send a given packet actually.
>
> True, and it reacts to signals that arrive asynchronously to the 
> packets, for the most part. They are either coming from L2 or from OAM.
>
>                |
>
>         packet | going
>
>       down the | stack
>
> +==========v==========+=====================+=====================+
>
>     |   (iOAM + iCTRL)    | (L2 Triggers, DLEP) |       (oOAM)        |
>
> +==========v==========+=====================+=====================+
>
>     |     Learn from      |         |    Learn from       |
>
>     |    packet tagging   > Maintain      <    end-to-end       |
>
>     +----------v----------+ Forwarding     |    OAM packets      |
>
>     | Forwarding decision < State        +---------^-----------|
>
> +----------v----------+                     |      Enrich or      |
>
>     +    Retag Packet     |  Learn abstracted   >     Regenerate      |
>
>     |    and Forward      | metrics about Links |     OAM packets     |
>
> +..........v..........+..........^..........+.........^.v.........+
>
>     |                          Lower layers                           |
>
> +..........v.....................^....................^.v.........+
>
>          frame | sent          Frame | L2 Ack        oOAM | | packet
>
>           over | wireless        In |                 In | | and out
>
>                v |                    | v
>
>                          Figure 10: PLR Interfaces
>
> > Imo, this is very similar to FRR, see, e.g.: 
> https://datatracker.ietf.org/doc/html/rfc5286. A key aspect is that 
> the alternate next hops are precalculated. That is, it resides in the 
> data plane, no control plane reaction is needed.
>
> Hum FRR is still reroute. It installs new path in the forwarding plane 
> (think software operation downloading a new FIB) and then the 
> forwarding plane (think silicon now) uses the new paths possibly with 
> no idea that it is a FRR path. Same here.
>
> > I guess a more recent (segment routing) document on the subject with 
> PLR in it is: 
> https://datatracker.ietf.org/doc/html/draft-ietf-rtgwg-segment-routing-ti-lfa, 
> see, e.g.: “no need for any co-ordination or message exchange between 
> the PLR and any other router in the network.”
>
> Note that segment routing is part of the DetNet Forwarding sub-layer 
> (not the Service sub-layer).
>
> Like RAW, SR is a lot of things; the SRH operation (execution) is 
> indeed forwarding plane. The decision of which SRH to insert upon 
> which condition is control plane. Now we have this interesting 3D 
> problem of how our planes intersect those planes with similar names, 
> e.g., DetNet Controller plane and usual control plane.
>
> > Perhaps I’m the only one with these kind of thoughts.
>
> > If not, then perhaps it might be good to converge on PLR (e.g., its 
> location in the architecture) before figuring out what actual changes 
> are needed in the draft.
>
> Let’s see how others react. For now, you debunked at least one bug and 
> that already very valuable. Committed as af8771a 
> <https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-03fb2e6dd7e27dec&q=1&e=5ab1955b-c117-42bb-bcee-c4e507e491b9&u=https%3A%2F%2Fgithub.com%2Fraw-wg%2Fraw-architecture%2Fcommit%2Faf8771aeb4bc7dad5d2ae012cd53a4581e63f9df>
>
>
> <JF>
>
> Yes, see what the group thinks.
>
> Thank you,
>
> Cheers,
>
> Janos
>
> Many thanks!
>
> Pascal
>
> *From:* Janos Farkas <Janos.Farkas=[e-mail]>
> *Sent:* Monday, July 31, 2023 4:02 PM
> *To:* Pascal Thubert (pthubert) <[e-mail]>
> *Cc:* [e-mail]; [e-mail]; Adrian Farrel 
> ([e-mail]) <[e-mail]>; Greg Mirsky 
> <[e-mail]>; Lou Berger ([e-mail]) <[e-mail]>
> *Subject:* RE: Terminology: draft-ietf-raw-architecture-14.txt and 
> what's still nuclear
>
> Dear Pascal, all,
>
> Many thanks for the updates!
>
> Really great improvements towards architectural cleanliness.
>
> Otoh, distillation may reveal further questions and further 
> distillation. 😉
>
> For instance, now that it is called PLR, it triggers a couple of 
> questions on my part. (Well, with the assumption that I have some clue 
> on PLR.)
>
> Perhaps one step back first:
>
> We actually introduced the division of DetNet data plane to Forwarding 
> sub-layer and Service sub-layer for architectural cleanliness during 
> DetNet data plane design team discussions. Furthermore, in order to 
> divide (and conquer) in the sense to be able to identify the functions 
> and to place them to some order, i.e., for functional decomposition.
>
> I suppose, the RAW architecture would follow DetNet architectural 
> principles and place the RAW related functions accordingly.
>
> I think RAW, i.e., wireless aspects actually rather reside in the 
> Forwarding sub-layer (not in the Service sub-layer).
>
> The Service sub-layer is pretty much IP in this case. Ideally, it is 
> hidden from the Service sub-layer as much as possible what kind of 
> forwarding is used in the Forwarding sub-layer, e.g., whether it is 
> wireline or wireless. (Well, the Forwarding sub-layer may provide some 
> certain APIs towards the Service sub-layer if we want to.)
>
> As I read, the 1^st paragraph in section 4.3 is actually along these lines
>
> https://www.ietf.org/archive/id/draft-ietf-raw-architecture-14.html#name-raw-and-detnet 
>
>
> “RAW leverages the DetNet Forwarding sub-layer”
>
> “RAW enhances DetNet to improve the protection against link errors 
> such as transient flapping that are far more common in wireless 
> links.” – Well, this reads to me to be in the Forwarding sub-layer.
>
> However, I’m quire unsure of the beginning of the 2^nd paragraph in 
> section 4.3:
>
> “RAW extends the DetNet Stack (see fig 4 of [RFC8655 
> <https://www.ietf.org/archive/id/draft-ietf-raw-architecture-14.html#RFC8655>]) 
> with additional functionality at the DetNet Service sub-layer for the 
> PLR operation.”
>
> and the location of PLR in Figure 4, where the PLR is not in the data 
> plane.
>
> I think PLR should be located in the Forwarding sub-layer, but I may 
> have misunderstood the intention.
>
> As far as I understood, the intention with the PLR is fast reaction to 
> actual wireless situation, i.e., where to send a given packet actually.
>
> Imo, this is very similar to FRR, see, e.g.: 
> https://datatracker.ietf.org/doc/html/rfc5286. A key aspect is that 
> the alternate next hops are precalculated. That is, it resides in the 
> data plane, no control plane reaction is needed.
>
> I guess a more recent (segment routing) document on the subject with 
> PLR in it is: 
> https://datatracker.ietf.org/doc/html/draft-ietf-rtgwg-segment-routing-ti-lfa, 
> see, e.g.: “no need for any co-ordination or message exchange between 
> the PLR and any other router in the network.”
>
> Note that segment routing is part of the DetNet Forwarding sub-layer 
> (not the Service sub-layer).
>
> Perhaps I’m the only one with these kind of thoughts.
>
> If not, then perhaps it might be good to converge on PLR (e.g., its 
> location in the architecture) before figuring out what actual changes 
> are needed in the draft.
>
> My 2 cents,
>
> János
>
> ps:
>
> Sorry for late commenting. It was not the plan. And now we are hit by 
> August = vacation. I’m not sure how much I can chime in the discussion 
> from vacation. I just thought sharing the comment instead of waiting 
> till September.
>
> *From:* detnet <[e-mail]> *On Behalf Of *Pascal Thubert 
> (pthubert)
> *Sent:* Sunday, July 30, 2023 10:39 PM
> *To:* [e-mail]; Adrian Farrel ([e-mail]) 
> <[e-mail]>; Greg Mirsky <[e-mail]>; Lou Berger 
> ([e-mail]) <[e-mail]>
> *Cc:* [e-mail]
> *Subject:* [Detnet] Terminology: draft-ietf-raw-architecture-14.txt 
> and what's still nuclear
>
> Dear all:
>
> the publication of 14 adds terminologies and typos. The goal is to 
> serve as the new reference for the WGLC so we can use the new terms in 
> our discussions. If someone still uses PSE and Track, well, I guess 
> we'll still understand for a little while, but he will be harshly 
> reprimanded.
>
> What I did not do yet though I started is work out the positioning of 
> the aCPF (the component that talks asynchronously to the rCPF == PCE 
> to report performance and get route updates), the Point of Local 
> Repair (PLR is the term that replaces the PSE) and the OAM supervisor 
> that triggers OAM and aggregates results for the PLR.
>
> These are 3 new architectural blocks, and we want to position them 
> well in the DetNet architecture.
>
> The DetNet architecture (section 4.4) has 3 planes that are mapped to 
> classical SDN, with the controller plane in the middle, a southbound 
> interface to the network plane (in the case of RAW used between rCPF 
> and aCPF) and a northbound interface to the Application Plane.
>
> The Controller plane has the typical route servers like PCEs, and 
> network management entities. In the SDN model they are "far away" and 
> monitor the whole network. Which is what causing the RAW issue of lack 
> of reactivity and pushed us to port functionality in the network 
> plane. In networking planes parlance, the PCE is control plane and the 
> NMEs are management plane.
>
> As we see, though the term controller plane looks like control plane, 
> they are different beasts. A classical IGP is a control plane function 
> that operates in the DetNet network plane. The controller plane hosts 
> controllers, which physically may embed routing and management 
> entities. In the DetNet architecture, "The Controller Plane 
> corresponds to the aggregation of the Control and Management Planes in 
> [RFC7426 <https://datatracker.ietf.org/doc/html/rfc7426>], though 
> Common Control and Measurement Plane (CCAMP) (as defined by the CCAMP 
> Working Group [CCAMP 
> <https://datatracker.ietf.org/wg/ccamp/charter/>]) makes an additional 
> distinction between management and measurement."
>
> In my book, the OAM supervisor, the aCPF and the PLR operate in the 
> control plane. The OAM supervisor talks to the OAM handlers in the 
> management plane. And all of the above being distributed in the 
> network, they operate in the DetNet Network plane.  So 1) we extend 
> the DetNet architecture to place functional blocks in the Network 
> Plane and 2) one could say we need 3D pictures to represent the 
> intersection of the DetNet planes and the traditional control and 
> management planes. While the data plane remains firmly in the Network 
> plane.
>
> Now this is my view and the way I intend to update the text to publish 
> 15, hopefully quite soon. But I need confirmation that we are on the 
> same line, or else debates to reach a consensus.
>
> What do you all think?
>
> Pascal
>
> ------------------------------------------------------------------------
>
> *De :*[e-mail] <[e-mail]>
> *Envoyé :* samedi 29 juillet 2023 15:40
> *À :* Pascal Thubert (pthubert) <[e-mail]>
> *Objet :* New Version Notification for draft-ietf-raw-architecture-14.txt
>
>
> A new version of I-D, draft-ietf-raw-architecture-14.txt
> has been successfully submitted by Pascal Thubert and posted to the
> IETF repository.
>
> Name:           draft-ietf-raw-architecture
> Revision:       14
> Title:          Reliable and Available Wireless Architecture
> Document date:  2023-07-29
> Group:          raw
> Pages:          43
> URL: https://www.ietf.org/archive/id/draft-ietf-raw-architecture-14.txt
> Status: https://datatracker.ietf.org/doc/draft-ietf-raw-architecture/
> Html: https://www.ietf.org/archive/id/draft-ietf-raw-architecture-14.html
> Htmlized: 
> https://datatracker.ietf.org/doc/html/draft-ietf-raw-architecture
> Diff: 
> https://author-tools.ietf.org/iddiff?url2=draft-ietf-raw-architecture-14
>
> Abstract:
>    Reliable and Available Wireless (RAW) provides for high reliability
>    and availability for IP connectivity across any combination of wired
>    and wireless network segments.  The RAW Architecture extends the
>    DetNet Architecture and other standard IETF concepts and mechanisms
>    to adapt to the specific challenges of the wireless medium, in
>    particular intermittently lossy connectivity.  This document defines
>    a network control loop that optimizes the use of constrained spectrum
>    and energy while maintaining the expected connectivity properties,
>    typically reliability and latency.  The loop involves OAM, DetNet
>    Controller Plane, and PREOF extensions, and specifically a new
>    recovery Function called PAREO and a new Point of Local Repair
>    operation, that dynamically selects the DetNet path(s) for the future
>    packets to route around local degradations and failures.
>
>
>
>
> The IETF Secretariat
>
>
> _______________________________________________
> detnet mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/detnet