Re: [Detnet] Last Call: <draft-ietf-detnet-yang-18.txt> (Deterministic Networking (DetNet) YANG Model) to Proposed Standard

[Florian Kauer <[e-mail]>]

Hi Don,
thanks for your response!
Below I removed everything that looks good and added my comments with [FK].

On 19.12.23 22:19, Don Fedyk wrote:
>       typedef ipsec-spi {
> @@ -1113,6 +1113,79 @@
>           description
>             "The Application flow type choices.";
>           container tsn-app-flow {
> +---
> +fk: In case of a tsn-app-flow, how does the actual encapsulation take place?
> 
> [Don] This is a YANG specification that satisfies the operations in DetNet RFCs  
> Listed in the references.  You can think of it as a configuration for an 
> SDN controller that specifies encapsulation/decapsulation on various nodes/devices
> along the data path. It is not specifying "how" it is specifying "what" aggregation,  
> disaggregation functions are applied along the data path and which technology is used.
> 
> Also reading through your comments I think there is mis understanding.  
> 
> DetNet is not an any to any encapsulation. It is specifically what is covered in the DetNet architecture to support DetNet encapsulation flows for aggregation or replication.  We have specific type of encapsulations and behaviors that must be specified on ingress. The reverse operations don’t need as much specification because you are removing an outer encapsulation and exposing another header. I will try to see if this point is spelled out in the ID. 

[FK]: Yes, I agree that the YANG specification document is not the correct place to specify how to implement the encapsulations in detail (for that we have for example RFC 9024 and others). The reason why I raised this in the context of the YANG model is that it makes it obvious how many different options of combining TSN/IP/MPLS/... there are. And there are options that are both reasonable and permitted by the YANG model, but in my opinion not unambiguously covered by any DetNet specification (including TSN over IP over TSN without MPLS).

Again, that is not a weakness of the YANG model, but I am just worried that if we come to an agreement about all combinations we might find something that needs to be added to the YANG model to properly cover that case.


>   
> +
> +For reference, lets look at a few cases and how I have understood them:
> +
> +1. ip-app-flow, nothing MPLS related in service or forwarding sub-layers:
> +   -> Just transmit unmodified packet with new L2 header.
> ++---------------+      +----------+
> +| Payload       |      | Payload  |
> ++---------------+      +----------+
> +|   UDP         |      |   UDP    |
> ++---------------+  ->  +----------+
> +|   IP          |      |   IP     |
> ++---------------+      +----------+
> +| L2 (original) |      | L2 (new) |
> ++---------------+      +----------+
> 
> [Don]  Is this is a DetNet Appflow?  Please see DetNet Architecture RFC 8655 Figure 3. Also Data Plane Framework RFC8938.
> We are only concerned with DetNet.
> 
> The drawing is a bit ambiguous for DetNet.  
> The Payload could be the TSN flow then the IP is a DetNet sub-network and you have a DetNet packet. The L2 is irrelevant.  
> If the whole stack Payload/UDP/IP is a TSN payload then TSN is the subnetwork layer. It would be a TSN application frame in ingress and the SAME frame on egress.    
> 
> The YANG model aggregates (encapsulates) and disaggregates (decapsulates) DetNet packet but it does not change/translate a TSN L2.  The encapsulation layers for DetNet are IP/MPLS not Ethernet L2.   (we don’t specify egress L2 headers we can match L2 ingress headers at the Application interface).  

[FK] Yes, sorry for the confusion! Of course, explicitly changing the L2 header not in the scope of DetNet (and thus the YANG model). I just added the L2 to indicate that the packet can move from one TSN domain to another, while everything L3 related in the packet stays constant.

In this example, the payload was meant to be the actual application payload (like MQTT) and not an encapsulated TSN L2 (that is covered by case 4).

>       
> 
> +
> +2. ip-app-flow, but MPLS label is added:
> +   -> Squeeze in MPLS header.
> ++---------------+      +----------+
> +| Payload       |      | Payload  |
> ++---------------+      +----------+
> +|   UDP         |      |   UDP    |
> ++---------------+  ->  +----------+
> +|   IP          |      |   IP     |
> ++---------------+      +----------+
> +| L2 (original) |      |   MPLS   |
> ++---------------+      +----------+
> +                       | L2 (new) |
> +                       +----------+
> [Don] Again if this is an IP App flow you have this. L2 Not relevant.
> 
> ++---------------+      +----------+
> +| Payload       |      | Payload  |
> ++---------------+      +----------+
> +|   UDP         |      |   UDP    |
> ++---------------+  ->  +----------+
> +|   IP          |      |   IP     |
> ++---------------+      +----------+
>                         |   MPLS   |
>                         +----------+

[FK] Yes, see my comment above regarding L2.
> 
> [Don] Or if it is TSN app flow you have this: 
> 
> ++---------------+      +----------+
> +| Payload       |      | Original |
> ++---------------+      | L2 Frame |
> +|   UDP         |      |          |
> ++---------------+  ->  |          |
> +|   IP          |      |          |
> ++---------------+      |          |
> +| L2 (original) |      |          |
> ++---------------+      +---------------------------+
> +                       | DetNet Service Sublayer|  |
> +                       +---------------------------+
> +                       | DetNet Forwarding Sublayer|  |
> +                       +---------------------------+


[FK] In my classification, TSN app flow is case 4. But anyway, the question I am having for case 4 is how what you denote as "DetNet Service Sublayer" and "DetNet Forwarding Sublayer" specifically looks like in the TSN app flow case (see below).
                        
>  
> 
> +
> +3. mpls-app-flow:
> +   -> Only optionally modify MPLS header
> ++-----------------+      +------------+
> +| Payload         |      | Payload    |
> ++-----------------+      +------------+
> +|   UDP           |      |   UDP      |
> ++-----------------+  ->  +------------+
> +|   IP            |      |   IP       |
> ++-----------------+      +------------+
> +| MPLS (original) |      | MPLS (new) |
> ++-----------------+      +------------+
> +| L2 (original)   |      | L2 (new)   |
> ++-----------------+      +------------+
> 
> 
> [Don] If it is an APP flow it gets encapsulated. We can encapsulate service sub-layer as application layer for example.  This header format could also be a service sub layer and we show examples of changing the labels as a serve sub layer and forwarding layer.   

[FK] Sorry that was depicted ambiguously since I merged all MPLS headers into one. I actually meant:

+-----------------+      +-----------------+
| Payload         |      | Payload         |
+-----------------+      +-----------------+
|   UDP           |      |   UDP           |
+-----------------+  ->  +-----------------+
|   IP            |      |   IP            |
+-----------------+      +-----------------+
| MPLS (original) |      | MPLS (original) |
+-----------------+      +-----------------+
                         | MPLS (new)      |
                         +-----------------+

Is that better this way or did I misinterpret your comment?


> +
> +4. tsn-app-flow:
> +   -> In order to be able to reconstruct the full original L2 packet, we need
> +      to retain the L2 header. So we need something like:
> +
> ++---------------+      +-----------------+
> +| Payload       |      |    Payload      |
> ++---------------+  ->  +-----------------+
> +| L2 (Original) |      |  L2 (Original)  |
> ++---------------+      +-----------------+
> +                       |      ???        |
> +                       +-----------------+
> +                       |   L2 (new)      |
> +                       +-----------------+
> 
> [Don] This is specified in IEEE TSN.    DetNet uses IP/MPLS as the service sub layer. 

[FK] ??? here is a placeholder for UDP/IP/MPLS headers, sorry that this was not clear.
In RFC 9024 Figure 3 the following two options are depicted (translated into the notation of my mail):

+---------------+      +-----------------+
| Payload       |      |    Payload      |
+---------------+  ->  +-----------------+
| L2 (Original) |      |  L2 (Original)  |
+---------------+      +-----------------+
                       |      MPLS       |
                       +-----------------+
                       |  L2 (new)       |
                       +-----------------+

+---------------+      +-----------------+
| Payload       |      |    Payload      |
+---------------+  ->  +-----------------+
| L2 (Original) |      |  L2 (Original)  |
+---------------+      +-----------------+
                       |      MPLS       |
                       +-----------------+
                       |      UDP        |
                       +-----------------+
                       |      IP         |
                       +-----------------+
                       |   L2 (new)      |
                       +-----------------+

(L2 can either be TSN or something else, therefore there are three options in RFC 9024 Figure 3.)

I would expect that at least these two cases could be differentiated via YANG somehow (either in the draft-ietf-detnet-yang or in another model). What do you think?

But there are certainly some more possibilities like:

+---------------+      +-----------------+
| Payload       |      |    Payload      |
+---------------+  ->  +-----------------+
| L2 (Original) |      |  L2 (Original)  |
+---------------+      +-----------------+
                       |      UDP        |
                       +-----------------+
                       |      IP         |
                       +-----------------+
                       |   L2 (new)      |
                       +-----------------+

But I have not seen that being specified anywhere yet, but it would be one option to realize the "TSN over IP over TSN without MPLS" case.


> +I know there are several different ways to realize this, including just 
> +transmitting the L2 packet over MPLS (apparently not IETF 
> +standardized?) or several others like L2TPv3 (RFC3931) or VXLAN (RFC 
> +7348) if no MPLS labels are to be added (like in case 1 above).
> 
> +
> +I would expect the YANG model (either this or a linked one) to somehow 
> +specify which of these options shall actually be applied. Otherwise, 
> +two DetNet routers might not be able to communicate with each other, 
> +because one might use L2TPv3 and the other one VXLAN. And also for 
> +example in the case of VXLAN, we would need the option to specify a VXLAN Network Identifier.
> 
> [Don] The scope of the YANG does what DetNet documents outline. There is no VXLAN or L2TP3 specified in the DetNet when this applied. 
> The DetNet YANG works by specified the whole path and applying the appropriate encapsulation format at the appropriate places. Yes the encapsulation specified on each device must match just the way SDN controllers specify packet forwarding in several technologies.

[FK] You have convinced me that it does not make sense to hold back the YANG model until someday someone might come along who finds a use case for VXLAN or L2TP3 in DetNet. And there is still the option to specify an extension to the YANG model in that particular spec if needed. 


>               uses data-flow-spec;
>               choice application-type {
>                 description
>                   "This is the application type choices.";
> +---
> +fk: This part is quite confusing to me, especially since it does not directly mirror the ingress since.
> +I guess one reason for that is there could be multiple interfaces that could be selected based on the packet headers?
> +But for the same reason you could also have multiple ingress interfaces.
> [Don]  For Egress the options are different because we get here by decapsulating the service sub-layer. We have either some type of TSN frame (Ethernet) or IP/MPLS but we are basically only associating the flow with an interface at the end of a service sub-layer.  

[FK] So the TSN case is clear to me because in both cases you have zero or one interface (app_flow->ingress->interface and app_flow->egress->ethernet->interface).

But in the case of IP or MPLS we have still zero or one interface for ingress (app_flow->ingress->interface), but since there can be multiple different next hops for A SINGLE app-flow,
we can have multiple interfaces for egress (app_flow->egress->ip-mpls->next_hop*->outgoing_interface).

That means the current model allows to split up A SINGLE app-flow like this:

                                                 +---------------+
                                            ---> | Destination 1 |
+----------+    +---------+    +--------+  /     +---------------+
| Source 1 | -> | Ingress | -> | Egress |--  
+----------+    +---------+    +--------+  \     +---------------+
                                            ---> | Destination 2 |
                                                 +---------------+

But not to merge one like this:

+----------+
| Source 1 | --
+----------+   \    +---------+    +--------+    +---------------+
                --> | Ingress | -> | Egress | -> | Destination 1 |
+----------+   /    +---------+    +--------+    +---------------+
| Source 2 | --
+----------+

So either we have exactly one source and one destination for an app flow. Then why do we have multiple next hops for egress?
Or we allow multiple sources and destinations per app flow, then we need to support multiple interfaces at ingress, too. Or do I overlook something?

>     /detnet/service/sub-layer/incoming/app-flow: This links applications
>     to services.
> @@ -2608,6 +2713,14 @@
>          |     +--rw operation?            operation
>          |     +--rw incoming
>          |     |  +--rw (incoming)?
> +---
> +fk: One thing I raised before, but it either my comment got lost or i missed the response:
> +Why is there no incoming forwarding sub-layer for the service sub-layer?
> +An app-flow has outgoing service-sub-layer and incoming 
> +service-sub-layer, a forwarding sub-layer has incoming 
> +service-sub-layer and outgoing service-sub-layer, a service sub-layer 
> +has incoming app-flow and outgoing app-flow and it has outgoing forwarding sub-layer, but NO incoming forwarding sub-layer.
> [Don] Sorry I missed your comment before.   
> The operation of Service-sub-layer to forwarding sub-layer is an association of an encapsulation operation. The reverse direction is simply a removal of the forwarding layer and a match in the now exposed service Sub-layer.  Therefore, you do not need the same level of description.

[FK] Yes, but what do I specify as service->sub-layer->incoming for an egress node? Just keep it empty? Especially if no MPLS is involved.

Greetings,
Florian