Re: [Detnet] FW: I-D Action: draft-km-detnet-for-ocn-01.txt
Kiran Makhijani <kiran.ietf@gmail.com> Thu, 29 June 2023 12:05 UTC
Return-Path: <kiran.ietf@gmail.com>
X-Original-To: detnet@ietfa.amsl.com
Delivered-To: detnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 51E24C151084; Thu, 29 Jun 2023 05:05:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dKaasu5pU8bA; Thu, 29 Jun 2023 05:05:15 -0700 (PDT)
Received: from mail-oo1-xc2d.google.com (mail-oo1-xc2d.google.com [IPv6:2607:f8b0:4864:20::c2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 98BEBC14CE2C; Thu, 29 Jun 2023 05:05:15 -0700 (PDT)
Received: by mail-oo1-xc2d.google.com with SMTP id 006d021491bc7-563393b63dbso31487eaf.1; Thu, 29 Jun 2023 05:05:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1688040315; x=1690632315; h=to:subject:message-id:date:mime-version:references:in-reply-to:from :from:to:cc:subject:date:message-id:reply-to; bh=NsAlJxoS+4Ju2Nw9rO9QwUGAeAIhY5s9oC4JxMlL3uk=; b=FLby9KNbWUT5/ilo06H91sRaxA7zgSvNkOGtocW0aZTXyD6bo/hhTaWohUBvCHmmnQ FblrRS+TtF6/R6mIPmRtimX/b1Akr2DW1zWJ7WtwAT4WwhLVIVOqUXmmM4yuxRG2S9hD jbhYIkvQLT+2tea2x6WpZgpuJwNh782dYR68rkIQh1D5V0xLpSz8Pc2Afmc6TOEm4KQ6 puVg1zTMmNHJX7t3E33etCsbkdAip0InrEq5ftNrKWFGrcFoWArZf33i1zqodC0mTUx4 0b6ZmqBOkYnjC17s2ynNBXdDHy6lE77x03pS/yzM4obUI1wt8VHwce0Gh5h75ACMeS3a w8eg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688040315; x=1690632315; h=to:subject:message-id:date:mime-version:references:in-reply-to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NsAlJxoS+4Ju2Nw9rO9QwUGAeAIhY5s9oC4JxMlL3uk=; b=GggcRgt5esthCK1y0nBjH3eHrg2giPKL2VsIxI/OJN3EGRu6Sddpr4nM0T4GiWGxQS 32i8gq+h69LXNzcHRqYAnqY27jPJXm35iMAre7ePpj9EKBTwszFLkxbReEqn4fGMOxwd 6TaasTKNIq2z07WIyXOz7nzpijFs81C6NfbQtK2Pch1Nm/g4DCaDXDGViu3b+a8IGwj2 zaljxKuHA/txg675Kj2jXc69360iCBZp0BIbZcYrX8Ap4Sh2iYwYOvNCajQaT5u8S0O9 wmabA1dbKYF2IBUPw1uRVbPH6wAx3pH7DjBLlceiO24uJ7FPnkZnvTZJSuMSOATBv8lg 0Jnw==
X-Gm-Message-State: AC+VfDxHHlqZxQfgdMs/gAAd5xS8SJGhsVSap7KIJOxeVhaF04fZvf8u 9LqG8Fjb1badwYYlJkgE1m+pWka64kznztcioaUMqeDW
X-Google-Smtp-Source: ACHHUZ5Xs/nc5kVa+n/y7TaOD22Q/tyBh3LhMei7FS45aKpl0s1Ofixixpy0+ue8AdgIh4XNXdkFOeTVO3pmUzQbhp8=
X-Received: by 2002:a05:6820:2083:b0:562:e437:3b41 with SMTP id bz3-20020a056820208300b00562e4373b41mr13828578oob.0.1688040314423; Thu, 29 Jun 2023 05:05:14 -0700 (PDT)
Received: from 1058052472880 named unknown by gmailapi.google.com with HTTPREST; Thu, 29 Jun 2023 05:05:13 -0700
From: Kiran Makhijani <kiran.ietf@gmail.com>
In-Reply-To: <dd21b3cc-a2d0-1d73-208e-a4881009120e@linutronix.de>
References: <168634441557.62732.13801631789067741280@ietfa.amsl.com> <SJ0PR03MB6469BFFDD777D9F6DC62E955F754A@SJ0PR03MB6469.namprd03.prod.outlook.com> <dd21b3cc-a2d0-1d73-208e-a4881009120e@linutronix.de>
MIME-Version: 1.0
Date: Thu, 29 Jun 2023 05:05:13 -0700
Message-ID: <CAFV7YBpYqtLCQEh=hpb3rs7tdMRXCPKvwaHdkbS5vGtqXyw6pg@mail.gmail.com>
To: Florian Kauer <florian.kauer@linutronix.de>, draft-km-detnet-for-ocn@ietf.org, DetNet WG <detnet@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000004ecda405ff438381"
Archived-At: <https://mailarchive.ietf.org/arch/msg/detnet/I10wfLzucqpeTsuB_BgdElrhXsE>
Subject: Re: [Detnet] FW: I-D Action: draft-km-detnet-for-ocn-01.txt
X-BeenThere: detnet@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Discussions on Deterministic Networking BoF and Proposed WG <detnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/detnet>, <mailto:detnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/detnet/>
List-Post: <mailto:detnet@ietf.org>
List-Help: <mailto:detnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/detnet>, <mailto:detnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Jun 2023 12:05:18 -0000
Hi Florian, Many thanks for a thorough review. I was about to push another version that would have cleaned up some terminology issues you have identified. Since it is a full document review, please give me a couple of days to address and reply to your comments. -Kiran On June 28, 2023 at 12:18:48 AM, Florian Kauer (florian.kauer@linutronix.de) wrote: Hi Kiran, thanks a lot for this very insightful draft! The interface between a DetNet and the exterior world is actually what is most interesting for myself. I have attached a review of your draft and I am looking forward to your response! Greetings, Florian --- 1/draft-km-detnet-for-ocn-01.txt 2023-06-28 09:13:20.749215052 +0200 +++ 2/draft-km-detnet-for-ocn-01-fk-comments.txt 2023-06-28 09:13:20.761215141 +0200 @@ -104,43 +104,68 @@ describes using DetNet to enable OCN applications since they provide mechanisms for guaranteed delay aware packet delivery, reliability, and for packet loss mitigation. This document defines the interface between an OCN application and DetNet framework. i.e., using DetNet services for communication between the controllers and the field-devices. This interface is used by an application to express its network-specific requirements. This document presents the perspective of an end-system. Because the controller-endpoint of the application stack is based on IP, the +--- +fk: It would be good to extend the reasoning here a little bit. +Do you want to say most current endpoint application stacks already +use IP (in my experience many just use plain Ethernet without IP or +even use completely proprietary network stacks) or do you mean they +SHOULD in the future? + +Also you should explain why MPLS has no benefit for industrial +applications in your opinion (as I understand the statement in section 5.2). +--- discussion is specific to the IP-enabled DetNet data planes [DETNET-DP]. For the other type of field-devices, service level proxy is assumed (section 4.1 in RFC8655). +--- +fk: The draft uses many hyphenated words like "field-devices", +"time-frame", "control-command", "end-system", "controller-endpoint", +"service-sublayer"... It just looks a little unusual to me, but I am +not a native speaker, so it might be completely valid. +--- - By mapping OCNs to DetNet helps with developing a better - understanding of how DetNets can be used in such scenarios. To this + Mapping OCNs to DetNets helps with developing a better + understanding of how DetNet can be used in such scenarios. To this end, the document provides a background on Section 3 the type of traffic patterns in OCN applications. It proposes an interface between an application and DetNet, and a potential solution direction to support OCN traffic patterns over DetNet, as covered in Section 5. 2. Terminology * Operational Technology (OT): Programmable systems or devices that interact with the physical environment (or manage devices that interact with the physical environment). These systems/devices detect or cause a direct change through the monitoring and/or control of devices, processes, and events. Examples include industrial control systems, building management systems, fire control systems, and physical access control mechanisms. Source: [NIST-OT] * Industry Automation: Mechanisms that enable machine-to-machine +--- +fk: Throughout the text (including the title) you switch between +"Industry" and "Industrial" in open compound words, sometimes even +for the same thing ("Industry control networks" in 4.2.5. +vs "Industrial Control Networks" just below). +Is there a deeper meaning behind that or is it just preference +of different authors? I would personally prefer "industrial" everywhere, +but again that might be my German imprint. +--- communication by use of technologies that enable automatic control and operation of industrial devices and processes leading to minimizing human intervention. * Control Loop: Control loops are part of process control systems in which desired process response is provided as input to the controller, which performs the corresponding action (using actuators) and reads the output values. Since no error correction is performed, these are called open control loops. @@ -175,29 +200,44 @@ 3. Background on Industrial Control Systems An industry control network interconnects devices used to operate, control and monitor physical equipment in industrial environments. Figure 1 below shows such systems' reference model and functional components. Closest to the physical equipment are field devices (actuators and sensors) that connect to the Programmable Logic Controllers (PLCs) or other types of controllers using serial bus technologies (and now Ethernet). Above those controllers are Human - Machine Interface (HMI) connecting different PLCs and performing + Machine Interfaces (HMI) connecting different PLCs and performing several controller functions along with exchanging data with the applications. A factory floor is divided into cell-sites. The PLCs or other types of controllers are physically located close to the equipment in the cell-sites. The collection of monitoring, status and sensing data is first done on the site and then transmitted over secure channels to the cloud applications. +-- +fk: In my opinion the term "cloud" can be very misleading at its first appearance +here without explanation since we are in the DetNet context (i.e. no Internet +according to the DetNet charter). I would either skip the term here for now +(and replace it with "data apps" as in the figure) and/or move the explanation up. + +One suggestion would be for example: +"...and then transmitted over secure channels to data applications for +aggregation and further processing. These applications can be hosted +in remote cloud infrastructure, but are also often hosted within a +limited domain environment, controlled by a single operator, like +on-premise, at the edge or in a private cloud. Both options gain +from infrastructure that scales out, has elastic compute and storage +resources, so they will both be referred to as cloud in the following." +-- +-+-+-+-+-+-+ ^ | Data Apps |.... External business-logic : +-+-+-+-+-+-+ : Network : | : v +-+-+-+-+-+-+ +-+-+-+-+--+ | vendor A | |vendor B | Interconnection of | controller| |controller| controllers ^ +-+-+-+-+-+-+ +-+-+-+-+-+ (system integrators) : | | @@ -205,20 +245,28 @@ : | Net X | | Net Y| v | PLCs | | PLCs |--+ device-controllers ^ +-+-+-+-+ +-+-+--+ | : | | | : +-+-+ +-+-+ +-+-+ v | | | | | | Field devices +-+-+ +-+-+ +-+-+ Figure 1: Functions in Industrial Control Networks +-- +fk: And with the preperation above you could continue here with e.g. +"Having full control over the network in a limited domain environment +allows for the deployment of a DetNet and this enables +the cloud applications to integrate process control functions to +improve automation and to make real-time decisions, programmatically..." +-- + What is changing now is that cloud applications are integrating process control functions to improve automation and to make real-time decisions, programmatically. The equipment control and collection of data generated by the sensors can be done directly over DetNet- enabled wide-area networks as illustrated in Figure 2. Inline with DetNet, cloud applications can also be hosted within a limited domain environment, controlled by a single operator. The term is implied to indicate that applications use infrastructure that scales out, has elastic compute and storage resources. The @@ -233,21 +281,37 @@ [Network] `-----' +-+-+-| |-+-+-+-+ | | | +-+-+ +-+-+ +-+-+ | | | | | | Field devices +-+-+ +-+-+ +-+-+ Figure 2: Converged Cloud based Industrial Control Networks +--- +fk: In the new architecture do you still have (some) controllers +on-site or do you move all to the cloud and just have the sensors +and actuators on-site? + +In the figure and also in the remainder of the draft the distinction +between the "Data apps" and the controllers is quite blurry. +Is the idea to deploy integrated applications that do everything +from running PID controllers to visualizing big data sets? +Or is there still a clear distinction between controllers and +data apps even if both are deployed in the cloud? +--- + One particular motivation is to provide the behavior of a serial bus +-- +fk: Maybe "field bus" would be a better fit here instead of "serial bus"? +-- between the cloud and the actuators/sensors with the same assurance of reliability and latency, albeit over wide-area networks (WAN). This is evident from many industry control applications, such as factory automation [FACTORY], PLC virtualization [VIRT-PLC], power grid operations [PTP-GRID], etc. that are now expected to operate in the cloud by leveraging virtualization and shared infrastructure wherever possible. 3.1. Connected Controllers, Sensors and Actuators @@ -274,20 +338,29 @@ intermittently provide asynchronous readings upon request from the controller. Sensors may report urgent messages regarding malfunctioning in certain equipment, cell-sites, or zones. Almost all control systems have at least one controlling entity on one end, and two other end points - the sensors and actuators. The interface to sensors and actuators is through the controllers; i.e., applications do not directly interact with the field-devices. Neither actuators nor sensors perform decision-making tasks. This responsibility belongs to the controller. +-- +fk: What do you mean by "Almost" here? I can think about two exceptions +(and maybe they should be mentioned explictly): +- Pure sensors (e.g. to record temperature, humidity... without taking + any active action in response). They will be directly interfaced + (but not controlled) by the applications. +- A single field device that integrates a controller, an actuator and sensors + for autonomous control. +-- 3.2. Traffic Patterns For either local or wide area, the process automation activities over the network can generate a variety of traffic patterns between the controllers and field-devices such as: 3.2.1. Control Loops The equipment being operated upon is sensitive to when a command @@ -302,32 +375,65 @@ therefore, getting the response back in a specified time is required, leading to a knowledge of timing. These types of bounded-time request and response mechanisms are called control loops. Unlike general purpose applications, commands cannot be batched, the parameters of the command that will follow depends on the result of the previous one. Each request in control loop takes up a minimal payload size (function code, value, device or bus address) and will often fit in a single short packet. - In Detnet-enabled network, it can be imagined as a small series of + In DetNet-enabled network, it can be imagined as a small series of packets with the same flow identifier, but with different latency constraints. It is required to support control loops where each request presents its own latency constraints to the network and where commands are small sized packets. +-- +fk: What do you mean by "own latency constraint" here? +For all applications I have in mind it would be sufficient if a +certain DetNet flow guarantees a common miniumum (Tmax(Flow)) and +maximum (Tmin(Flow)) latency for all packets of a certain application +(e.g. one data stream between a sensor and a controller) instead of +individual Tmax(Packet) and Tmin(Packet) for every packet. + +Tmax(Packet) > Tmax(Flow) and Tmin(Packet) < Tmin(Flow): +Packet latency constraint is fulfilled. +Tmax(Packet) < Tmax(Flow): +Since the flow has to guarantee a shorter latency for SOME packets, +what is the advantage of having a relaxed Tmax(Flow) for other +packets? IMHO lower resource utilization will not be achieved, +because the network still needs to support a high number of +packets with low Tmax(Packet) in succession without sufficient time +for adaptation, so all required resources for the extreme case +need to be preallocated anyway. + +Tmin(Packet) > Tmin(Flow): +The DetNet COULD provide an additional buffer that delays each +packet individually for Tmin(Packet) - Tmin(Flow), but what is the +advantage of that over the controller buffering the packet or just +generating it later? + +What do I overlook here? +-- 3.2.2. Periodicity Sensors emit data at regular intervals, but this information may not always be time-constrained. Usually, controllers are programmed to +-- +fk: It is unclear to me what you mean by "time-constrained" here. +That there might be no bounds on the jitter of the measurement intervals? +Or that it is not necessarily possible to determine when exactly a +certain sensor reading was generated? Or...? +-- tolerate and record intermittent losses. Automation software can make a more informed decision by monitoring a lot of sensor data. Thus, the traffic volume generated by sensors is expected to high. The periodicity of each sensor can also vary based on the equipment. It is required that network capacity is planned appropriately for the periodic traffic generated from the different sensors. The periodic interval should also be preserved in the network because any variations could provide false indications that the equipment is misbehaving. @@ -339,56 +445,99 @@ as request and reply, or a sequence of commands may be correlated therefore, both time constraints and order must be preserved. The traffic is generated when software triggers control-commands to field-devices. This may not always map into asynchronous DetNet flows if observation interval is not known. The network should be capable of supporting sporadic on-demand short- term flows. This does not imply instantaneous resource provisioning, instead it would be more efficient if the provisioned resources could be shared for such asynchronous traffic patterns. +--- +fk: It would be good to elaborate more on what you have in mind. +Do you assume that the resource reservation should be large enough to +cover the case that all end systems are able to generate sporadic +on-demand short-term flows at the same time? +If the resource provisioning is smaller, how do you prevent that no +overload situation with a negative impact on the application occurs? + +Also, it is not quite clear to me why this is in the ordering section. +--- Another consideration with ordering is that both actuators and sensors are low-resource devices. They can not buffer multiple packets and execute them in order while maintaining the latency bounds of each command execution. This means the network must pace packets that may arrive early. +--- +fk: IMHO the fact that actuators have low resources is not a premise +that results in the inability to implement a POF, but rather the +consequence of the fact that it was not required in the past. + +So if we talk about existing actuators that are unable to act as +DetNet end systems and require a gateway then THAT is the reason +they do not implement a POF. + +If we talk about future to be developed DetNet-enabled actuators +I still see it as open for discussion if it is better to spend some +more resources for the actuators to implement a POF instead of +pushing that requirement to an intermediate network participant. +--- + 3.2.4. Urgency Besides latency constrained and periodic messages, sensors also report failures as fault notifications, such as pressure valve failure, abnormally high humidity, etc. These messages must be delivered with utmost urgency and immediately. +--- +fk: Why is that an additional traffic pattern and not a special case +of transmitting latency constrained packets with a low maximum allowed latency? + +By the way: You might want to mention heartbeat functionality for covering +very critical messages (i.e. instead of requiring a device to send out +"I just exploded" which might not be possible in that case, it might be better +to send out "I did not explode yet" in an interval that is sufficient for the +receiver to act appropriately if it does not arrive in time). +--- 3.3. Communication Patterns Control systems follow a specific communication discipline. The field-devices (sensors and actuators) are always controlled, i.e., interact with the system through controllers in the following - manner:- + manner: * Sensor to controller: data emitted at periodic interval providing status/health of the environment or equipment. The traffic volume for this communication is determined by the payload size of each sensor data and the interval. These are a kind of synchronous - Detnet flows but with much higher time intervals; still the inter- + DetNet flows but with much higher time intervals; still the inter- packet gap should be minimum. * Controller to/from actuator: the commands/instructions to write or read. Actuators generally do not initiate a command unless requested by the controller. Actuators will often execute a command, read the corresponding result, and send that in response to the original write command. The traffic profile will be balanced in both directions due to requests/ response behavior. These are like asynchronous flows but without the observation interval constraint. +--- +fk: Is that really always asynchronous traffic? In many closed loop +applications the controller sends just as many actuator commands than +it receives sensor readings. Also in many such applications handling +of deviations when applying a certain command is the responsibility of +the controller by reading the sensor values instead of requiring a +dedicated response from the actuator. +--- 4. Gap Analysis Today, most of the operations and control solutions are split approaches. This means that the controller is on-premises close to the equipment, sensor data is first collected on-site, and then bulk transmitted to the enterprise cloud for further processing. To support delivering remote instructions to the machines over wide- area networks using Deterministic Network data plane architecture @@ -439,93 +588,146 @@ Realistically, leveraging DetNets for Operations and Control (OCN) traffic patterns Section 3.2 directly depends on how DetNets will provide network knowledge to the applications. 4.2. DetNet related Considerations and Dependencies A DetNet-aware node should express the network requirements as part of either forwarding sublayer or service-sublayer. [DETNET-IP] doesnot specify the interface to how those sublayers are mapped. - This can be a non-trivial task, as an OCN-application, will first + This can be a non-trivial task, as an OCN-application will first need some way to request its DetNet service provider (DN-SP). The +--- +fk: Is DN-SP a term that is introduced by this draft? If yes, please +explain it in more detail. Is it a dedicated device in the network +(similar to a TSN CNC)? Or just a virtual distributed network functionality? +--- DN-SP is expected to allocate resources and return a mapping - - possibly a DSCP (DetNet Qos) for each pair. This could be become a + possibly a DSCP (DetNet QoS) for each pair. This could be become a +--- +fk: ... for each pair of controller and sensor/actuator? +Or each pair of controller and data application? Or...? +--- tedius scaling problem as the number of controller-device pairs start to grow or keep changing. Given that only DSCP is available, field-device pair can pose issues such as: +--- +fk: Is the sentence garbled or am I just unable to parse it properly? +--- - * How can application request the proper network-resource for each + * How can applications request the proper network-resource for each command? * How can an application receive periodic data from sensors and with - what interval? + which interval? * What are the ways to differentiate a less sensitive (periodic) updates from urgent alarms. +--- +fk: As written above, it would be good to outline more why that is required. +--- * Or how to differentiate data received from a sensor vs an actuator (with stringent latency requirements) and process them accordingly? These issues are described below in more detail. 4.2.1. Operator vs Application view The DetNet data plane is designed with a network-operator-centric approach. The operator's view on dealing with large-scale networks is discussed as newer requirement in +--- +fk: What do you mean by "newer" here? +--- [I-D.ietf-detnet-scaling-requirements]. In order to use resources efficiently, flow aggregation is done. The operators in industrial control networks are not necessarily network experts; they simply need an application side tool to dispatch their request to the Deterministic networks' entry point. Especially, an OCN application may need many controller-field-device (ctrl-flddev) pairs to the be +--- +fk: You only refer to ctrl-flddev once directly below. +No need to introduce another abbreviation. +Still, the term "fld device" is used several times later in the draft, +but I would also prefer to just write field device. +--- mapped to different aggregates. As the number of ctrl-flddev pairs grow, their variable traffic profiles can become hard to manage. - The Detnet service provisioning internals should be transparent to an + The DetNet service provisioning internals should be transparent to an OCN application. This can be achieved with a common signaling or user-to-network interface between the applications and DetNet. 4.2.2. Flow reservation and classification Inside the DetNet, flow identification is done using IP header and DSCP information. These flow identifiers are then used by DetNet +-- +fk: ...or TCP/UDP port that is not part of the IP header, +while DSCP is part of the IP header +-- nodes to provide the corresponding traffic treatment. For a variety of commands and sensor data, latency or interval parameters will vary and DSCP maybe limited in expressing all the possible requirements. Embedding requirements explicitly can provide deterministic scheduling even in an otherwise link that can be congested when used with non-deterministic flows. +--- +fk: What do you mean with "embedding"? Embedding the requirements into +each packet or embedding a mapping of flow indentification to requirements +into each network participant (e.g. via YANG)? +(I assume the first after reading the subsequent sections, but it would +be good to elaborate more why you say the other interpretation is not +feasible for the given application.) +--- 4.2.3. Split Traffic flows One of the most constrained design elements in today's industrial control systems is that data from the sensors is collected on-site and often aggregated before transporting to the cloud. Historical reasons for this approach do not apply anymore. Due to growth in sensor data, it now requires a much larger on-site storage infrastructure which is expensive. Applications also expect real- +--- +fk: Again, be careful here since we are still talking about dedicated +infrastructure and not the common cloud hyperscalers, so the cost +advantage is lower than with other cloud applications that are not +restricted to dedicated infrastructure. + +Instead of stretching the fine lines between "on-site", "edge" and +"private cloud", I would put more emphasis on the fact that collecting +unaggregated data from several different sites and acting upon it +dynamically enables new applications (e.g. better predictive +maintainance to just name one) and that in the end reduces the costs. +--- time streaming telemetry data. Although latency constraints are not as strict as for control loops, sensor data need to preserve periodicity (Section 3.2.2), thus could use DetNet service support. Leveraging DetNet could eliminate split traffic flows by collecting the sensor data by the applications. This also allows controllers to be run and operated from the cloud platforms where much more powerful - compute capabilities and available. + compute capabilities are available. +--- +fk: The same holds here where it would be good to reiterate the fact of +dedicated infrastructure and why that is still more powerful than what +is reasonable to implement on the factory floor. +--- 4.2.4. Provisioning for variety of Traffic flows Different operational scenarios have different constraints; even commands within the same application will have different time requirements. * Different types of latency bounds will be required between a controller and an actuator pair based on the type of end-equipment and precision requirements. Out-of-order message processing may @@ -545,50 +747,62 @@ It is not clear if all these variations can be predictably resolved without any additional information offered to the DetNet forwarding plane. For example, if two independent OCN flowlets (that is, ordered group of packets that are related at process control logic) with variable bounded latency are classified to the same DetNet flow, they will receive the same treatment, regardless if one has the shorter latency than the other and may end up behind a flowlet with longer latency value. On the other hand, if an OCN flowlet have packets with different latency values, they could end up in different DetNet flow and may not reach destination in a specific order. +--- +fk: Does latency in this paragraph refer to the actual latency of a +concrete set of packets experiences or the specified latency bound +requirements of these packets? +--- 4.2.5. Security Industry control networks also have split security boundaries. They have been designed to be air-gapped or secure by separation. Current systems have strict admission control, ingress and egress policies. >From network layer security perspective, how DetNet-enabled network deals with security falls in the [RFC9055], the end-systems expect those mechanisms in place. In particular if additional information is distributed for datapath decisions, integrity protection as per Section 7.2 of [RFC9055]. The border gateways and firewalls will be more prone to errors related to provisioning churns if the system is dynamic or continuously changing. The transport layer deals with the end-to-end encryption. It should evolve to incorporate additional IoT-friendly(lightweight) protocols such as COAP, MQTT and their encryption mechanisms. +--- +fk: AFAIK encryption of COAP and MQTT is still implemented on transport layer +(SSL/TLS/DTLS) and not part of the application layer. +--- 4.3. Summary of Gaps - * Application view (Section 4.2.1: An OCN application is unaware of + * Application view (Section 4.2.1): An OCN application is unaware of how DetNet services are provisioned. A common UNI between the applications and DetNet-enabled network needs to be added to the - current framework to better map the expectations better. + current framework to better map the expectations. * Security (Section 4.2.5): of process control related metadata to be used by network must be secured. +--- +fk: Is the sentence garbled or am I just unable to parse it properly? +--- * Traffic behavior (Section 4.2.4 and Section 4.2.2): Within the same DetNet flow, classified via 6-tuple, additional information/ metadata must be supported so that dynamic traffic patterns can be scheduled deterministically. * Split traffic (Section 4.2.3): Leveraging DetNet should eliminate split traffic flows by direct collection of sensor data by the applications. This also allows controllers to be run and operated from the cloud platforms where much more powerful compute @@ -609,29 +823,39 @@ the separation between forwarding and service sublayer functions should be maintained. This means, the DSCP should not be overloaded and DetNet-IP forwarding layer should be extended. 5.1. Application association to Forwarding sub-layer Applications should convey specific resource requirements to the DetNets they connect to. There are two potential options: (a) The DetNet Relay-node performs translation and binding to one of the DetNet services in the DetNet; or (b) or carry the application +--- +fk: double "or" +--- defined data over DetNet as is and enable processing on transit nodes. 5.2. Encapsulation OCN applications are expected to be IP-based end-stations. (MPLS DetNet will not apply). It is also reasonable to assume that the data plane is IPv6 and Extension Headers are used for support in DetNet. +--- +fk: Can we really already exclude IPv4? Would be overdue, but does +not match my experience unfortunately... What is yours? +Or is your argument that your proposal anyway requires special +handling by the whole network and thus it does not make any +sense to deploy a new OCN with IPv4? +--- The end-system network requirement is expressed as 'OCN flow QoS'. Each packet carries its own unique OCN-QoS. The meta data to be transmitted to DetNet are: - Async traffic with latency-information. - Sync, periodic traffic - urgency of messages - Flowlet identification (for related packets). @@ -659,21 +883,21 @@ Option Type: 8-bit identifier of the type of option. The option identifier for the OCN Option (0x??) to be allocated by the IANA. First two bits will be 00 (skip over this option and continue processing the header.) Option Length: 8-bit unsigned integer. Multiple of 8-octets. OCN Function Flags: - Some flags require metadata, others dont. So process flags in + Some flags require metadata, others don't. So process flags in order, if the flag is off, following metadata will not be present. +======+===============================================+ | Flag | Description | +======+===============================================+ | U | send message immediately. its an alarm | +------+-----------------------------------------------+ | P | periodic packet (intervals in ~ms) | +------+-----------------------------------------------+ | F | part of flowlet. see Nonce and seq | @@ -686,20 +910,27 @@ +------+-----------------------------------------------+ Table 1 Flowlet nonce: 16-bit. identifies that a packet is associated to group of packets and shares fate. as an example, an application can set same nonce for a set of an actuator and sensors. when set to 0,flow id flowid is set to same value in related flows. when flow id is also 0, no relationship exists. +--- +fk: Since you call it nonce and not ID, I assume it is generated autonomously +be the application? Is there any danger of flowlet nonce collision (especially +since you only use 16 bit)? + +Furthermore, this paragraph is partly garbled (e.g. "flow id flowid"). +--- Flowlet sequence: 8-bit. sequence to be used for ordering with in flowlets. Bound Latency Spec: 32-bit. Encodings, to be defined. 16-bit (upper bound), 16-bit (lower-bound). This field will provide upper and lower latency bounds describing the the latency bounds in milliseconds corresponding to the packet. @@ -715,38 +946,67 @@ | OCNO-EH :--UNI-->: Service :-- DetNet -: Service : +-----+ +----------+ +----------+ +----------+ |IPv6 | | Ipv6 | |Forwarding| |Forwarding| +-----+ +--------.-+ +---.------+ +----------+ : : OCN scope : : : +..............+ : :--------------------------------------------: extended scope Figure 5: An interface from controller application to DetNet +--- +fk: IMHO the figure would benefit from some cleanup (e.g. the field device +is not aligned properly with the other entities and thus it is not clear +for me how the actual communcation between the egress relay node and the +field device takes place). +--- 5.5. OCNO Extension Header Signaling The current definition of OCNO only covers application to DetNet unidirectional interface. It is possibile to extend OCNO EH for +--- +fk: Is the first sentence garbled or am I just unable to parse it properly? +--- conveying errors from DetNet to the controller as well - for example, when service violation happened in the DetNet, Relay node will set error flag in OCNO EH. Field devices are considered resource constrained and are not expected to insert or process extension headers. Due to flow aggregation, it is upto the DetNet operator to +--- +fk: Again, if there is a reasonable advantage of implementing extension +header support in the field devices, I do not see why it wouldn't be +reasonable to increase the resources of the field device accordingly. +--- design mapping between an application and the DetNet. Two different options of carrying hop-by-hop options are considered. 1. EH is inserted by application and Relay node performs mapping to DetNet flow. 2. if the DetNet dataplane is IPv6, then EH can be carried all the way to last Relay node, which acts as gateway for the fld device and performs EH specific functions. +--- +fk: It would be good to elaborate more on how the DetNet should act upon +the information provided by the OCNO. Some questions I have in mind: +- For the first option: Is there still a separate static configuration of + the DetNet flows required or should the relay dynamically allocate new + flows according to the incoming requirements as provided by the OCNO? +- If static, how should the relay node react if it does not find an + appropriate DetNet flow to process a certain packet? +- If dynamic, how to mitigate the time required to setup a new flow when + a new packet comes in with a requirement that can not be fulfilled yet? +- For the second option: Does that mean that every intermediate relay + needs to be OCNO aware (e.g. to forward the packet according to the + requirements)? +--- + 6. IANA Considerations To request an option code. 7. Security Considerations See section on security above. 8. Acknowledgements
- [Detnet] FW: I-D Action: draft-km-detnet-for-ocn-… Kiran Makhijani
- Re: [Detnet] FW: I-D Action: draft-km-detnet-for-… Florian Kauer
- Re: [Detnet] FW: I-D Action: draft-km-detnet-for-… Kiran Makhijani
- Re: [Detnet] FW: I-D Action: draft-km-detnet-for-… Kiran Makhijani