Re: [Detnet] Fwd: review DetNet OAM framework draft

["Pascal Thubert (pthubert)" <[e-mail]>]

Hello Greg

Please see below

On Thu, Apr 29, 2021 at 6:07 AM Pascal Thubert (pthubert) <[e-mail]<mailto:[e-mail]>> wrote:
Dear authors and all

I do support this document but do not see that in its present shape it is ready for WGLC. Please find some review comments below.

- Sadly RFC 8655 does not define either “latency” or “delay”. Reading this draft, I now see it as an oversight. I’d suggest that we define them in the terminology section. it is my sense that RFC 8655 uses “delay”  with the meaning of incremental flight duration, and uses “latency” for the total time end-to-end time across the network, IOW, minimal_latency + delay <= bounded_latency.
GIM>> Thank you, Pascal, for bringing this to the discussion. I agree with you and I strongly believe that it is utterly important to set the dictionary so that we can discuss technology and use cases based on the common understanding of terms. I like your suggestion to provide definitions of "latency" and "delay" in the Terminology section. Before going into definitions, I'd like us to define the scope of these terms. Would it be for the draft and all DetNet OAM documents or it can serve as the clarification of the terminology used in DetNet in general since RFC 8655?
About the definitions. Do you think that the interpretation of "delay" is close to the definition of the residence time (RFC 8169):
   Residence time is the sum of the difference between
   the time of receipt at an ingress interface and the time of
   transmission from an egress interface for each node along the network
   path from an ingress node to an egress node.
Also, in RFC 8169 we've assumed that the e2e delay, i.e., e2e latency consists of two parts - fixed propagation delay determined by the link speed and variable portion, i.e., residence time. In your opinion, could this model be applied to DetNet?


PT> Well the residence time seems like the latency of the network transit, and I see the delay as the difference between the best possible latency and the actual latency experienced by a packet. Like when I say, I’ve been delayed by 30mn in a trip that takes 5 hours when all the lights are green so the residence time in my car (aka latency) is 5H30mn. The DetNet architecture seems consistent with that interpretation, at least as far as I intended to write and can read the language.







- “It is critical for the

      quality of information obtained using an active method that

      generated test packets are in-band with the monitored data flow.

”. this duplicates text in 3.3 and does not belong in a terminology. Remove?
GIM>> I agree with you. Text in Section 3.3 is stronger but seems out of place in that section. I think moving it up into Section 3 might be appropriate. What do you think?





PT> Cool





- “                                                                                                         OAM represents the

   essential elements of the network operation and necessary for OAM

   resources that need to be accounted for to maintain the network

   operational.

” does not parse too well
GIM>> I propose the following update:
 OLD TEXT:
   OAM represents the
   essential elements of the network operation and necessary for OAM
   resources that need to be accounted for to maintain the network
   operational.
NEW TEXT:
   Because OAM is an
   essential element of the network operation,  resources, necessary for
   OAM, need to be accounted for in addition to DetNet flows.

PT> great




- “information about the state of the network being

   collected

” is being? Maybe remove that text ? it seems to repeat the first sentence of the paragraph.
GIM>> The purpose of the sentence is in the final part "... and sent to the controller".Would the following update make it better:
OLD TEXT:
   In either way, information about the state of the network being
   collected and sent to the controller.
NEW TEXT:
   In either way, information is collected and sent to the controller.

PT> I see now





- “   Also, we can characterize methods of transporting OAM information

   relative to the path of data.  For instance, OAM information may be

   transported out-of-band or in-band with the data flow.

” this deserves its own subsection with some more explanation and possibly some references to relevant IETF work
GIM>>  Thank you for highlighting this question. I propose adding references to work at the IPPM WG. Here's the updated text:
NEW TEXT:
   Also, we can characterize methods of transporting OAM information
   relative to the path of data.  For instance, OAM information may be
   transported in-band or out-of-band with the data flow.  In case of
   the former, the telemetry information uses resources allocated for
   the monitored DetNet flow.  If an in-band method of transporting
   telemetry is used, the amount of generated information needs to be
   carefully analyzed, and additional resources must be reserved.
   [I-D.ietf-ippm-ioam-data] defines the in-band transport mechanism
   where telemetry information is collected in the data packet on which
   information is generated.  Two tracing methods are described - end-
   to-end, i.e., from the ingress and egress nodes, and hop-by-hop,
   i.e., like end-to-end with additional information from transit nodes.
   [I-D.ietf-ippm-ioam-direct-export] and
   [I-D.mirsky-ippm-hybrid-two-step] are examples of out-of-band
   telemetry transport.  In the former case, information is transported
   by each node traversed by the data packet of the monitored DetNet
   flow in a specially constructed packet.  In the latter, information
   is collected in a sequence of follow-up packets that traverse the
   same path as the data packet of the monitored DetNet flow.  In both
   methods, transport of the telemetry can avoid using resources
   allocated for the DetNet domain.

PT> This is where RAW needs a third method, similar to the latter but that collects the information on the reverse path to help the PSE at ingress make its decision




“   Similarly, the destination does not receive packets from different

   flows through its interface.

” sorry I do not get the intended meaning
GIM>> We're pointing to the distinction between the Continuity Check (CC) and Connectivity Verification (CV). The former only verifies that a path between two systems exists, while the latter, in addition to that path, verifies that the system does not receive packets from any flow other than one being monitored. Associated with the CV, is a misconnection error state and conditions of raising and clearing it. I propose removing the paragraph with that sentence and update the first paragraph as follows:
OLD TEXT:
   In addition to the Continuity Check, DetNet solutions have to verify
   the connectivity.  This verification considers additional
   constraints, i.e., the absence of misconnection.
NEW TEXT:
   In addition to the Continuity Check, DetNet solutions have to verify
   the connectivity.  This verification considers additional
   constraints, i.e., the absence of misconnection.  The misconnection
   error state is entered after several consecutive test packets from
   other DetNet flows received.  The definition of the conditions of
   entry and exit for misconnection error state is outside the scope of
   this document.

PT> WFM




- section 3.3 –“It is worth noting that the test and data packets MUST follow the

   same path, i.e., the connectivity verification has to be conducted

   in-band without impacting the data traffic.”

 there appears to be a tension between the bandwidth that is reserved for the flow and the extra bandwidth that the OAM needs .

Do we need to know in advance how much OAM there will be? Must that be included in the reservation?
GIM>> Yes, the impact of OAM, whether active or hybrid, must be evaluated and accounted for when reserving resources in the DetNet domain. The update above is intended to clarify that:
   Because OAM is an
   essential element of the network operation,  resources, necessary for
   OAM, need to be accounted for in addition to DetNet flows.

PT> I hoped you would clarify that the DetNet controller must be aware of the amount of resources that OAM needs, and would reserve those resources together with those needed for the flow







- section 3.4. What do we do for PREOF?
GIM>> Excellent question, thank you! I think of two updates:

  *   in Section 3.4:
   Also, tracing can be used for the discovery of the Path
   Maximum Transmission Unit or location of elements of
   PREOF for the particular route in the DetNet domain.

  *   in Section 6:
   DetNet OAM MUST support the discovery of PREOF along a route in
   the given DetNet domain.



PT> Cool. Do you intend to flood the OAM along the PREOF graph? Can you do local assurance, like, whether an individual node that is supposed to replicate effectively does so, without impacting the end to end flow?





- “The network has isolated and identified the cause of the fault.”

Again I fail to understand the intention.
GIM>> Indeed. How about we change this text as the following:
OLD TEXT:
   The network has isolated and identified the cause of the fault.  For
   instance, the replication process behaves not as expected to a
   specific intermediary router.
NEW TEXT:
   An ability to localize the network defect and provide its
   characterization are necessary elements of network operation.

* Fault localization, a process of deducing the location of a network failure
   from a set of observed failure indications, might be achieved, for
   example, by tracing the route of the DetNet flow in which the network
   failure was detected.  Another method of fault localization can
   correlate reports of failures from a set of interleaving sessions
   monitoring path continuity.
* Fault characterization is a process of identifying the root cause
  of the problem.  For instance, misconfiguration or malfunction of
  PREOF elements can be the cause of erroneous packet replication or
  extra packets being flooded in the DetNet domain.


PT> Much clearer and more informative to me, thanks you!





-“                                  One of the advantages of the use of

   AMM in a DetNet domain with the IP data plane is that the marking is

   applied to a data flow, thus ensuring that measured metrics are

   directly applicable to the DetNet flow.

” isn’t this the case for IPPM in general? Isn’t the benefit of AMM more related to conciseness?
GIM>> You are right.  Do you think that the update below makes the text better:
OLD TEXT:
   One of the advantages of the use of   AMM in a DetNet domain with
   the IP data plane is that the marking is   applied to a data flow, thus
   ensuring that measured metrics are   directly applicable to the DetNet
   flow.
NEW TEXT:
   As with all on-path telemetry methods,
   AMM in a DetNet domain with the IP data plane is natively in-band in
   respect to the monitored DetNet flow.  Because the marking is applied
   to a data flow, measured metrics are directly applicable to the
   DetNet flow.  AMM minimizes the additional load on the DetNet domain
   by using nodal collection and computation of performance metrics in
   combination with optionally using out-of-band telemetry collection
   for further network analysis.

PT> cool




- Maybe we should discuss the tension between RFC 8939 and the capability to use separate packets for OAM, since the flow is routed on the 5/6 tuple. Does that mean that only in-situ is feasible with RFC 8939 (ref draft-ietf-ippm-ioam-ipv6-options)?
GIM>> I agree, that the definition of the DetNet IP flow is an important aspect when considering the applicability of IOAM in a DetNet domain. That may be discussed in this document, though, as I think of it, that discussion seems also in place in draft-ietf-detnet-ip-oam <https://datatracker.ietf.org/doc/draft-ietf-detnet-ip-oam/> . What do you think?

PT> I’d like to see some text explaining the problem and the requirement here. It does not hurt to copy paste a bit in the IP OAM draft if that helps the reader of either doc.




Section 4. This section needs more thoughts. For instance there are packets like PTP packets for which it is important to know the transit duration in each hop. Isn’t that true for DetNet as well?
GIM>> We can discuss if and how RFC 8169<https://datatracker.ietf.org/doc/rfc8169/> may be applicable. Perhaps in a new discussion thread.

let’s do that. I wish this doc to be high level but holistic, showing all the aspects and angles and not constrained by the drafts we already have. It is the framework for the present capabilities, but also for the future ones.


Radio links have numerous metrics that can be reported (think RFC 8175), we need to know that information per hop as opposed to per path/circuit. We need to know if the hop does store-and-forward or passthrough – e.g., in the former case using smaller frames / packets / segments / fragments and/or network coding may improve the latency. Maybe we could dump the WG collective mind with a dedicated thread?
GIM>> I agree, a new discussion thread will certainly help. I've been thinking of per node metrics too. I think that to be useful, per node metrics might as well be for the given ordered pair of ingress-egress interfaces. I understand that blows the amount of information we generate and collect, but by doing on-node PM calculations exported will be median, percentiles values. Definetely needs discussion.

PT> we’re in sync




- section 5.2: Is it the intention to include the repair action in OAM work?
GIM>> Thank you for the question. Thinking of it, OAM role is the detection of a performance metric degradation. Restorative actions, in my opinion, are taken by the control or management plane. Since the detection of performance monitoring and SLO are discussed in other sections, the value of Section 5.2 may be not sufficient to keep it in the document. What do you think?

PT> confirming what we said at the call, I think it does not belong. There’s a spectrum of things that can be done based on the OAM information, from simple dashboard and assurance to network automation (ala MPLS TP) to, at the extreme, the  RAW Path selection. In the extreme cases, there’s an SLO for the reaction, so the OAM flow that feeds the PSE becomes time-sensitive as well.



- I’m not sure about requirement 5, 6, 10, 11, and of the use of “excellent” in 16.
GIM>> Let's remove the "excellent" :)
GIM>> The intention of listing OAM functionality in that particular manner as requirements is to provide the check list for the gap analysis and defining extensions. Are you concerned wth the described OAM functions or the manner requirements are expressed?

PT> Yes, items 5, 6, 10, 11 need more discussion. Maybe we can discuss that in yet another thread?

Take care,

Pascal


From: detnet <[e-mail]<mailto:[e-mail]>> On Behalf Of Janos Farkas
Sent: jeudi 29 avril 2021 0:28
To: [e-mail]<mailto:[e-mail]>
Subject: [Detnet] review DetNet OAM framework draft

WG,

Please review the DetNet OAM framework draft: https://datatracker.ietf.org/doc/draft-ietf-detnet-oam-framework/.

The authors consider the draft in a pretty good shape, do not see any major piece missing.

WG review and comments would be great in order to develop the draft towards WG Last Call.

Please send your comments to the list before the next informal OAM discussion on May 11 or bring it to the informal meeting: https://ietf.webex.com/ietf/j.php?MTID=m27bb8ee31f025670e4071bfdfcd47432. (More details on the OAM informal meetings: https://mailarchive.ietf.org/arch/msg/detnet/4ik3tI-E3uhOFNaKhpkZiWwyiZc/.)

Regards,
Janos