Re: [ippm] WGLC for IOAM Flags

Tal Mizrahi <> Thu, 30 September 2021 07:42 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 1D89C3A1769; Thu, 30 Sep 2021 00:42:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id iQTKn6RzQQDg; Thu, 30 Sep 2021 00:42:31 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::431]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id F00563A1737; Thu, 30 Sep 2021 00:42:29 -0700 (PDT)
Received: by with SMTP id t8so8475865wrq.4; Thu, 30 Sep 2021 00:42:29 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=095DUiz9R55WohpIG0Cj/NcoduVCwSduaLRbsGkQo94=; b=jUbHY8J+wjkpJ10x6zGw/ZF+00wTnztyA9B+O+vgttjHzQr5sUxt4COCo64ASGW0MZ OzldMXmb3Rz+hcxoleqGWjK24eazVzW+GgamhuTFRmUPl3VmycxnzkhIU3kCHlxgnNgs XafGDNQrMh8qExSWLt+UovtIzYOi8p3Uu7uncIQWGXYNA5sYcbvcFe+D3QG46e9RbTFj w3vXQlFNtzXu966d521VmvWHGlEMMUZWT5DN4v29xFBmsVzF5bed4ah4uOrxVL2zlMGC z2RugMnWH4hK0cCbgnnuXZvn5kwF9jkHYkiN4UAgFiJkLNm11eJxtXtmVePD5wZlMY4b vK3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=095DUiz9R55WohpIG0Cj/NcoduVCwSduaLRbsGkQo94=; b=nwG4wtcm3etzrYgGPmWxm7rBEnY5s9zAE7SRi+Nig+d9NcZ6hpNDNLsO4GXRv3IWps ny+twnwwprcbdLDPax/TrjVj75fsoAe2Uq5WNU6zDSas+3nG4d2YbhHlqF1ZWDrj98KG PIMUNo+yLuR6/W/QHd1gO9O/IMc32JoEcjPyYq8idFoKQrXLwDfmKjiMxCW+hcifycsN azP6MIZK3z8zny1Vy4cnPDgWrjlAy7NrijVDNtOhpIVZ/hC4/sYWhWi/cUd6mqfZqJ7U lQVwTypnFP5AYBHN0CP8Q8ntf3CR3Y0oLMnpl4DMha8t2ICfryOuNN23x722ppHWBjwB MtMQ==
X-Gm-Message-State: AOAM533wIt48tTLibBmLUzgHosTVkhYAkeHJCPjYWAY8prVs0nHlxNpk I36GGvrIPNHpMbaY9dJBJkRKLo0408WMWp7g0+U=
X-Google-Smtp-Source: ABdhPJy7348SF6UNMxz2KCc0zUcqIRU6Qbel5xEWaJNS5lajQvg/hXaxJ3Vw0oYfoLq07b26nmGvlIM0Qv9NxTFe47s=
X-Received: by 2002:adf:e0cc:: with SMTP id m12mr4573197wri.62.1632987747820; Thu, 30 Sep 2021 00:42:27 -0700 (PDT)
MIME-Version: 1.0
References: <> <>
In-Reply-To: <>
From: Tal Mizrahi <>
Date: Thu, 30 Sep 2021 10:42:14 +0300
Message-ID: <>
To: Greg Mirsky <>, Tommy Pauly <>, IPPM Chairs <>
Cc: "IETF IPPM WG (" <>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <>
Subject: Re: [ippm] WGLC for IOAM Flags
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 30 Sep 2021 07:42:45 -0000

Hi Greg,

Many thanks for the detailed comments.
Please see the responses below, as well as suggested text edits in some cases.

The authors.

On Wed, Sep 8, 2021 at 1:00 AM Greg Mirsky <> wrote:
> Dear Authors, WG Chairs, and All,
> I've read the latest version of the draft and have several comments and questions to share with you. I would much appreciate the opportunity to discuss these before I can state my support for progressing this draft to publication.
> The Abstract states that:
>    In-situ Operations, Administration, and Maintenance (IOAM) records
>    operational and telemetry information in packets while they traverse
>    a path between two points in the network.
> I think that is not a complete description of IOAM options as it excludes the Direct Export option. It might be easier to improve by s/records/collects/. Also, what, in your opinion, is the distinction between operational and telemetry information? I think that operational information collected from remote nodes is one example of telemetry information. What do you think?

[TM] Agreed. Your proposal makes sense: s/records/collects/
Regarding the difference between operational and telemetry
information, the way I see it "telemetry" refers to measurements that
are being exported to an external/remote entity, whereas operational
information is not measurement-based, such as the Node ID or Interface

> nit s/the the/the/

[TM] Agreed.

> I think that Abstract and Introduction can be improved by explaining the purpose of defining the Loopback and Active flags. What each of them is intended to achieve and improve?

[TM] The point is well-taken, and the following text will be added to
the introduction. In order to avoid text repetition, I believe it
would be best not to add it to the abstract as well.

The Loopback flag is used to request that each transit device along
the path loops back a copy of the data packet to the sender. The
Active flag indicates that a packet is used for active measurement.

> It is stated in the Introduction that:
>    This document defines
>    two new flags in the Pre-allocated and Incremental Trace options: the
>    Loopback and Active flags.
> Does that mean that neither of the two flags is applicable in the Direct Export option? Should draft-ietf-ippm-ioam-direct-export have an explicit statement regarding the applicability of the Loopback and Active flags? And if any is applicable, describe the behavior of an IOAM node receiving it.

[TM] Right.
The direct exporting draft explicitly states the behavior of each
field in the DEX option header including the flags, so there seems to
be no need to specify flags that are supported in other options.

> In Section 4 the purpose of using the Loopback flag is explained as:
>    Loopback allows an IOAM encapsulating node to trace the path to a
>    given destination, and to receive per-hop data about both the forward
>    and the return path.
> In my previous comments, I've pointed out that the Loopback mode is just a special case of the Direct Export option. In addition to that, I would appreciate your explanation of how the IOAM encapsulating node receives per-hop data about the return path. How does that work, for example, in an MPLS or SFC NSH environment?

[TM] It is slightly similar to direct exporting, but is not a special
case of direct exporting. In direct exporting telemetry/operational
data is not pushed into the data packet, but the data packet with the
DEX option triggers the data collection and optionally also exporting,
but exporting is not mandatory and may be performed later, for example
by the management plane. On the other hand, a trace option with the
loopback flag causes all IOAM transit nodes to push IOAM data onto the
trace option, and also loop a copy of the packet back to the sender.

> Further it Section 4, it is stated that:
>    The two IOAM looped back copies are
>    terminated by the encapsulating node.
> But I don't see any technical explanation that could sustain that assumption. As I understand the diagram in figure 1, a transit node doesn't know the identity of the IOAM encapsulating node. Correct? If that is the case, where does the transit node direct the looped packet?

[TM] As described in the draft, the loopback flag is not necessarily
applicable to all possible encapsulations. The way the return path is
defined is encapsulation-specific. The draft currently states:

   Loopback can be used only if a return path from transit nodes and
   destination nodes towards the source (encapsulating node) exists.
   Specifically, loopback is only applicable in encapsulations in which
   the identity of the encapsulating node is available in the
   encapsulation header.  If an encapsulating node receives a looped
   back packet that was not originated from the current encapsulating
   node, the packet is dropped.

> It seems that the text in the last paragraph attempts to walk around my questions. But I don't find that paragraph sufficient. It lacks the normative language to guide a developer to produce resilient implementation. For example, how to interpret:
>    Loopback can be used only if a return path from transit nodes and
>    destination nodes towards the source (encapsulating node) exists.
> That reads like a recommendation, not a requirement. As a result, an implementation may transmit an IOAM packet with the Loopback fag set regardless of whether there's a return path or not. Then, how must a transit node process such an IOAM packet?

[TM] Right, the last response and quoted text is relevant here as
well. Please also note the following text from the draft:

it is assumed that the
   source address is available in the encapsulation header.  Thus, the
   source address of the original packet is used as the destination
   address in the copied packet.

I propose to add the following sentence:
The loopback flag MUST NOT be set if it is not guaranteed that there
is a return path from each of the IOAM transit and IOAM decapsulating
nodes, or if the encapsulating node's address is not available in the
encapsulation header.

> A very concerning statement I've found in Section 4.1
>    The encapsulating node either generates synthetic packets with an
>    IOAM trace option that has the Loopback flag set ...
> IOAM has been defined as a hybrid measurement protocol. This statement implies that it is to be used as an active OAM protocol (all per RFC 7799 definitions). The requirements for active OAM protocols in terms of their applicability and, most of, security are very different from those set for a hybrid measurement protocol. I believe that extending IOAM into an active mode of measurement requires more discussion and the audience from other IETF WGs, e.g., 6man, SFC, BIER, MPLS, BFD, RTGWG.

[TM] This draft does not define an active measurement protocol. The
point in the draft is that the loopback flag can be used in active
measurement packets.
The following text edit is proposed:

   The encapsulating node either generates synthetic packets with an
   IOAM trace option that has the Loopback flag set, or sets the loopack
   flag in a subset of the in-transit data packets.  Loopback is used
   either proactively or on-demand, i.e., when a failure is detected.
   The loopback flag can be set in the IOAM trace option either in a
   subset of the in-transit data packets, or in synthetic packets that
   are generated by the encapsulating node. Loopback can be used
   either proactively or on-demand, i.e., when a failure is detected.

> The next statement explains the use case for the Loopback flag:
>    Loopback is used
>    either proactively or on-demand, i.e., when a failure is detected.
> That is the same use of ICMP and LSP Ping. What is the benefit of using the Loopback flag instead of readily available ICMP? Does it work as L2 Linktrace? I don't see that as a sufficient benefit considering the complexity, exclusions, and risks associated and identified in the draft. It is clear that there are tools like ICMP and LSP Ping that adequately address scenarios listed in the draft. Don't see any technical reason for the introduction of another mechanism that does not have any significant benefit compared to the existing OAM tools. Especially, considering the limited information that is allowed to be collected using the Loopback IOAM flag ("every transit node that processes this trace option only adds a single data field, which is the Hop_Lim and node_id data field"). With the abundance of information and additional verification achieved using ICMP and LSP Ping, there's even less rationale in the introduction of this over-complex and unnatural mechanism.

[TM] Right, as the draft says, the loopback flag is an "...alternative
to Traceroute, that allows the encapsulating node to receive responses
from multiple transit nodes along the path". Indeed, it slightly
resembles ETH-LT (Ethernet Linktrace) in the sense that a single
packet from the encapsulating node allows to quickly trace the entire
path, but as with IOAM in general, using the loopback flag does not
mandate a specific encapsulation.

> The last paragraph states the requirement:
>    IOAM encapsulating nodes MUST NOT push an IOAM encapsulation with the
>    Loopback flag onto data packets that already include an IOAM
>    encapsulation.
> I couldn't find any guidance on how an IOAM node that receives such invalid combination processes the packet.

[TM] The current draft only specifies the flags. Specifically in the
scenario described above, the encapsulating node does not set the
loopback flag. This is intended to reduce the impact of potential
amplification attacks.

> It is assumed that the identity of the IOAM encapsulating node is known to the node that is expected to sent a copy of the received packet back. But what if that is not the case? How must that node act?

[TM] Agreed. The following new text is proposed to be added to Section 4.1:
If the address of the encapsulating node is not available in the
encapsulation header, then the transit/decapsulating node does not
loop back a copy of the original packet.

> I believe that there's inconsistency between Figure 1 and the following text in Section 4.2
>    Thus, the
>    source address of the original packet is used as the destination
>    address in the copied packet.
> It seems that the text assumes that the IOAM encapsulating node adds also a network layer encapsulation. But how would it work, for example, in SFC NSH data plane? Or MPLS?

[TM] Right, the paragraph beginning with "Loopback can be used
only..." (discussed in response to one of the previous comments) makes
that restriction.

> Section 4.4 states that
>    If the Node ID matches the current
>    IOAM node, it indicates that this is a looped back packet that was
>    initiated by the current IOAM node, and processed accordingly.
> But according to the definition of node_id in Section of draft-ietf-ippm-ioam-data, "Node identifier field to uniquely identify a node within the IOAM-Namespace and associated IOAM-Domain." It appears that simply comparing Node IDs is not sufficient to verify that the node received its looped back packet. What is your opinion?

[TM] Agreed. The following update is proposed:
If the Node ID matches
If the Node ID and IOAM-Namespace match

> If, as stated in Section 5,
>    It [Active flag] is not intended as a replacement for existing
>    active OAM protocols, which may run in higher layers and make use of
>    the Active flag.
> Then the question, What is the purpose of the Active flag? Existing active protocols, Fault Management and Performance Monitoring, are identified by, for example, use of a well-known destination port number or, in case of MPLS, Generic Associated Channel type. There's no apparent need of using IOAM option header to identify the payload as a control message of an active OAM protocol. On the contrary, that would likely cause more confusion and challenges to interoperation among different implementations.
> Three cases analyzed in Section 5 require special consideration, as these likely present the motivation for the introduction of the Active flag:
> the first use case is not really using the Active flag at all and thus the text doesn't seem adding any value to the draft;
> it is not clear what is viewed as the benefit of using the Active flag compared with using any of the existing active OAM protocols in the second use case. I would appreciate it if you can clarify that for me. Since an SFC NSH network is used as an example, is it required that a Service Function (SF) or an SF Proxy process the IOAM header with Active flag set? Or would a Service Function Forwarder (SFF) avoid sending a packet with the IOAM header that has an Active flag set? And can you compare that with the procedures defined in the Active OAM for SFC draft?
> My questions for the third use case described in Section 5 are the same as above because I don't see the benefit of using a replicated data packet (less benefits if it is truncated) compared to a special test packet.
> In my conclusion on the Active flag, I don't see any clear benefit of introducing it and using whether in place or in addition to the existing active OAM protocols.

[TM] General response to all the Active flag comments: this was
extensively discussed on the mailing list and in previous IETF
meetings, and your comments about the previous versions of the
document have contributed to greatly improving Section 5, which
describes the main use cases of the Active flag.

> I am looking forward to your feedback and the continued discussion.
> Regards,
> Greg
> On Mon, Aug 30, 2021 at 1:58 PM Tommy Pauly <> wrote:
>> Hello IPPM,
>> This email starts a Working Group Last Call on two related IOAM documents, draft-ietf-ippm-ioam-flags and draft-ietf-ippm-ioam-direct-export.
>> In-situ OAM Loopback and Active Flags
>> In-situ OAM Direct Exporting
>> Please provide feedback by replying to this email with your reviews and if you think these documents are ready to progress, by Wednesday, September 15.
>> Best,
>> Tommy & Ian
>> _______________________________________________
>> ippm mailing list
> _______________________________________________
> ippm mailing list