Re: [ippm] WGLC for IOAM Direct Exports

Tal Mizrahi <> Thu, 30 September 2021 07:18 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 4359D3A1651; Thu, 30 Sep 2021 00:18:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 3a_c4J-G8vpq; Thu, 30 Sep 2021 00:17:58 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::42c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 089123A164F; Thu, 30 Sep 2021 00:17:57 -0700 (PDT)
Received: by with SMTP id t8so8390973wri.1; Thu, 30 Sep 2021 00:17:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=i7pN4jGRVxBNtaFmK3gzbhYlxystYLkiXEQfGKVgpVs=; b=oaGeO79v1lATaIMfTH04fNFwKDSy2Rgey8FvacmfOl9esXNEfETCmQOaOFTB91Ag1D EK8vUG5LzFQ/YoIPpbn9QsKny0ahIVsUBbD1W34l4fjlBoj/J19cie2KjiOTY1AImN7f Y39TIWGaaqNF3F6KDjj2O8rfxCE6yf1gOn5Ck3d2rLBWglQ5kPHITQq9zkr9axuX5zW7 ZODOcxZRkFlnyDpGmHCkiurHUz1UExbdR/NYXag5FvInrjfYt4P889gFP/rWbFC7U8pi goq7/mUjGPMaXcLB5TLiwJhGeCK0JbWlQ9YBc5Qg5bbQWJMYvr0kXdbgj31vl3lUzULF OiQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=i7pN4jGRVxBNtaFmK3gzbhYlxystYLkiXEQfGKVgpVs=; b=HwuBS8SMhYh7VSJ7qCo/kuIaGlIv6LefDLHncqThAh/YlBkF5f3wLz6KHfrGnxYqOy 5+MpQxF2HMFysnwls8h+v9TEeZvpwU3QNyKQ1e2YnZDMfvPqYxgcrFZ2TWzoKwTfMi51 mpdgOB8MBIh0dcHeRCmI00t9MnospGXPpahwIYMZXMa4ov7f6TOKAGpXhi4JZlMcFhqH zxQdPUneOV3/N7QApGVBjsvu3XFg8gRIt+Wx5zos5ZoJHB3voYyXKnEssa8eTwV9EKi8 unHoI73hDJNuipcanmc8HbN18khSSOZERWG8HzbzE2uQsvtkuZdo2xi9J1vmxqTmQt+Z MbTw==
X-Gm-Message-State: AOAM533faRM0d/PUA9Itx4b3dr+s4opSpKZwDtCDcJinNiXfog8nZlax 6TL/uDtuVJ8iSQ1rbWE9DKpSgD0zVMig1XiNDYgSMCQwOds=
X-Google-Smtp-Source: ABdhPJxr+ThmsskLa3082jNu3t1+lamcbJLhXhI+XqUBqB6rl5ki9XAnbxvjKfmRAV+pBW6yBj9wVdtoW5O9f9Q3SxY=
X-Received: by 2002:adf:e0cc:: with SMTP id m12mr4460133wri.62.1632986276176; Thu, 30 Sep 2021 00:17:56 -0700 (PDT)
MIME-Version: 1.0
References: <> <>
In-Reply-To: <>
From: Tal Mizrahi <>
Date: Thu, 30 Sep 2021 10:17:43 +0300
Message-ID: <>
To: Greg Mirsky <>, Tommy Pauly <>, IPPM Chairs <>
Cc: "IETF IPPM WG (" <>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <>
Subject: Re: [ippm] WGLC for IOAM Direct Exports
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 30 Sep 2021 07:18:02 -0000

Hi Greg,

Many thanks for the thorough review.
Below you can find responses to your comments, and in some cases
proposed text edits.

If there are no further comments we will update the document accordingly.

The authors.

On Wed, Sep 8, 2021 at 1:01 AM Greg Mirsky <> wrote:
> Dear Authors, WG Chairs, and All,
> I've read the latest version of the draft and have several comments and questions to share with you. None of these are blocking, I support progressing this draft to publication. Please consider the notes below as an invitation to a discussion:
> It seems that the characterization of the Direct Export IOAM option in the following is not entirely accurate:
> The DEX option is used as a trigger to collect and/or export IOAM data.
> I propose re-wording:
> The DEX option is used as a trigger to collect and export or only collect locally the IOAM data.
> Using "exporting and/or collecting" reflects the use of the Direct Export option type technically accurate but the order of actions is backward. What are your thoughts?

[TM] Right. The following text edit is proposed in the introduction.

   This option is used as
   a trigger for IOAM nodes to locally aggregate and process IOAM data,
   and/or to export it to a receiving entity (or entities).
   This option is used as
   a trigger for IOAM nodes to locally aggregate and process IOAM data,
   and/or to export it to a receiving entity (or entities). Throughout
   the document this functionality is referred to as collection and/or

> I find references to RFC 5475 and 7014 very appropriate and helpful. But I don't see these RFCs being referenced in draft-ietf-ippm-ioam-data. I think that excessive use of not only the Direct Export IAOM option but the IOAM method, in general, may adversely affect a network and the recommendations listed in RFC 5475 and 7014 are equally applicable.
> Similarly, the good and helpful discussion of how to practically control the use of the Direct Export option is, in my opinion, equally applicable to other IOAM trace options and should be in the Security Considerations section of draft-ietf-ippm-ioam-data.

[TM] The last two comments are about draft-ietf-ippm-ioam-data, which
is currently in IESG review. I will take this feedback with the other
authors to see how we can address this input at this point.

> The second paragraph in Section 3.1.2 is re-phrasing if not repeating what is already expressed in the last paragraph of the previous sub-section (though the limit is applied to exporting telemetry information). I think that it can be removed without any loss to the document.

[TM] The second par of 3.1.2 refers to *exporting*, while the last par
of 3.1.1 refers to *encapsulating*, so these are two complementary

> I think that it will be helpful to provide some recommendations, i.e., SHOULD do this, how an implementation handles the excess of packets to export. Drop silently, drop and log notification, or try to shape that flow?

[TM] Assuming an amplification scenario, a large number of packets may
be dropped, so logging each drop may defeat the purpose. A drop
counter may be a good idea here, but this document does not really
define counter requirements in general, so it seems like more of an
implementation-specific aspect, or possibly can be considered as part
of the YANG model.

> I was really puzzled by this requirement in Section 3.1.2
>    Exported packets SHOULD NOT be exported over a path or a tunnel that
>    is subject to IOAM direct exporting.
> I hope you can clarify it for me.

[TM] As explained later in that paragraph: "This requirement is
intended to prevent nested exporting and/or exporting loops."
This was thoroughly discussed in the working group, and came out of
the security-related review.

> This requirement
>    Furthermore, IOAM encapsulating
>    nodes that can identify a packet as an IOAM exported packet MUST NOT
>    push a DEX option into such a packet.
> brings up questions:
> How can a node identify a packet as the IOAM exported packet?

[TM] The requirement means that if an encapsulating node's ACL can
detect exported packets, then DEX should not be applied.

> What should be done if the IOAM exported packet contains the IOAM header with the DEX option?

[TM] The current document does not require an IOAM transit node to
parse beyond the (external) IOAM encapsulation if there are nested
IOAM encapsulations.

> What is special about the decapsulating node that is not DEX-capable?
>    A decapsulating node that does not support the DEX option
>    MUST remove it, along with any other IOAM options carried in the
>    packet if such exist.
> Shouldn't that be a general behavior of a decapsulating IOAM node?

[TM] Right. The following text edit is proposed:

   A transit IOAM node that does not support the DEX option SHOULD
   ignore it.  A decapsulating node that does not support the DEX option
   MUST remove it, along with any other IOAM options carried in the
   packet if such exist.
   A transit or decapsulating IOAM node that does not support the DEX
option SHOULD
   ignore it.  Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
   decapsulating node removes the IOAM encapsulation and all its IOAM options,
   and specifically in the case where one of these options is a
(possibly unknown)
   DEX option.

> The format of the first four-octet word in Figure 2 is different from the format defined in draft-ietf-ippm-ioam-data:
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |        Namespace-ID           |NodeLen  | Flags | RemainingLen|
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |               IOAM-Trace-Type                 |  Reserved     |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Is this intentional? If it is, how does an implementation know which format to use?

[TM] Each IOAM Option-Type has a different header format. The current
document defines a new Option-Type, which was not defined in
The IOAM-Type field in the lower layer header determines which
Option-Type is used and how its header should be parsed.

> It is stated in the draft that:
> The length of the DEX option is at least 8 octets.
> But the figure 2 in the draft does not include any field that reflects the length of the IOAM option. Is that because the figure is out of sync? If not, how can an implementation verify the length? Also, what is the behavior if the length is less than 8 octets?

[TM] Quoting the draft:

   The existence of the optional fields is
   indicated by the corresponding flags in the Extension-Flags field.
   Thus, the Extension-Flags field explicitly indicates
   the length of the DEX option.

> RE: PerformanceConsiderations
> I agree with you that a DEX-encapsulating node and DEX-transit nodes must have rate-limiting controls (as noted earlier, the behavior of the transit node needs more detailed specification). But I think that the document should also note that exported packets could be sent over the network out-of-band, relative to the monitored flow. As a result, the impact of packets carrying telemetry information might be not direct and harmful to the data flow. I see that as one of the important benefits of the Direct Export method transporting on-path telemetry information.

[TM] Right. The following new text is proposed:

It should be noted that in some networks DEX data may be exported over
an out-of-band network, in which a large volume of exported traffic
does not compromise user traffic. In this case an operator may choose
to disable the exporting rate limiting.

> Regards,
> Greg
> On Mon, Aug 30, 2021 at 1:58 PM Tommy Pauly <> wrote:
>> Hello IPPM,
>> This email starts a Working Group Last Call on two related IOAM documents, draft-ietf-ippm-ioam-flags and draft-ietf-ippm-ioam-direct-export.
>> In-situ OAM Loopback and Active Flags
>> In-situ OAM Direct Exporting
>> Please provide feedback by replying to this email with your reviews and if you think these documents are ready to progress, by Wednesday, September 15.
>> Best,
>> Tommy & Ian
>> _______________________________________________
>> ippm mailing list
> _______________________________________________
> ippm mailing list