Re: [ippm] WGLC for IOAM Direct Exports

Tal Mizrahi <tal.mizrahi.phd@gmail.com> Wed, 13 October 2021 12:12 UTC

Return-Path: <tal.mizrahi.phd@gmail.com>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7F1EA3A08AA; Wed, 13 Oct 2021 05:12:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level:
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wMnydZ_QVlQy; Wed, 13 Oct 2021 05:12:05 -0700 (PDT)
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7C49B3A089C; Wed, 13 Oct 2021 05:12:05 -0700 (PDT)
Received: by mail-wr1-x42d.google.com with SMTP id t2so7604698wrb.8; Wed, 13 Oct 2021 05:12:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=a7I9uykP89YBcmpLV5oxjgUaQmlCJxEZ7+nMAdRrPoo=; b=LxEQfIvzGVsVVSlcw9UlIdpqmTLUIrJLdlp9OS203If0uhNH0hr6ApjJVXsBq+HrG4 1Hjl5/bqKAK2u3+LaD2QVrNCLzGxBcAon5eNJb9Y2t9JcPC083g7lMKBNtJR7f8S/wSl ubsVJlTXhiL8rvriIos6r9EWFZQN2ZHz+G+/m6/vtKy0pGdUS9R2k6EYtPK8VdyeiHpI YAG/ZfDz09KL6cAXoIUOkO3IIEq8G9oSHdpYm8q4dvSQxOwZy6oanhvmcvM/cLLRK/aD IP87BdUSCEKUGQ3Y6cwOijpFUZUqpVDikgrf/UfABJ/bmkX7pj7t0SywmAkIDHkhlRYG 0ltg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=a7I9uykP89YBcmpLV5oxjgUaQmlCJxEZ7+nMAdRrPoo=; b=5diRxoB84y+aPWmCtdLJ1+h61ICc2IKodYGVUUzVHZMphVrYA5XLYCeNlxXoiMsFVF LsiadFcSUTTfHTcx9W7u7SXyzoPd5pKOTfAvi5WgT5mYyd/tBVMVwcmQ8No9B2rFO1s0 tfXXCTZGAUpH8tTOaC/rVogr2TahKGK6/K/BINFqxfZ1+ZkqwxdTGvReU3L0IoWKKeer WHRY4+ikve889FGD9v0fCquFDhMqmlNhX2j95JWYzN4OGiW1Fb+r4nr5yy/hx6pVag2G 6u2C/MV853HM4VN7dk6M9ovwyPyBvKEKIyTZviOVhOz6sZz/667ssfa3i6mI71/zLgR5 WRAw==
X-Gm-Message-State: AOAM531VUynIBg3+teP7kTt9HAbmMpPcLhj7Yy3NuiuNWEBNTT5tgidq EWSdBHrTLFGe6GCjvxws9ZeIzyG8DF6KTcOoOVYOAUKD4S5Yvw==
X-Google-Smtp-Source: ABdhPJwYMGgjEXFSK431UyZONn3UgYneTPoULtw5PqYpBlgULLRk6hPzM17az+Kjq3NEiNnRUt+ctVW6cDLH18/esBY=
X-Received: by 2002:a05:600c:21cc:: with SMTP id x12mr4041399wmj.8.1634127122950; Wed, 13 Oct 2021 05:12:02 -0700 (PDT)
MIME-Version: 1.0
References: <A4F59F82-7F47-452B-9E63-0FDFBB812CEC@apple.com> <CA+RyBmW3nKqMftmEZ+e=7OcZ_jukc71BHBt0BgVCxB8S9QvCRQ@mail.gmail.com> <CABUE3XnQLraizDGnH7NhYO9jfTbN_-G01RwnxH1H9xMitv+mUA@mail.gmail.com> <CA+RyBmVACcB6+W+x7o07y0k93otzxf6=F2fR5NOcND36WVp_zg@mail.gmail.com>
In-Reply-To: <CA+RyBmVACcB6+W+x7o07y0k93otzxf6=F2fR5NOcND36WVp_zg@mail.gmail.com>
From: Tal Mizrahi <tal.mizrahi.phd@gmail.com>
Date: Wed, 13 Oct 2021 15:11:49 +0300
Message-ID: <CABUE3X=Tj_2tT_5EmXNg-sDG+3An80Qir-hvusDaXHfBj3Hzfw@mail.gmail.com>
To: IPPM Chairs <ippm-chairs@ietf.org>, Tommy Pauly <tpauly=40apple.com@dmarc.ietf.org>
Cc: "IETF IPPM WG (ippm@ietf.org)" <ippm@ietf.org>, Greg Mirsky <gregimirsky@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/-WpgvLhelp4zZbkovfKHYOd2tv4>
Subject: Re: [ippm] WGLC for IOAM Direct Exports
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Oct 2021 12:12:11 -0000

Dear Chairs,

We want to thank Greg for his thorough review and useful comments.
Although these comments were defined by Greg as non-blocking we made
an effort to address them either by updating the draft or by
responding to Greg's comments.
Some further responses can be found below, marked [[TM]].

We have posted an updated version (07) that seems to address these
comments, as well as comments received from Bernard Aboba (TSV-ART
reviewer).
https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/


We believe the draft is ready to be submitted to the IESG.

Thanks,
Authors.



On Sun, Oct 10, 2021 at 8:23 PM Greg Mirsky <gregimirsky@gmail.com> wrote:
>
> Hi Tal,
> thank you for carefully considering my comments and suggestions. Please find my follow-up notes in-lined below under the GIM>> tag.
>
> Regards,
> Greg
>
> On Thu, Sep 30, 2021 at 12:17 AM Tal Mizrahi <tal.mizrahi.phd@gmail.com> wrote:
>>
>> Hi Greg,
>>
>> Many thanks for the thorough review.
>> Below you can find responses to your comments, and in some cases
>> proposed text edits.
>>
>> If there are no further comments we will update the document accordingly.
>>
>> Thanks,
>> The authors.
>>
>>
>> On Wed, Sep 8, 2021 at 1:01 AM Greg Mirsky <gregimirsky@gmail.com> wrote:
>> >
>> > Dear Authors, WG Chairs, and All,
>> > I've read the latest version of the draft and have several comments and questions to share with you. None of these are blocking, I support progressing this draft to publication. Please consider the notes below as an invitation to a discussion:
>> >
>> > It seems that the characterization of the Direct Export IOAM option in the following is not entirely accurate:
>> >
>> > The DEX option is used as a trigger to collect and/or export IOAM data.
>> >
>> > I propose re-wording:
>> > The DEX option is used as a trigger to collect and export or only collect locally the IOAM data.
>> >
>> > Using "exporting and/or collecting" reflects the use of the Direct Export option type technically accurate but the order of actions is backward. What are your thoughts?
>> >
>>
>> [TM] Right. The following text edit is proposed in the introduction.
>>
>> OLD:
>>    This option is used as
>>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
>>    and/or to export it to a receiving entity (or entities).
>> NEW:
>>    This option is used as
>>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
>>    and/or to export it to a receiving entity (or entities). Throughout
>>    the document this functionality is referred to as collection and/or
>> exporting.
>
> GIM>> As I understand it, it is the local policy that controls the behavior of the node receiving the IOAM DEX indicator. Similar to IPFIX, the information can be aggregated and exported as a calculated metric, e.g., percentile or mean value, or exported as soon as possible as recorded. The new text is certainly better but it seems it does not express the role of the local policy in defining the processing behavior, the transport to be used, and the identity of the Collector. One of possible ways could be to state that the definition of all these controls is outside the scope of the document.

[[TM]] Right. The following text edit is proposed.
OLD:
Note that even though the IOAM Option-Type is called "Direct Export",
      it depends on the deployment whether the receipt of a packet with DEX
      option type leads to the creation of another packet. Some deployments
      might simply use the packet with the DEX option type to trigger local
      processing of OAM data.
NEW:
Note that even though the IOAM Option-Type is called "Direct Export",
      it depends on the deployment whether the receipt of a packet with DEX
      option type leads to the creation of another packet. Some deployments
      might simply use the packet with the DEX option type to trigger local
      processing of OAM data. The functionality of this local processing is not
  within the scope of this document.

>>
>>
>>
>> > I find references to RFC 5475 and 7014 very appropriate and helpful. But I don't see these RFCs being referenced in draft-ietf-ippm-ioam-data. I think that excessive use of not only the Direct Export IAOM option but the IOAM method, in general, may adversely affect a network and the recommendations listed in RFC 5475 and 7014 are equally applicable.
>> > Similarly, the good and helpful discussion of how to practically control the use of the Direct Export option is, in my opinion, equally applicable to other IOAM trace options and should be in the Security Considerations section of draft-ietf-ippm-ioam-data.
>>
>>
>> [TM] The last two comments are about draft-ietf-ippm-ioam-data, which
>> is currently in IESG review. I will take this feedback with the other
>> authors to see how we can address this input at this point.
>
> GIM>> Thanks.
>>
>>
>>
>> > The second paragraph in Section 3.1.2 is re-phrasing if not repeating what is already expressed in the last paragraph of the previous sub-section (though the limit is applied to exporting telemetry information). I think that it can be removed without any loss to the document.
>>
>> [TM] The second par of 3.1.2 refers to *exporting*, while the last par
>> of 3.1.1 refers to *encapsulating*, so these are two complementary
>> requirements.
>
> GIM>> Is it the intention of this draft to define the encapsulation of exported information? What if the local policy mandates the node calculating p95 and p99 of packet delay and inter-packet delay variation? Can you help me by describing how these metrics are encapsulated?

[[TM]] As emphasized in the document: "The exporting method and format
are outside the scope of this document."
Section 3.1.2 actually refers (as its name suggests) to "Responding to
the DEX Trigger".

>>
>>
>>
>> > I think that it will be helpful to provide some recommendations, i.e., SHOULD do this, how an implementation handles the excess of packets to export. Drop silently, drop and log notification, or try to shape that flow?
>>
>> [TM] Assuming an amplification scenario, a large number of packets may
>> be dropped, so logging each drop may defeat the purpose. A drop
>> counter may be a good idea here, but this document does not really
>> define counter requirements in general, so it seems like more of an
>> implementation-specific aspect, or possibly can be considered as part
>> of the YANG model.
>
> GIM>> I am still struggling to understand how IOAM DEX triggers amplification. Can you add more details, steps to help me understand it?

[[TM]] This is explained in the document:

   The amplification effect of DEX may be worse in wide area networks in
   which there are multiple IOAM domains.  For example, if DEX is used
   in IOAM domain 1 for exporting IOAM data to a receiving entity, then
   the exported packets of domain 1 can be forwarded through IOAM domain
   2, in which they are subject to DEX.  The exported packets of domain
   2 may in turn be forwarded through another IOAM domain (or through
   domain 1), and theoretically this recursive amplification may
   continue infinitely.


>>
>>
>> > I was really puzzled by this requirement in Section 3.1.2
>> >
>> >    Exported packets SHOULD NOT be exported over a path or a tunnel that
>> >
>> >    is subject to IOAM direct exporting.
>> >
>> > I hope you can clarify it for me.
>>
>>
>> [TM] As explained later in that paragraph: "This requirement is
>> intended to prevent nested exporting and/or exporting loops."
>> This was thoroughly discussed in the working group, and came out of
>> the security-related review.
>
> GIM>> If the definition of exporting, including the encapsulation, choice of transport, and the Collector is outside the scope, then this statement, in my opinion, should be removed. If all these issues are in the scope, then  there is more, much more information that is missing.

[[TM]] As explained in the abstract, "The exporting method and format
are outside the scope of this document."
However, the current document has a paragraph that defines security
requirements for the export method. These requirements came out of the
very detailed security discussions we had with Martin Duke and Mirja
K├╝hlewind.

>>
>>
>> >
>> > This requirement
>> >
>> >    Furthermore, IOAM encapsulating
>> >    nodes that can identify a packet as an IOAM exported packet MUST NOT
>> >    push a DEX option into such a packet.
>> > brings up questions:
>> >
>> > How can a node identify a packet as the IOAM exported packet?
>>
>> [TM] The requirement means that if an encapsulating node's ACL can
>> detect exported packets, then DEX should not be applied.
>
> GIM>> I think that whether this belongs in the document depends on the answer to my question above - what is in the scope of this document.

[[TM]] See the response above.

>>
>>
>> > What should be done if the IOAM exported packet contains the IOAM header with the DEX option?
>>
>> [TM] The current document does not require an IOAM transit node to
>> parse beyond the (external) IOAM encapsulation if there are nested
>> IOAM encapsulations.
>
> GIM>> Same, as above - need to clarify the scope of the document first.
>>
>>
>> >
>> > What is special about the decapsulating node that is not DEX-capable?
>> >
>> >    A decapsulating node that does not support the DEX option
>> >    MUST remove it, along with any other IOAM options carried in the
>> >    packet if such exist.
>> > Shouldn't that be a general behavior of a decapsulating IOAM node?
>>
>> [TM] Right. The following text edit is proposed:
>>
>> OLD:
>>    A transit IOAM node that does not support the DEX option SHOULD
>>    ignore it.  A decapsulating node that does not support the DEX option
>>    MUST remove it, along with any other IOAM options carried in the
>>    packet if such exist.
>> NEW:
>>    A transit or decapsulating IOAM node that does not support the DEX
>> option SHOULD
>>    ignore it.  Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
>>    decapsulating node removes the IOAM encapsulation and all its IOAM options,
>>    and specifically in the case where one of these options is a
>> (possibly unknown)
>>    DEX option.
>
> GIM>> Why is it SHOULD? In what situation a node MAY act on the DEX?

Right.
Proposed text:
          A transit or decapsulating IOAM node that receives an
unknown IOAM option
          type ignores it (as defined in <xref
target="I-D.ietf-ippm-ioam-data"/>), and
          specifically nodes that do not support the DEX option ignore it.
          Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
          decapsulating node removes the IOAM encapsulation and all
its IOAM options,
          and specifically in the case where one of these options is a
          (possibly unknown) DEX option.

>>
>>
>>
>> >
>> > The format of the first four-octet word in Figure 2 is different from the format defined in draft-ietf-ippm-ioam-data:
>> >
>> >     0                   1                   2                   3
>> >     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> >    |        Namespace-ID           |NodeLen  | Flags | RemainingLen|
>> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> >    |               IOAM-Trace-Type                 |  Reserved     |
>> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> > Is this intentional? If it is, how does an implementation know which format to use?
>>
>> [TM] Each IOAM Option-Type has a different header format. The current
>> document defines a new Option-Type, which was not defined in
>> draft-ietf-ippm-ioam-data.
>> The IOAM-Type field in the lower layer header determines which
>> Option-Type is used and how its header should be parsed.
>
> GIM>> Thank you for the clarification. Is that the intention to start new drafts to define the DEX IOAM Option-Type separately?

[[TM]] Right. IOAM-Type values are defined in per encapsulation
drafts, such as draft-ietf-ippm-ioam-ipv6-options.


>>
>>
>>
>> >
>> > It is stated in the draft that:
>> >
>> > The length of the DEX option is at least 8 octets.
>> >
>> > But the figure 2 in the draft does not include any field that reflects the length of the IOAM option. Is that because the figure is out of sync? If not, how can an implementation verify the length? Also, what is the behavior if the length is less than 8 octets?
>> >
>>
>> [TM] Quoting the draft:
>>
>>    The existence of the optional fields is
>>    indicated by the corresponding flags in the Extension-Flags field.
>>    ...
>>    Thus, the Extension-Flags field explicitly indicates
>>    the length of the DEX option.
>>
>>
>> > RE: PerformanceConsiderations
>> >
>> > I agree with you that a DEX-encapsulating node and DEX-transit nodes must have rate-limiting controls (as noted earlier, the behavior of the transit node needs more detailed specification). But I think that the document should also note that exported packets could be sent over the network out-of-band, relative to the monitored flow. As a result, the impact of packets carrying telemetry information might be not direct and harmful to the data flow. I see that as one of the important benefits of the Direct Export method transporting on-path telemetry information.
>> >
>>
>> [TM] Right. The following new text is proposed:
>>
>> It should be noted that in some networks DEX data may be exported over
>> an out-of-band network, in which a large volume of exported traffic
>> does not compromise user traffic. In this case an operator may choose
>> to disable the exporting rate limiting.
>
> GIM>> I think that the text can be stronger, recommending that information is exported out-of-band and explaining the interpretation of out-of-band in the case of IOAM.
>>
>>
>> >
>> > Regards,
>> > Greg
>> >
>> > On Mon, Aug 30, 2021 at 1:58 PM Tommy Pauly <tpauly=40apple.com@dmarc.ietf.org> wrote:
>> >>
>> >> Hello IPPM,
>> >>
>> >> This email starts a Working Group Last Call on two related IOAM documents, draft-ietf-ippm-ioam-flags and draft-ietf-ippm-ioam-direct-export.
>> >>
>> >> In-situ OAM Loopback and Active Flags
>> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/
>> >> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06
>> >>
>> >> In-situ OAM Direct Exporting
>> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/
>> >> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06
>> >>
>> >> Please provide feedback by replying to this email with your reviews and if you think these documents are ready to progress, by Wednesday, September 15.
>> >>
>> >> Best,
>> >> Tommy & Ian
>> >>
>> >> _______________________________________________
>> >> ippm mailing list
>> >> ippm@ietf.org
>> >> https://www.ietf.org/mailman/listinfo/ippm
>> >
>> > _______________________________________________
>> > ippm mailing list
>> > ippm@ietf.org
>> > https://www.ietf.org/mailman/listinfo/ippm