Re: [ippm] WGLC for IOAM Direct Exports

Greg Mirsky <gregimirsky@gmail.com> Sat, 16 October 2021 23:09 UTC

Return-Path: <gregimirsky@gmail.com>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 986BB3A0C5A; Sat, 16 Oct 2021 16:09:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NtgB1Pxj7VPK; Sat, 16 Oct 2021 16:09:45 -0700 (PDT)
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D026B3A0C57; Sat, 16 Oct 2021 16:09:44 -0700 (PDT)
Received: by mail-ed1-x533.google.com with SMTP id 5so23570535edw.7; Sat, 16 Oct 2021 16:09:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AE2NZKOydmcrCsm1GX46L4KpCnJv2ilp2NqYyCHW1wc=; b=DiVQzcsuKBa3aEBNsTzyt8O1PEYYXbW9OB8ROt2lE212Jqhof7Hs9dRkFphFweOYo9 HGaexN7Y5qECtJCQXFLlktbKSLxZTw1ukG0xtONZtHLjKpMw9zJMMAl3WO0h4EESTI+b cgZFsv3ZPXqZJ/qkYoVqCVdD7xXqydHOqk3iGXgFkKmYqyxLl4SpBoJNXpC8AoEFjcF5 xvKAj2gJeCB6b4TwFUp21HCeKUfkhUMTtsxG6k3s1QO+naemKT/QJ8J8HE1Dt7uEuIrn dAe6c4w1FP5pgX+Ksck6QTGroOPT891B5rI0GYJofLXtobHnKt++Op0zN4wkCFlaXiTU bLjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AE2NZKOydmcrCsm1GX46L4KpCnJv2ilp2NqYyCHW1wc=; b=8PkLWH71I02ng8wVVKVQ7B9ogtdcHkh59BUjlboLqCBXd6uW4+UMX8DqapfCcWxWYJ Of8EhCsSvbjN4+Mvo/h3Ez5fkal+ta8VNaViX8YtBWrQuYPhGQGU4fUYbL7vbtZvFiv1 bp1Iq4T5wvPvpkqfZSCospDq2YX/lHyVjxAdhjmkA5V5kcUdCMpSzthRm8IXb2Iw0Tjm 7JvFWZvQojEm7M3B8apqo2W6tg0jIb9HrULXgCGZ+ZW+MFg/Smw9/TZD/SYGEzTXxxfk hS/Jy/PMOMxKzTbzVde2S7HIfZyKDl9x8DKWykhAugFgOpEz+/iUp1tyCM/ih+mhRaKl Y+og==
X-Gm-Message-State: AOAM532ud2s+30re++qrcaOJBvcyrInRjET4htfc18Of4vVbz9qcxAVJ q8NVbcWH3ODTaBixN15uqqsmp30O8KK7Ay/TPeU=
X-Google-Smtp-Source: ABdhPJyqQup/quK3P4A/LUAIVWWGB4hujjgxyC+DHMFllUlb+YNHCclRZCLjiADqA6gBBZuWIbFYfxZuftv5sOLEx9w=
X-Received: by 2002:a50:e106:: with SMTP id h6mr30510102edl.295.1634425782637; Sat, 16 Oct 2021 16:09:42 -0700 (PDT)
MIME-Version: 1.0
References: <A4F59F82-7F47-452B-9E63-0FDFBB812CEC@apple.com> <CA+RyBmW3nKqMftmEZ+e=7OcZ_jukc71BHBt0BgVCxB8S9QvCRQ@mail.gmail.com> <CABUE3XnQLraizDGnH7NhYO9jfTbN_-G01RwnxH1H9xMitv+mUA@mail.gmail.com> <CA+RyBmVACcB6+W+x7o07y0k93otzxf6=F2fR5NOcND36WVp_zg@mail.gmail.com> <CABUE3X=Tj_2tT_5EmXNg-sDG+3An80Qir-hvusDaXHfBj3Hzfw@mail.gmail.com>
In-Reply-To: <CABUE3X=Tj_2tT_5EmXNg-sDG+3An80Qir-hvusDaXHfBj3Hzfw@mail.gmail.com>
From: Greg Mirsky <gregimirsky@gmail.com>
Date: Sat, 16 Oct 2021 16:09:31 -0700
Message-ID: <CA+RyBmXHV-z3zbN1H4JawDsvcUwPmMbk_AFKkjAU+4JqknJh4Q@mail.gmail.com>
To: Tal Mizrahi <tal.mizrahi.phd@gmail.com>
Cc: IPPM Chairs <ippm-chairs@ietf.org>, Tommy Pauly <tpauly=40apple.com@dmarc.ietf.org>, "IETF IPPM WG (ippm@ietf.org)" <ippm@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000002f968e05ce8068ed"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/u-FEIxLqc2Z4Y_yDCQLtgGyX1l8>
Subject: Re: [ippm] WGLC for IOAM Direct Exports
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Oct 2021 23:09:51 -0000

Hi Tal,
thank you for posting a new version with updates resulting from our
discussion. I greatly appreciate the consideration you extended to my
comments. As I've mentioned, my comments are non-blocking further progress
of this document. I've read the new version and have some suggestions you
can consider for the next update of the draft:

   - I think that the added text can be a recommendation as the out-of-band
   transport has a lesser adverse impact on the customer traffic:

           It should be noted that in some networks DEX data may be exported
     over an out-of-band network, in which a large volume of exported
     traffic does not compromise user traffic.


   - I propose a minor editorial update as follows:

OLD TEXT:
           In this case an operator
     may choose to disable the exporting rate limiting.
NEW TEXT:
           In this case an operator
     may choose to relax the exporting rate limiting.

   - A new requirement added to the Security Consideration is very useful
   but it seems to suggest a particular implementation:

           Furthermore, an IOAM node MUST
     gain explicit consent to export data to a receiving entity before
     starting to send exported data.

I can imagine that an orchestrator, not an exporting telemetry information
node, obtains the required consent. The sentence might be re-worded as
follows:

           Furthermore, explicit consent to export data to a receiving
entity
           MUST be obtained before activating the IOAM Direct Export trace
           option.

Regards,
Greg

On Wed, Oct 13, 2021 at 5:12 AM Tal Mizrahi <tal.mizrahi.phd@gmail.com>
wrote:

> Dear Chairs,
>
> We want to thank Greg for his thorough review and useful comments.
> Although these comments were defined by Greg as non-blocking we made
> an effort to address them either by updating the draft or by
> responding to Greg's comments.
> Some further responses can be found below, marked [[TM]].
>
> We have posted an updated version (07) that seems to address these
> comments, as well as comments received from Bernard Aboba (TSV-ART
> reviewer).
> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/
>
>
> We believe the draft is ready to be submitted to the IESG.
>
> Thanks,
> Authors.
>
>
>
> On Sun, Oct 10, 2021 at 8:23 PM Greg Mirsky <gregimirsky@gmail.com> wrote:
> >
> > Hi Tal,
> > thank you for carefully considering my comments and suggestions. Please
> find my follow-up notes in-lined below under the GIM>> tag.
> >
> > Regards,
> > Greg
> >
> > On Thu, Sep 30, 2021 at 12:17 AM Tal Mizrahi <tal.mizrahi.phd@gmail.com>
> wrote:
> >>
> >> Hi Greg,
> >>
> >> Many thanks for the thorough review.
> >> Below you can find responses to your comments, and in some cases
> >> proposed text edits.
> >>
> >> If there are no further comments we will update the document
> accordingly.
> >>
> >> Thanks,
> >> The authors.
> >>
> >>
> >> On Wed, Sep 8, 2021 at 1:01 AM Greg Mirsky <gregimirsky@gmail.com>
> wrote:
> >> >
> >> > Dear Authors, WG Chairs, and All,
> >> > I've read the latest version of the draft and have several comments
> and questions to share with you. None of these are blocking, I support
> progressing this draft to publication. Please consider the notes below as
> an invitation to a discussion:
> >> >
> >> > It seems that the characterization of the Direct Export IOAM option
> in the following is not entirely accurate:
> >> >
> >> > The DEX option is used as a trigger to collect and/or export IOAM
> data.
> >> >
> >> > I propose re-wording:
> >> > The DEX option is used as a trigger to collect and export or only
> collect locally the IOAM data.
> >> >
> >> > Using "exporting and/or collecting" reflects the use of the Direct
> Export option type technically accurate but the order of actions is
> backward. What are your thoughts?
> >> >
> >>
> >> [TM] Right. The following text edit is proposed in the introduction.
> >>
> >> OLD:
> >>    This option is used as
> >>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
> >>    and/or to export it to a receiving entity (or entities).
> >> NEW:
> >>    This option is used as
> >>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
> >>    and/or to export it to a receiving entity (or entities). Throughout
> >>    the document this functionality is referred to as collection and/or
> >> exporting.
> >
> > GIM>> As I understand it, it is the local policy that controls the
> behavior of the node receiving the IOAM DEX indicator. Similar to IPFIX,
> the information can be aggregated and exported as a calculated metric,
> e.g., percentile or mean value, or exported as soon as possible as
> recorded. The new text is certainly better but it seems it does not express
> the role of the local policy in defining the processing behavior, the
> transport to be used, and the identity of the Collector. One of possible
> ways could be to state that the definition of all these controls is outside
> the scope of the document.
>
> [[TM]] Right. The following text edit is proposed.
> OLD:
> Note that even though the IOAM Option-Type is called "Direct Export",
>       it depends on the deployment whether the receipt of a packet with DEX
>       option type leads to the creation of another packet. Some deployments
>       might simply use the packet with the DEX option type to trigger local
>       processing of OAM data.
> NEW:
> Note that even though the IOAM Option-Type is called "Direct Export",
>       it depends on the deployment whether the receipt of a packet with DEX
>       option type leads to the creation of another packet. Some deployments
>       might simply use the packet with the DEX option type to trigger local
>       processing of OAM data. The functionality of this local processing
> is not
>   within the scope of this document.
>
> >>
> >>
> >>
> >> > I find references to RFC 5475 and 7014 very appropriate and helpful.
> But I don't see these RFCs being referenced in draft-ietf-ippm-ioam-data. I
> think that excessive use of not only the Direct Export IAOM option but the
> IOAM method, in general, may adversely affect a network and the
> recommendations listed in RFC 5475 and 7014 are equally applicable.
> >> > Similarly, the good and helpful discussion of how to practically
> control the use of the Direct Export option is, in my opinion, equally
> applicable to other IOAM trace options and should be in the Security
> Considerations section of draft-ietf-ippm-ioam-data.
> >>
> >>
> >> [TM] The last two comments are about draft-ietf-ippm-ioam-data, which
> >> is currently in IESG review. I will take this feedback with the other
> >> authors to see how we can address this input at this point.
> >
> > GIM>> Thanks.
> >>
> >>
> >>
> >> > The second paragraph in Section 3.1.2 is re-phrasing if not repeating
> what is already expressed in the last paragraph of the previous sub-section
> (though the limit is applied to exporting telemetry information). I think
> that it can be removed without any loss to the document.
> >>
> >> [TM] The second par of 3.1.2 refers to *exporting*, while the last par
> >> of 3.1.1 refers to *encapsulating*, so these are two complementary
> >> requirements.
> >
> > GIM>> Is it the intention of this draft to define the encapsulation of
> exported information? What if the local policy mandates the node
> calculating p95 and p99 of packet delay and inter-packet delay variation?
> Can you help me by describing how these metrics are encapsulated?
>
> [[TM]] As emphasized in the document: "The exporting method and format
> are outside the scope of this document."
> Section 3.1.2 actually refers (as its name suggests) to "Responding to
> the DEX Trigger".
>
> >>
> >>
> >>
> >> > I think that it will be helpful to provide some recommendations,
> i.e., SHOULD do this, how an implementation handles the excess of packets
> to export. Drop silently, drop and log notification, or try to shape that
> flow?
> >>
> >> [TM] Assuming an amplification scenario, a large number of packets may
> >> be dropped, so logging each drop may defeat the purpose. A drop
> >> counter may be a good idea here, but this document does not really
> >> define counter requirements in general, so it seems like more of an
> >> implementation-specific aspect, or possibly can be considered as part
> >> of the YANG model.
> >
> > GIM>> I am still struggling to understand how IOAM DEX triggers
> amplification. Can you add more details, steps to help me understand it?
>
> [[TM]] This is explained in the document:
>
>    The amplification effect of DEX may be worse in wide area networks in
>    which there are multiple IOAM domains.  For example, if DEX is used
>    in IOAM domain 1 for exporting IOAM data to a receiving entity, then
>    the exported packets of domain 1 can be forwarded through IOAM domain
>    2, in which they are subject to DEX.  The exported packets of domain
>    2 may in turn be forwarded through another IOAM domain (or through
>    domain 1), and theoretically this recursive amplification may
>    continue infinitely.
>
>
> >>
> >>
> >> > I was really puzzled by this requirement in Section 3.1.2
> >> >
> >> >    Exported packets SHOULD NOT be exported over a path or a tunnel
> that
> >> >
> >> >    is subject to IOAM direct exporting.
> >> >
> >> > I hope you can clarify it for me.
> >>
> >>
> >> [TM] As explained later in that paragraph: "This requirement is
> >> intended to prevent nested exporting and/or exporting loops."
> >> This was thoroughly discussed in the working group, and came out of
> >> the security-related review.
> >
> > GIM>> If the definition of exporting, including the encapsulation,
> choice of transport, and the Collector is outside the scope, then this
> statement, in my opinion, should be removed. If all these issues are in the
> scope, then  there is more, much more information that is missing.
>
> [[TM]] As explained in the abstract, "The exporting method and format
> are outside the scope of this document."
> However, the current document has a paragraph that defines security
> requirements for the export method. These requirements came out of the
> very detailed security discussions we had with Martin Duke and Mirja
> K├╝hlewind.
>
> >>
> >>
> >> >
> >> > This requirement
> >> >
> >> >    Furthermore, IOAM encapsulating
> >> >    nodes that can identify a packet as an IOAM exported packet MUST
> NOT
> >> >    push a DEX option into such a packet.
> >> > brings up questions:
> >> >
> >> > How can a node identify a packet as the IOAM exported packet?
> >>
> >> [TM] The requirement means that if an encapsulating node's ACL can
> >> detect exported packets, then DEX should not be applied.
> >
> > GIM>> I think that whether this belongs in the document depends on the
> answer to my question above - what is in the scope of this document.
>
> [[TM]] See the response above.
>
> >>
> >>
> >> > What should be done if the IOAM exported packet contains the IOAM
> header with the DEX option?
> >>
> >> [TM] The current document does not require an IOAM transit node to
> >> parse beyond the (external) IOAM encapsulation if there are nested
> >> IOAM encapsulations.
> >
> > GIM>> Same, as above - need to clarify the scope of the document first.
> >>
> >>
> >> >
> >> > What is special about the decapsulating node that is not DEX-capable?
> >> >
> >> >    A decapsulating node that does not support the DEX option
> >> >    MUST remove it, along with any other IOAM options carried in the
> >> >    packet if such exist.
> >> > Shouldn't that be a general behavior of a decapsulating IOAM node?
> >>
> >> [TM] Right. The following text edit is proposed:
> >>
> >> OLD:
> >>    A transit IOAM node that does not support the DEX option SHOULD
> >>    ignore it.  A decapsulating node that does not support the DEX option
> >>    MUST remove it, along with any other IOAM options carried in the
> >>    packet if such exist.
> >> NEW:
> >>    A transit or decapsulating IOAM node that does not support the DEX
> >> option SHOULD
> >>    ignore it.  Note that as per <xref
> target="I-D.ietf-ippm-ioam-data"/> a
> >>    decapsulating node removes the IOAM encapsulation and all its IOAM
> options,
> >>    and specifically in the case where one of these options is a
> >> (possibly unknown)
> >>    DEX option.
> >
> > GIM>> Why is it SHOULD? In what situation a node MAY act on the DEX?
>
> Right.
> Proposed text:
>           A transit or decapsulating IOAM node that receives an
> unknown IOAM option
>           type ignores it (as defined in <xref
> target="I-D.ietf-ippm-ioam-data"/>), and
>           specifically nodes that do not support the DEX option ignore it.
>           Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
>           decapsulating node removes the IOAM encapsulation and all
> its IOAM options,
>           and specifically in the case where one of these options is a
>           (possibly unknown) DEX option.
>
> >>
> >>
> >>
> >> >
> >> > The format of the first four-octet word in Figure 2 is different from
> the format defined in draft-ietf-ippm-ioam-data:
> >> >
> >> >     0                   1                   2                   3
> >> >     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> >> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >> >    |        Namespace-ID           |NodeLen  | Flags | RemainingLen|
> >> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >> >    |               IOAM-Trace-Type                 |  Reserved     |
> >> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >> > Is this intentional? If it is, how does an implementation know which
> format to use?
> >>
> >> [TM] Each IOAM Option-Type has a different header format. The current
> >> document defines a new Option-Type, which was not defined in
> >> draft-ietf-ippm-ioam-data.
> >> The IOAM-Type field in the lower layer header determines which
> >> Option-Type is used and how its header should be parsed.
> >
> > GIM>> Thank you for the clarification. Is that the intention to start
> new drafts to define the DEX IOAM Option-Type separately?
>
> [[TM]] Right. IOAM-Type values are defined in per encapsulation
> drafts, such as draft-ietf-ippm-ioam-ipv6-options.
>
>
> >>
> >>
> >>
> >> >
> >> > It is stated in the draft that:
> >> >
> >> > The length of the DEX option is at least 8 octets.
> >> >
> >> > But the figure 2 in the draft does not include any field that
> reflects the length of the IOAM option. Is that because the figure is out
> of sync? If not, how can an implementation verify the length? Also, what is
> the behavior if the length is less than 8 octets?
> >> >
> >>
> >> [TM] Quoting the draft:
> >>
> >>    The existence of the optional fields is
> >>    indicated by the corresponding flags in the Extension-Flags field.
> >>    ...
> >>    Thus, the Extension-Flags field explicitly indicates
> >>    the length of the DEX option.
> >>
> >>
> >> > RE: PerformanceConsiderations
> >> >
> >> > I agree with you that a DEX-encapsulating node and DEX-transit nodes
> must have rate-limiting controls (as noted earlier, the behavior of the
> transit node needs more detailed specification). But I think that the
> document should also note that exported packets could be sent over the
> network out-of-band, relative to the monitored flow. As a result, the
> impact of packets carrying telemetry information might be not direct and
> harmful to the data flow. I see that as one of the important benefits of
> the Direct Export method transporting on-path telemetry information.
> >> >
> >>
> >> [TM] Right. The following new text is proposed:
> >>
> >> It should be noted that in some networks DEX data may be exported over
> >> an out-of-band network, in which a large volume of exported traffic
> >> does not compromise user traffic. In this case an operator may choose
> >> to disable the exporting rate limiting.
> >
> > GIM>> I think that the text can be stronger, recommending that
> information is exported out-of-band and explaining the interpretation of
> out-of-band in the case of IOAM.
> >>
> >>
> >> >
> >> > Regards,
> >> > Greg
> >> >
> >> > On Mon, Aug 30, 2021 at 1:58 PM Tommy Pauly <tpauly=
> 40apple.com@dmarc.ietf.org> wrote:
> >> >>
> >> >> Hello IPPM,
> >> >>
> >> >> This email starts a Working Group Last Call on two related IOAM
> documents, draft-ietf-ippm-ioam-flags and
> draft-ietf-ippm-ioam-direct-export.
> >> >>
> >> >> In-situ OAM Loopback and Active Flags
> >> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/
> >> >> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06
> >> >>
> >> >> In-situ OAM Direct Exporting
> >> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/
> >> >>
> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06
> >> >>
> >> >> Please provide feedback by replying to this email with your reviews
> and if you think these documents are ready to progress, by Wednesday,
> September 15.
> >> >>
> >> >> Best,
> >> >> Tommy & Ian
> >> >>
> >> >> _______________________________________________
> >> >> ippm mailing list
> >> >> ippm@ietf.org
> >> >> https://www.ietf.org/mailman/listinfo/ippm
> >> >
> >> > _______________________________________________
> >> > ippm mailing list
> >> > ippm@ietf.org
> >> > https://www.ietf.org/mailman/listinfo/ippm
>