Re: [ippm] WGLC for IOAM Direct Exports

[Greg Mirsky <[e-mail]>]

Hi Tal,
thank you for posting a new version with updates resulting from our
discussion. I greatly appreciate the consideration you extended to my
comments. As I've mentioned, my comments are non-blocking further progress
of this document. I've read the new version and have some suggestions you
can consider for the next update of the draft:

   - I think that the added text can be a recommendation as the out-of-band
   transport has a lesser adverse impact on the customer traffic:

           It should be noted that in some networks DEX data may be exported
     over an out-of-band network, in which a large volume of exported
     traffic does not compromise user traffic.


   - I propose a minor editorial update as follows:

OLD TEXT:
           In this case an operator
     may choose to disable the exporting rate limiting.
NEW TEXT:
           In this case an operator
     may choose to relax the exporting rate limiting.

   - A new requirement added to the Security Consideration is very useful
   but it seems to suggest a particular implementation:

           Furthermore, an IOAM node MUST
     gain explicit consent to export data to a receiving entity before
     starting to send exported data.

I can imagine that an orchestrator, not an exporting telemetry information
node, obtains the required consent. The sentence might be re-worded as
follows:

           Furthermore, explicit consent to export data to a receiving
entity
           MUST be obtained before activating the IOAM Direct Export trace
           option.

Regards,
Greg

On Wed, Oct 13, 2021 at 5:12 AM Tal Mizrahi <[e-mail]>
wrote:

> Dear Chairs,
>
> We want to thank Greg for his thorough review and useful comments.
> Although these comments were defined by Greg as non-blocking we made
> an effort to address them either by updating the draft or by
> responding to Greg's comments.
> Some further responses can be found below, marked [[TM]].
>
> We have posted an updated version (07) that seems to address these
> comments, as well as comments received from Bernard Aboba (TSV-ART
> reviewer).
> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/
>
>
> We believe the draft is ready to be submitted to the IESG.
>
> Thanks,
> Authors.
>
>
>
> On Sun, Oct 10, 2021 at 8:23 PM Greg Mirsky <[e-mail]> wrote:
> >
> > Hi Tal,
> > thank you for carefully considering my comments and suggestions. Please
> find my follow-up notes in-lined below under the GIM>> tag.
> >
> > Regards,
> > Greg
> >
> > On Thu, Sep 30, 2021 at 12:17 AM Tal Mizrahi <[e-mail]>
> wrote:
> >>
> >> Hi Greg,
> >>
> >> Many thanks for the thorough review.
> >> Below you can find responses to your comments, and in some cases
> >> proposed text edits.
> >>
> >> If there are no further comments we will update the document
> accordingly.
> >>
> >> Thanks,
> >> The authors.
> >>
> >>
> >> On Wed, Sep 8, 2021 at 1:01 AM Greg Mirsky <[e-mail]>
> wrote:
> >> >
> >> > Dear Authors, WG Chairs, and All,
> >> > I've read the latest version of the draft and have several comments
> and questions to share with you. None of these are blocking, I support
> progressing this draft to publication. Please consider the notes below as
> an invitation to a discussion:
> >> >
> >> > It seems that the characterization of the Direct Export IOAM option
> in the following is not entirely accurate:
> >> >
> >> > The DEX option is used as a trigger to collect and/or export IOAM
> data.
> >> >
> >> > I propose re-wording:
> >> > The DEX option is used as a trigger to collect and export or only
> collect locally the IOAM data.
> >> >
> >> > Using "exporting and/or collecting" reflects the use of the Direct
> Export option type technically accurate but the order of actions is
> backward. What are your thoughts?
> >> >
> >>
> >> [TM] Right. The following text edit is proposed in the introduction.
> >>
> >> OLD:
> >>    This option is used as
> >>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
> >>    and/or to export it to a receiving entity (or entities).
> >> NEW:
> >>    This option is used as
> >>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
> >>    and/or to export it to a receiving entity (or entities). Throughout
> >>    the document this functionality is referred to as collection and/or
> >> exporting.
> >
> > GIM>> As I understand it, it is the local policy that controls the
> behavior of the node receiving the IOAM DEX indicator. Similar to IPFIX,
> the information can be aggregated and exported as a calculated metric,
> e.g., percentile or mean value, or exported as soon as possible as
> recorded. The new text is certainly better but it seems it does not express
> the role of the local policy in defining the processing behavior, the
> transport to be used, and the identity of the Collector. One of possible
> ways could be to state that the definition of all these controls is outside
> the scope of the document.
>
> [[TM]] Right. The following text edit is proposed.
> OLD:
> Note that even though the IOAM Option-Type is called "Direct Export",
>       it depends on the deployment whether the receipt of a packet with DEX
>       option type leads to the creation of another packet. Some deployments
>       might simply use the packet with the DEX option type to trigger local
>       processing of OAM data.
> NEW:
> Note that even though the IOAM Option-Type is called "Direct Export",
>       it depends on the deployment whether the receipt of a packet with DEX
>       option type leads to the creation of another packet. Some deployments
>       might simply use the packet with the DEX option type to trigger local
>       processing of OAM data. The functionality of this local processing
> is not
>   within the scope of this document.
>
> >>
> >>
> >>
> >> > I find references to RFC 5475 and 7014 very appropriate and helpful.
> But I don't see these RFCs being referenced in draft-ietf-ippm-ioam-data. I
> think that excessive use of not only the Direct Export IAOM option but the
> IOAM method, in general, may adversely affect a network and the
> recommendations listed in RFC 5475 and 7014 are equally applicable.
> >> > Similarly, the good and helpful discussion of how to practically
> control the use of the Direct Export option is, in my opinion, equally
> applicable to other IOAM trace options and should be in the Security
> Considerations section of draft-ietf-ippm-ioam-data.
> >>
> >>
> >> [TM] The last two comments are about draft-ietf-ippm-ioam-data, which
> >> is currently in IESG review. I will take this feedback with the other
> >> authors to see how we can address this input at this point.
> >
> > GIM>> Thanks.
> >>
> >>
> >>
> >> > The second paragraph in Section 3.1.2 is re-phrasing if not repeating
> what is already expressed in the last paragraph of the previous sub-section
> (though the limit is applied to exporting telemetry information). I think
> that it can be removed without any loss to the document.
> >>
> >> [TM] The second par of 3.1.2 refers to *exporting*, while the last par
> >> of 3.1.1 refers to *encapsulating*, so these are two complementary
> >> requirements.
> >
> > GIM>> Is it the intention of this draft to define the encapsulation of
> exported information? What if the local policy mandates the node
> calculating p95 and p99 of packet delay and inter-packet delay variation?
> Can you help me by describing how these metrics are encapsulated?
>
> [[TM]] As emphasized in the document: "The exporting method and format
> are outside the scope of this document."
> Section 3.1.2 actually refers (as its name suggests) to "Responding to
> the DEX Trigger".
>
> >>
> >>
> >>
> >> > I think that it will be helpful to provide some recommendations,
> i.e., SHOULD do this, how an implementation handles the excess of packets
> to export. Drop silently, drop and log notification, or try to shape that
> flow?
> >>
> >> [TM] Assuming an amplification scenario, a large number of packets may
> >> be dropped, so logging each drop may defeat the purpose. A drop
> >> counter may be a good idea here, but this document does not really
> >> define counter requirements in general, so it seems like more of an
> >> implementation-specific aspect, or possibly can be considered as part
> >> of the YANG model.
> >
> > GIM>> I am still struggling to understand how IOAM DEX triggers
> amplification. Can you add more details, steps to help me understand it?
>
> [[TM]] This is explained in the document:
>
>    The amplification effect of DEX may be worse in wide area networks in
>    which there are multiple IOAM domains.  For example, if DEX is used
>    in IOAM domain 1 for exporting IOAM data to a receiving entity, then
>    the exported packets of domain 1 can be forwarded through IOAM domain
>    2, in which they are subject to DEX.  The exported packets of domain
>    2 may in turn be forwarded through another IOAM domain (or through
>    domain 1), and theoretically this recursive amplification may
>    continue infinitely.
>
>
> >>
> >>
> >> > I was really puzzled by this requirement in Section 3.1.2
> >> >
> >> >    Exported packets SHOULD NOT be exported over a path or a tunnel
> that
> >> >
> >> >    is subject to IOAM direct exporting.
> >> >
> >> > I hope you can clarify it for me.
> >>
> >>
> >> [TM] As explained later in that paragraph: "This requirement is
> >> intended to prevent nested exporting and/or exporting loops."
> >> This was thoroughly discussed in the working group, and came out of
> >> the security-related review.
> >
> > GIM>> If the definition of exporting, including the encapsulation,
> choice of transport, and the Collector is outside the scope, then this
> statement, in my opinion, should be removed. If all these issues are in the
> scope, then  there is more, much more information that is missing.
>
> [[TM]] As explained in the abstract, "The exporting method and format
> are outside the scope of this document."
> However, the current document has a paragraph that defines security
> requirements for the export method. These requirements came out of the
> very detailed security discussions we had with Martin Duke and Mirja
> Kühlewind.
>
> >>
> >>
> >> >
> >> > This requirement
> >> >
> >> >    Furthermore, IOAM encapsulating
> >> >    nodes that can identify a packet as an IOAM exported packet MUST
> NOT
> >> >    push a DEX option into such a packet.
> >> > brings up questions:
> >> >
> >> > How can a node identify a packet as the IOAM exported packet?
> >>
> >> [TM] The requirement means that if an encapsulating node's ACL can
> >> detect exported packets, then DEX should not be applied.
> >
> > GIM>> I think that whether this belongs in the document depends on the
> answer to my question above - what is in the scope of this document.
>
> [[TM]] See the response above.
>
> >>
> >>
> >> > What should be done if the IOAM exported packet contains the IOAM
> header with the DEX option?
> >>
> >> [TM] The current document does not require an IOAM transit node to
> >> parse beyond the (external) IOAM encapsulation if there are nested
> >> IOAM encapsulations.
> >
> > GIM>> Same, as above - need to clarify the scope of the document first.
> >>
> >>
> >> >
> >> > What is special about the decapsulating node that is not DEX-capable?
> >> >
> >> >    A decapsulating node that does not support the DEX option
> >> >    MUST remove it, along with any other IOAM options carried in the
> >> >    packet if such exist.
> >> > Shouldn't that be a general behavior of a decapsulating IOAM node?
> >>
> >> [TM] Right. The following text edit is proposed:
> >>
> >> OLD:
> >>    A transit IOAM node that does not support the DEX option SHOULD
> >>    ignore it.  A decapsulating node that does not support the DEX option
> >>    MUST remove it, along with any other IOAM options carried in the
> >>    packet if such exist.
> >> NEW:
> >>    A transit or decapsulating IOAM node that does not support the DEX
> >> option SHOULD
> >>    ignore it.  Note that as per <xref
> target="I-D.ietf-ippm-ioam-data"/> a
> >>    decapsulating node removes the IOAM encapsulation and all its IOAM
> options,
> >>    and specifically in the case where one of these options is a
> >> (possibly unknown)
> >>    DEX option.
> >
> > GIM>> Why is it SHOULD? In what situation a node MAY act on the DEX?
>
> Right.
> Proposed text:
>           A transit or decapsulating IOAM node that receives an
> unknown IOAM option
>           type ignores it (as defined in <xref
> target="I-D.ietf-ippm-ioam-data"/>), and
>           specifically nodes that do not support the DEX option ignore it.
>           Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
>           decapsulating node removes the IOAM encapsulation and all
> its IOAM options,
>           and specifically in the case where one of these options is a
>           (possibly unknown) DEX option.
>
> >>
> >>
> >>
> >> >
> >> > The format of the first four-octet word in Figure 2 is different from
> the format defined in draft-ietf-ippm-ioam-data:
> >> >
> >> >     0                   1                   2                   3
> >> >     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> >> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >> >    |        Namespace-ID           |NodeLen  | Flags | RemainingLen|
> >> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >> >    |               IOAM-Trace-Type                 |  Reserved     |
> >> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >> > Is this intentional? If it is, how does an implementation know which
> format to use?
> >>
> >> [TM] Each IOAM Option-Type has a different header format. The current
> >> document defines a new Option-Type, which was not defined in
> >> draft-ietf-ippm-ioam-data.
> >> The IOAM-Type field in the lower layer header determines which
> >> Option-Type is used and how its header should be parsed.
> >
> > GIM>> Thank you for the clarification. Is that the intention to start
> new drafts to define the DEX IOAM Option-Type separately?
>
> [[TM]] Right. IOAM-Type values are defined in per encapsulation
> drafts, such as draft-ietf-ippm-ioam-ipv6-options.
>
>
> >>
> >>
> >>
> >> >
> >> > It is stated in the draft that:
> >> >
> >> > The length of the DEX option is at least 8 octets.
> >> >
> >> > But the figure 2 in the draft does not include any field that
> reflects the length of the IOAM option. Is that because the figure is out
> of sync? If not, how can an implementation verify the length? Also, what is
> the behavior if the length is less than 8 octets?
> >> >
> >>
> >> [TM] Quoting the draft:
> >>
> >>    The existence of the optional fields is
> >>    indicated by the corresponding flags in the Extension-Flags field.
> >>    ...
> >>    Thus, the Extension-Flags field explicitly indicates
> >>    the length of the DEX option.
> >>
> >>
> >> > RE: PerformanceConsiderations
> >> >
> >> > I agree with you that a DEX-encapsulating node and DEX-transit nodes
> must have rate-limiting controls (as noted earlier, the behavior of the
> transit node needs more detailed specification). But I think that the
> document should also note that exported packets could be sent over the
> network out-of-band, relative to the monitored flow. As a result, the
> impact of packets carrying telemetry information might be not direct and
> harmful to the data flow. I see that as one of the important benefits of
> the Direct Export method transporting on-path telemetry information.
> >> >
> >>
> >> [TM] Right. The following new text is proposed:
> >>
> >> It should be noted that in some networks DEX data may be exported over
> >> an out-of-band network, in which a large volume of exported traffic
> >> does not compromise user traffic. In this case an operator may choose
> >> to disable the exporting rate limiting.
> >
> > GIM>> I think that the text can be stronger, recommending that
> information is exported out-of-band and explaining the interpretation of
> out-of-band in the case of IOAM.
> >>
> >>
> >> >
> >> > Regards,
> >> > Greg
> >> >
> >> > On Mon, Aug 30, 2021 at 1:58 PM Tommy Pauly <tpauly=
> [e-mail]> wrote:
> >> >>
> >> >> Hello IPPM,
> >> >>
> >> >> This email starts a Working Group Last Call on two related IOAM
> documents, draft-ietf-ippm-ioam-flags and
> draft-ietf-ippm-ioam-direct-export.
> >> >>
> >> >> In-situ OAM Loopback and Active Flags
> >> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/
> >> >> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06
> >> >>
> >> >> In-situ OAM Direct Exporting
> >> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/
> >> >>
> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06
> >> >>
> >> >> Please provide feedback by replying to this email with your reviews
> and if you think these documents are ready to progress, by Wednesday,
> September 15.
> >> >>
> >> >> Best,
> >> >> Tommy & Ian
> >> >>
> >> >> _______________________________________________
> >> >> ippm mailing list
> >> >> [e-mail]
> >> >> https://www.ietf.org/mailman/listinfo/ippm
> >> >
> >> > _______________________________________________
> >> > ippm mailing list
> >> > [e-mail]
> >> > https://www.ietf.org/mailman/listinfo/ippm
>