Re: [ippm] WGLC for IOAM Direct Exports

[Tal Mizrahi <[e-mail]>]

Hi Greg,

Thanks for the responses.
Please see additional responses below marked with [[TM]].
If there are no further comments we will post an updated version.

Cheers,
Tal.

On Thu, Oct 7, 2021 at 2:30 AM Greg Mirsky <[e-mail]> wrote:
>
> Hi Tal,
> thank you for your kind consideration of my comments and constructive proposals. Please find my notes in-lined below under the GIM>> tag.
>
> Regards,
> Greg
>
> On Thu, Sep 30, 2021 at 12:17 AM Tal Mizrahi <[e-mail]> wrote:
>>
>> Hi Greg,
>>
>> Many thanks for the thorough review.
>> Below you can find responses to your comments, and in some cases
>> proposed text edits.
>>
>> If there are no further comments we will update the document accordingly.
>>
>> Thanks,
>> The authors.
>>
>>
>> On Wed, Sep 8, 2021 at 1:01 AM Greg Mirsky <[e-mail]> wrote:
>> >
>> > Dear Authors, WG Chairs, and All,
>> > I've read the latest version of the draft and have several comments and questions to share with you. None of these are blocking, I support progressing this draft to publication. Please consider the notes below as an invitation to a discussion:
>> >
>> > It seems that the characterization of the Direct Export IOAM option in the following is not entirely accurate:
>> >
>> > The DEX option is used as a trigger to collect and/or export IOAM data.
>> >
>> > I propose re-wording:
>> > The DEX option is used as a trigger to collect and export or only collect locally the IOAM data.
>> >
>> > Using "exporting and/or collecting" reflects the use of the Direct Export option type technically accurate but the order of actions is backward. What are your thoughts?
>> >
>>
>> [TM] Right. The following text edit is proposed in the introduction.
>>
>> OLD:
>>    This option is used as
>>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
>>    and/or to export it to a receiving entity (or entities).
>> NEW:
>>    This option is used as
>>    a trigger for IOAM nodes to locally aggregate and process IOAM data,
>>    and/or to export it to a receiving entity (or entities). Throughout
>>    the document this functionality is referred to as collection and/or
>> exporting.
>
> GIM>> Thank you, the new text is very helpful.
>>
>>
>>
>> > I find references to RFC 5475 and 7014 very appropriate and helpful. But I don't see these RFCs being referenced in draft-ietf-ippm-ioam-data. I think that excessive use of not only the Direct Export IAOM option but the IOAM method, in general, may adversely affect a network and the recommendations listed in RFC 5475 and 7014 are equally applicable.
>> > Similarly, the good and helpful discussion of how to practically control the use of the Direct Export option is, in my opinion, equally applicable to other IOAM trace options and should be in the Security Considerations section of draft-ietf-ippm-ioam-data.
>>
>>
>> [TM] The last two comments are about draft-ietf-ippm-ioam-data, which
>> is currently in IESG review. I will take this feedback with the other
>> authors to see how we can address this input at this point.
>
> GIM>> Thank you.
>>
>>
>>
>> > The second paragraph in Section 3.1.2 is re-phrasing if not repeating what is already expressed in the last paragraph of the previous sub-section (though the limit is applied to exporting telemetry information). I think that it can be removed without any loss to the document.
>>
>> [TM] The second par of 3.1.2 refers to *exporting*, while the last par
>> of 3.1.1 refers to *encapsulating*, so these are two complementary
>> requirements.
>
> GIM>> Thank you for clarifying that to me.
>>
>>
>>
>> > I think that it will be helpful to provide some recommendations, i.e., SHOULD do this, how an implementation handles the excess of packets to export. Drop silently, drop and log notification, or try to shape that flow?
>>
>> [TM] Assuming an amplification scenario, a large number of packets may
>> be dropped, so logging each drop may defeat the purpose. A drop
>> counter may be a good idea here, but this document does not really
>> define counter requirements in general, so it seems like more of an
>> implementation-specific aspect, or possibly can be considered as part
>> of the YANG model.
>
> GIM>> I think that logging a warning with the throttling control is a good option. What is your opinion? A counter of throttled DEX packets is helpful. Perhaps both mechanisms can be recommended.

[[TM]] The following new sentence is proposed to be added to Section 3.1.2:
NEW:
An IOAM node MAY maintain a counter or a set of counters that count
the events in which the IOAM node receives a packet with the DEX
option type and does not collect and/or export data due to the rate
limits.

>>
>>
>> > I was really puzzled by this requirement in Section 3.1.2
>> >
>> >    Exported packets SHOULD NOT be exported over a path or a tunnel that
>> >
>> >    is subject to IOAM direct exporting.
>> >
>> > I hope you can clarify it for me.
>>
>>
>> [TM] As explained later in that paragraph: "This requirement is
>> intended to prevent nested exporting and/or exporting loops."
>> This was thoroughly discussed in the working group, and came out of
>> the security-related review.
>
> GIM>> As I understand IOAM DEX, packets with telemetry information are out-of-band relative to the monitored data flow and encapsulated to avoid being taken for an IOAM packet. I imagine that exported telemetry data can be collected via, for example, Kafka or gRPC. Am I missing something?

[[TM]] Right, the export encapsulation may be any of the methods you
mentioned. However, if the export path runs through another IOAM
domain it creates a nested IOAM scenario. This scenario was raised by
Martin Duke in his security-related review, and we added this text in
order to address this scenario. This also relates to the next comment
(about pushing the DEX option onto exported packets).

>>
>>
>> >
>> > This requirement
>> >
>> >    Furthermore, IOAM encapsulating
>> >    nodes that can identify a packet as an IOAM exported packet MUST NOT
>> >    push a DEX option into such a packet.
>> > brings up questions:
>> >
>> > How can a node identify a packet as the IOAM exported packet?
>>
>> [TM] The requirement means that if an encapsulating node's ACL can
>> detect exported packets, then DEX should not be applied.
>
> GIM>> Please consider my previous note. I understand that that gets very close to an implementation choice but it seems that the document should be clear in its normative statements.
>>
>>
>> > What should be done if the IOAM exported packet contains the IOAM header with the DEX option?
>>
>> [TM] The current document does not require an IOAM transit node to
>> parse beyond the (external) IOAM encapsulation if there are nested
>> IOAM encapsulations.
>
> GIM>> I have to admit that I'm lost with the reference to an "external IOAM encapsulation". What is the purpose of using IOAM in the exported packet?

[[TM]] My point was that in the scenario in which an exported packet
is forwarded through an IOAM domain we do not expect IOAM transit
nodes to detect this. We expect IOAM transit nodes to parse the outer
encapsulation, but not beyond that.

>>
>>
>> >
>> > What is special about the decapsulating node that is not DEX-capable?
>> >
>> >    A decapsulating node that does not support the DEX option
>> >    MUST remove it, along with any other IOAM options carried in the
>> >    packet if such exist.
>> > Shouldn't that be a general behavior of a decapsulating IOAM node?
>>
>> [TM] Right. The following text edit is proposed:
>>
>> OLD:
>>    A transit IOAM node that does not support the DEX option SHOULD
>>    ignore it.  A decapsulating node that does not support the DEX option
>>    MUST remove it, along with any other IOAM options carried in the
>>    packet if such exist.
>> NEW:
>>    A transit or decapsulating IOAM node that does not support the DEX
>> option SHOULD
>>    ignore it.  Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
>>    decapsulating node removes the IOAM encapsulation and all its IOAM options,
>>    and specifically in the case where one of these options is a
>> (possibly unknown)
>>    DEX option.
>
> GIM>> What could be an alternative to ignoring IOAM DEX option? I feel that MUST is what is required here. Would you agree?

[[TM]] Interesting point, because a node that does not support the DEX
option does not comply to this document, and so the sentence we
suggested would be pointless.
I suggest the following text instead:
NEW:
A transit or decapsulating IOAM node that receives an unknown IOAM option
type ignores it (as defined in <xref target="I-D.ietf-ippm-ioam-data"/>), and
specifically nodes that do not support the DEX option ignore it.
Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
decapsulating node removes the IOAM encapsulation and all its IOAM options,
and specifically in the case where one of these options is a
(possibly unknown) DEX option.


>>
>>
>>
>> >
>> > The format of the first four-octet word in Figure 2 is different from the format defined in draft-ietf-ippm-ioam-data:
>> >
>> >     0                   1                   2                   3
>> >     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> >    |        Namespace-ID           |NodeLen  | Flags | RemainingLen|
>> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> >    |               IOAM-Trace-Type                 |  Reserved     |
>> >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>> > Is this intentional? If it is, how does an implementation know which format to use?
>>
>> [TM] Each IOAM Option-Type has a different header format. The current
>> document defines a new Option-Type, which was not defined in
>> draft-ietf-ippm-ioam-data.
>> The IOAM-Type field in the lower layer header determines which
>> Option-Type is used and how its header should be parsed.
>
> GIM>> How does an IOAM node differentiate between different Option-Types? Is that expected to be communicated to an IOAM node over the management or control plane and not part of the IOAM encapsulation? If that is the case, then, I imagine, changing between IOAM tracing modes will require reconfiguration of all IOAM nodes. Doesn't seem operationally friendly.

[[TM]] This question is not strictly related to the current draft, but
anyway an IOAM node differentiates between different option types
according to the IOAM Type field which resides in the lower layer
header. For example, in the IPv6 option
(draft-ietf-ippm-ioam-ipv6-options-06), the IPv6 Hop-by-Hop Option
header includes the IOAM-Type field:

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Option Type  |  Opt Data Len |   Reserved    |   IOAM Type   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

>>
>>
>>
>> >
>> > It is stated in the draft that:
>> >
>> > The length of the DEX option is at least 8 octets.
>> >
>> > But the figure 2 in the draft does not include any field that reflects the length of the IOAM option. Is that because the figure is out of sync? If not, how can an implementation verify the length? Also, what is the behavior if the length is less than 8 octets?
>> >
>>
>> [TM] Quoting the draft:
>>
>>    The existence of the optional fields is
>>    indicated by the corresponding flags in the Extension-Flags field.
>>    ...
>>    Thus, the Extension-Flags field explicitly indicates
>>    the length of the DEX option.
>>
>>
>> > RE: PerformanceConsiderations
>> >
>> > I agree with you that a DEX-encapsulating node and DEX-transit nodes must have rate-limiting controls (as noted earlier, the behavior of the transit node needs more detailed specification). But I think that the document should also note that exported packets could be sent over the network out-of-band, relative to the monitored flow. As a result, the impact of packets carrying telemetry information might be not direct and harmful to the data flow. I see that as one of the important benefits of the Direct Export method transporting on-path telemetry information.
>> >
>>
>> [TM] Right. The following new text is proposed:
>>
>> It should be noted that in some networks DEX data may be exported over
>> an out-of-band network, in which a large volume of exported traffic
>> does not compromise user traffic. In this case an operator may choose
>> to disable the exporting rate limiting.
>
> GIM>> Thank you for the proposed update. I thought that the main advantage of the IOAM DEX mode is the ability to export the telemetry information out-of-band. And, as I've noted above, the mechanism to collect and transport the telemetry is outside the scope of this specification. I think that would significantly simplify the document without any adverse impact on the value it provides.
>>
>>
>> >
>> > Regards,
>> > Greg
>> >
>> > On Mon, Aug 30, 2021 at 1:58 PM Tommy Pauly <tpauly=[e-mail]> wrote:
>> >>
>> >> Hello IPPM,
>> >>
>> >> This email starts a Working Group Last Call on two related IOAM documents, draft-ietf-ippm-ioam-flags and draft-ietf-ippm-ioam-direct-export.
>> >>
>> >> In-situ OAM Loopback and Active Flags
>> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/
>> >> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06
>> >>
>> >> In-situ OAM Direct Exporting
>> >> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/
>> >> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06
>> >>
>> >> Please provide feedback by replying to this email with your reviews and if you think these documents are ready to progress, by Wednesday, September 15.
>> >>
>> >> Best,
>> >> Tommy & Ian
>> >>
>> >> _______________________________________________
>> >> ippm mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/ippm
>> >
>> > _______________________________________________
>> > ippm mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/ippm