Re: [ippm] John Scudder's Discuss on draft-ietf-ippm-ioam-ipv6-options-09: (with DISCUSS and COMMENT)

["Frank Brockners (fbrockne)" <[e-mail]>]

Hi John,

Thanks. We're almost done with creating a new revision. We hope to publish -10 shortly.

Cheers, Frank

> -----Original Message-----
> From: John Scudder <[e-mail]>
> Sent: Monday, 20 February 2023 21:38
> To: Frank Brockners (fbrockne) <[e-mail]>
> Cc: The IESG <[e-mail]>; [e-mail];
> [e-mail]; [e-mail]; [e-mail]
> Subject: Re: John Scudder's Discuss on draft-ietf-ippm-ioam-ipv6-options-09:
> (with DISCUSS and COMMENT)
> 
> Hi Frank,
> 
> I was revisiting old DISCUSSes and see I didn’t reply to you on this, sorry for that.
> The approach you describe makes sense to me. I’ll look forward to reviewing the
> next version if this is how you still plan to proceed.
> 
> Please let me know if you’re waiting on me for anything.
> 
> Regards,
> 
> —John
> 
> > On Dec 30, 2022, at 10:30 AM, Frank Brockners (fbrockne)
> <[e-mail]> wrote:
> >
> > Hi John,
> >
> > Thanks a lot for your review and comments.
> >
> > Per the discussion in the IESG telechat: I can understand your frustration with
> draft-ietf-ippm-ioam-ipv6-options-09. Sorry for that. Part of the vagueness you
> noted stems from the fact that the document  is a merger of two documents:
> > * one which was focused on allocating two code points for v6 options
> > (draft-ioametal-ippm-6man-ioam-ipv6-options)
> > * one which was focused on deployment considerations
> > (draft-ioametal-ippm-6man-ioam-ipv6-deployment)
> > where the second document was just to provide additional information and
> context for the first one.
> > As a result, section 5 turned out to be a set of considerations, that never got
> turned into a proper set of requirements.
> >
> > We could consider splitting the document up again - into a standards track doc
> that allocates the code points - and an informational document which discusses
> deployment aspects, though the IPPM WG originally felt like combining the
> documents was the way to go.
> >
> > Assuming that the we'd stay with a single document, my suggestion
> > would be to clean up section 5.1 and
> > * use references to other documents, such as RFC 9197, RFC8799, or
> > draft-ietf-ippm-ioam-deployment where applicable
> > * switch from "considerations" to "requirements" and be more specific
> >
> > In particular:
> > C1 - Change to a reference to draft-ietf-ippm-ioam-deployment
> > C2 - Keep (the section already uses formal requirements language) - and adjust
> the wording per Eric Vyncke's suggestions.
> > C3 - Keep - and clarify the wording ("entities") per your note below
> > C4, C5 - Refer to RFC 9197 and explicitly restate that IOAM only
> > applies to limited domains per RFC8799
> > C6 - Reword as a requirement.
> > C7 - Expand/Clarify, pending the discussion on the applicability of the
> incremental trace option for IOAM with v6.
> >
> > With that approach, we would also avoid the discussion and potential
> restatements of "how to ensure that a limited domain does not leak packets" in
> this document, i.e. the current topics C4 and C5. The document would rather
> make use of the already existing definition of a limited domain (in RFC8799) and
> its associated qualities.
> >
> > Is this an acceptable approach? If so, we'll create a revision which also
> addresses your other comments (your comments on section 4 are really helpful -
> and can be woven into the next rev).
> >
> > Thanks again, Frank
> >
> >>
> >> ---------------------------------------------------------------------
> >> -
> >> DISCUSS:
> >> ---------------------------------------------------------------------
> >> -
> >>
> >> # John Scudder, RTG AD, comments for
> >> draft-ietf-ippm-ioam-ipv6-options-09
> >> CC @jgscudder
> >>
> >> ## DISCUSS
> >>
> >> As noted in
> >> https://urldefense.com/v3/__https://www.ietf.org/blog/handling-iesg-
> ballot-positions/__;!!NEt6yMaO-
> gk!HWtPKZFNvzKAvGGibIraT_FJ14meyU4ob82_XY1jCzk9LXQr32P_G_2jcU_QikNt
> tNa1riQzmGyk0g$ , a DISCUSS ballot is a request to have a discussion on the
> following topics:
> >>
> >> ### Structure of the document; Section 5 is vague
> >>
> >> The first part of the document, through Section 4, is unproblematic
> >> for me -- you're simply allocating some IPv6 extension header option
> >> types and defining how to use them to transport fields you defined in RFC
> 9197. Fine.
> >>
> >> But Section 5 is giving me a headache. For some reason, even though
> >> this is a Standards Track document, you've structured it as some
> >> rather high-level "considerations" (or are they "requirements"? It's
> >> unclear) and then some generally-worded polite suggestions about how
> >> you could deploy it this way or that way.
> >>
> >> Is there some reason you've shied away from being prescriptive in
> >> Section 5? As the document stands, I felt a bit like I'd been
> >> presented with a puzzle. "The solution of this problem is left as an
> >> exercise for the student" is great in textbooks, but not so wonderful in
> Standards Track documents.
> >>
> >> ### Section 5.1, C5 is problematic
> >>
> >> ```
> >> An Autonomous System (AS) that inserts and leaks the IOAM data needs
> >> to be easy to identify for the purpose of troubleshooting, due to the
> >> high complexity in identifying the source of the leak. Such a
> >> troubleshooting process might require coordination between multiple
> >> operators, complex configuration verification, packet capture
> >> analysis, etc. This requirement may require additional option or
> >> fields to be defined to identify the domain that inserted the IOAM
> >> data, this is out of the scope of this document. ```
> >>
> >> First, just as in C4, the underlying assumption that it's OK if an AS
> >> "leaks the IOAM data" appears problematic.
> >>
> >> Second, how can you both say "this is a requirement" and in the same
> >> paragraph "it's out of scope"? Surely, if this functionality is
> >> required, a finished spec is required to support it. And if the spec
> >> isn't finished, we shouldn't be advancing it, the WG should take it
> >> up and finish it, then send it back when done.
> >>
> >> Is it that this isn't truly a requirement? Or is the spec incomplete?
> >> If neither, please help me understand.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> -
> >> COMMENT:
> >> ---------------------------------------------------------------------
> >> -
> >>
> >> ## COMMENTS
> >>
> >> These comments are non-blocking, but I'd still appreciate responses.
> >>
> >> ### Section 4, a particular interface
> >>
> >> ```
> >> Unless a particular interface is explicitly enabled (i.e., explicitly
> >> configured) for IOAM, a router MUST ignore IOAM Options. ```
> >>
> >> I suppose what you mean is, the router MUST ignore IOAM Options on
> >> packets received on that interface? If so, please say so. (If not,
> >> let's discuss.)
> >>
> >> ### Section 4, reference for IOAM Type, nomenclature
> >>
> >> ```
> >> IOAM Type: 8-bit field as defined in section 7.1 in [RFC9197].
> >> ```
> >>
> >> Section 7.1 of RFC 9197 is the IOAM Option-Type Registry in the IANA
> >> Considerations section. I wouldn't say an IANA registry "defines"
> >> anything, it lists code points. I think a better reference would be
> >> Section 4 of 9197, which does indeed define IOAM-Option-Types (in Section
> 4.1).
> >>
> >> Also, it would be better to be consistent with your naming, in RFC
> >> 9197 you don't call this the "IOAM Type" but rather the
> >> "IOAM-Option-Type" (34 occurrences in 9197) or "IOAM Option-Type"
> >> without the first hyphen (4 occurrences in 9197). I see why you don't
> >> want to use the full string from RFC
> >> 9197 in your packet diagram (too many characters) but "IOAM-Opt-Type"
> >> would fit in the character budget.
> >>
> >> ### Section 4, Trace-Type constraints
> >>
> >> ```
> >> In addition, to maintain IPv6 extension header 8-octet alignment and
> >> avoid the need to add or remove padding at every hop, the Trace-Type
> >> for Incremental Trace Option in IPv6 MUST be selected such that the
> >> IOAM node data length is a multiple of 8-octets. ```
> >>
> >> I spent some time trying to understand exactly what it means for the
> >> Trace-Type to be selected such that, etc. I suppose this isn't too
> >> complicated in the end, since all the types defined in RFC 9197 are either
> four- or eight-byte fields.
> >> I don't see an explicit pad field being defined, though, so I wonder
> >> if it would be a helpful hint to note that the Opaque State Snapshot
> >> with Length 0 and Schema ID 0xFFFFFF can be used as a four-octet pad
> >> if needed. I see you use 0xFFFFFFFF as a pad in some places in RFC
> >> 9197, but there's no corresponding Trace-Type and it doesn't seem
> >> prudent to just poach a currently undefined bit to indicate the pad.
> >> I guess the other alternative would be to allocate a new Trace-Type
> >> bit to explicitly indicate a four-byte pad field, but given you can use Opaque
> State for this purpose, I don't see why you'd burn the bit.
> >>
> >> Anyway, if I've missed some explicit way eight-byte alignment is
> >> supported in RFC 9197, please point it out to me? Otherwise, I think
> >> you need to be clearer about this, to discourage implementors from
> exercising "creativity".
> >>
> >> ### Section 5.1, C3, entities that communicate the errors
> >>
> >> ```
> >> The entities that communicate the errors to devices outside of the
> >> IOAM domain MUST remove any IOAM data from the packet included in the
> >> error message ```
> >>
> >> This is quite non-specific. Who are “the entities“ in this context?
> >> The host or router that originates the ICMP report? The router that
> >> forwards it outside the domain? If the former, I guess that means
> >> every entity within the domain that might originate an ICMP message
> >> has to know a priori if it's sending its message to an internal or
> >> external recipient. If the latter, that's a notable new requirement on border
> routers.
> >>
> >> ### Section 5.1, C4 violates closed domain
> >>
> >> ```
> >> OAM data leaks can affect the forwarding behavior and state of
> >> network elements outside an IOAM domain. IOAM domains MUST provide a
> >> mechanism to prevent data leaks or be able to ensure that if a leak
> >> occurs, network elements outside the domain are not affected (i.e.,
> >> they continue to process other valid packets).
> >> ```
> >>
> >> I have a few difficulties with C4. Among them --
> >>
> >> - The first sentence is quite nonspecific, I don't know if you have
> >> some actual failure modes in mind but I suppose you must. Can you
> >> elucidate? - The second sentence, starting with "or", appears to
> >> violate the limited domain assumption, or at the least, it introduces
> >> a creative understanding of it ("go ahead and leak as long as you're
> >> sure the leaks are fine"). - I don't understand how an operator could
> >> possibly fulfill the "or" clause with confidence since by definition
> >> whatever is happening outside the border of the limited domain is not under
> the control of, or even necessarily known to, the operator.
> >>
> >> My guess is that you're trying to motivate the ULA deployment option
> >> here, without talking about it specifically. Is that right?
> >>
> >> ### Section 5.1, C6 is inaccurate
> >>
> >> ```
> >> Compliance with [RFC8200] requires OAM data to be encapsulated
> >> instead of header/option insertion directly into in-flight packets
> >> using the original IPv6 header. ```
> >>
> >> I don't think you mean the OAM data (by which I think you mean IOAM
> >> data) needs to be encapsulated, but rather that it needs to be within
> >> an encapsulation header? That's what's implied by your Deployment
> >> Options section, anyway. If so, please say so, if not, let's discuss.
> >>
> >> Assuming I've guessed correctly, a possible rewrite could look
> >> something like
> >>
> >> ```
> >> [RFC8200] precludes insertion of IOAM data directly into the original
> >> IPv6 header of in-flight packets. Therefore, in order to add IOAM
> >> data, in-flight packets must be encapsulated in an outer header, and
> >> the IOAM data added to that header. ```
> >>
> >> ### Section 5.1, C7, wording
> >>
> >> ```
> >> Hence when the IOAM Incremental Trace Option-Type is used in the
> >> deployment the RemainingLen field of the option MUST follow the
> >> guidance in [RFC9197] and must be computed and set appropriately. ```
> >>
> >> I assume you mean "used in deployment" and didn't intend the definite
> >> article (which would imply you're talking about a specific
> >> deployment). But I think instead of just dropping "the", you might as
> >> well go ahead and drop "in the deployment" (where else is it going to
> >> be used after all, where this guidance wouldn't apply?).
> >>
> >> ### Section 5.1, C7, lack of specificity
> >>
> >> ```
> >> The IOAM Incremental Trace Option-Type expands the option length that
> >> may affect the processing of extension headers and options that
> >> follow IOAM options. Hence when the IOAM Incremental Trace
> >> Option-Type is used in the deployment the RemainingLen field of the
> >> option MUST follow the guidance in [RFC9197] and must be computed and
> >> set appropriately. ```
> >>
> >> What guidance, exactly, is supposed to be followed? I dug around in
> >> RFC 9197 and the best guess I could come up with is
> >>
> >> ```
> >> The sender MAY calculate the value of the RemainingLen field by
> >> computing the number of node data bytes allowed before exceeding the
> >> PMTU, given that the PMTU is known to the sender. ```
> >>
> >> I'm not very happy with that guess, because it doesn't seem to be
> >> responsive to the problem you've posed. What I've quoted from RFC
> >> 9197 is a way to avoid exceeding the PMTU, but in the present
> >> document you've posed the concern that if you shove too much data
> >> into the IOAM header, other extension headers may be ignored. That's
> >> not the same as PMTU, which relates to total packet size, not header
> >> options size.
> >>
> >> Whatever it is you're trying to do here, I think you need to be more
> >> specific about it.
> >>
> >> ### Section 5.2, encapsulation?
> >>
> >> ```
> >> For deployments where the IOAM domain is bounded by hosts, hosts will
> >> perform the operation of IOAM data field encapsulation and
> >> decapsulation. ```
> >>
> >> Do you intend to imply that the hosts at the edge of the domain are
> >> sending
> >> IP-in-IPv6 encapsulated data? It wouldn't seem to be required; since
> >> the hosts are the source/sink of the packets, the problem described
> >> in C6 doesn't apply, and the host can directly place the IOAM data in
> >> the header. (What would be the "inner header" in an overlay
> >> solution.)
> >>
> >> I suppose it's technically accurate to describe this as an
> >> "encapsulation and decapsulation" operation, insofar as any data
> >> placed in any header might be said to be encapsulated in that header
> >> -- but it's misleading. I think this section needs to be rewritten to
> >> make the meaning plain. For example, something like "... hosts will
> >> place the IOAM data field directly in the IPv6 header."
> >> (You could say "outer IPv6 header" since there's nothing precluding a
> >> host from sending tunneled packets for some purpose.)
> >>
> >> (I notice Éric Vyncke raises a similar concern about the
> >> nontraditional use of the term "encapsulation" in his comments.)
> >>
> >> ### Section 5.3, encapsulation again
> >>
> >> This one is less misleading, but by analogy to 5.2 I suspect more
> >> clarity is required here too.
> >>
> >> ```
> >> For deployments where the IOAM domain is bounded by network devices,
> >> network devices such as routers form the edge of an IOAM domain.
> >> Network devices will perform the operation of IOAM data field
> >> encapsulation and decapsulation. ```
> >>
> >> For example, a more straightforwardly understandable version of the
> >> last sentence might be "Network devices will encapsulate in-flight
> >> packets in an outer IPv6 header which carries the IOAM data field."
> >>
> >> ### Section 5.4.1 infeasible requirement
> >>
> >> ```
> >> Addressing and routing in the IOAM Overlay Network are to be
> >> configured so that the IP-in-IPv6 encapsulated packets follow the
> >> same path as the original, non-encapsulated packet would have taken.
> >> ```
> >>
> >> This doesn't seem to be feasible in the face of ECMP.
> >>
> >> ### Section 5.4.2, wording
> >>
> >> ```
> >> In this case IOAM can be encapsulated in as an extension header in
> >> the tunnel
> >> (outer) IPv6 header. ```
> >>
> >> Is the "as" unintended in the quoted text? If so, please remove it,
> >> if not, I don't understand the meaning.
> >>
> >> ## NITS
> >>
> >> ### Section 5.1
> >>
> >> "if IOAM is used in in transit devices"
> >>
> >> s/in in/in/
> >>
> >> ### Section 5.4
> >>
> >> "This section lists out"
> >>
> >> Just "lists", no "out".
> >>
> >> ### 5.4.1
> >>
> >> "almost close to zero"
> >>
> >> I assume you mean either "almost zero" or "close to zero". Because as
> >> written, I would parse "almost close to zero" as "not close to zero"
> >> which is probably not what you're going for.
> >>
> >> ## Notes
> >>
> >> This review is in the ["IETF Comments" Markdown format][ICMF], You
> >> can use the [`ietf-comments` tool][ICT] to automatically convert this
> >> review into individual GitHub issues.
> >>
> >> [ICMF]:
> >> https://urldefense.com/v3/__https://github.com/mnot/ietf-comments/blo
> >> b/main/format.md__;!!NEt6yMaO-
> gk!HWtPKZFNvzKAvGGibIraT_FJ14meyU4ob82_
> >> XY1jCzk9LXQr32P_G_2jcU_QikNttNa1riQo4wYhBw$
> >> [ICT]:
> >> https://urldefense.com/v3/__https://github.com/mnot/ietf-comments__;!
> >> !NEt6yMaO-
> gk!HWtPKZFNvzKAvGGibIraT_FJ14meyU4ob82_XY1jCzk9LXQr32P_G_2j
> >> cU_QikNttNa1riSkUdWP9w$
> >>
> >>
> >