Re: [ippm] John Scudder's Discuss on draft-ietf-ippm-ioam-ipv6-options-09: (with DISCUSS and COMMENT)

[John Scudder <[e-mail]>]

Hi Frank,

I was revisiting old DISCUSSes and see I didn’t reply to you on this, sorry for that. The approach you describe makes sense to me. I’ll look forward to reviewing the next version if this is how you still plan to proceed.

Please let me know if you’re waiting on me for anything. 

Regards,

—John

> On Dec 30, 2022, at 10:30 AM, Frank Brockners (fbrockne) <[e-mail]> wrote:
> 
> Hi John,
> 
> Thanks a lot for your review and comments.
> 
> Per the discussion in the IESG telechat: I can understand your frustration with draft-ietf-ippm-ioam-ipv6-options-09. Sorry for that. Part of the vagueness you noted stems from the fact that the document  is a merger of two documents:
> * one which was focused on allocating two code points for v6 options (draft-ioametal-ippm-6man-ioam-ipv6-options)
> * one which was focused on deployment considerations (draft-ioametal-ippm-6man-ioam-ipv6-deployment)
> where the second document was just to provide additional information and context for the first one.
> As a result, section 5 turned out to be a set of considerations, that never got turned into a proper set of requirements.
> 
> We could consider splitting the document up again - into a standards track doc that allocates the code points - and an informational document which discusses deployment aspects, though the IPPM WG originally felt like combining the documents was the way to go.
> 
> Assuming that the we'd stay with a single document, my suggestion would be to clean up section 5.1 and
> * use references to other documents, such as RFC 9197, RFC8799, or draft-ietf-ippm-ioam-deployment where applicable
> * switch from "considerations" to "requirements" and be more specific
> 
> In particular:
> C1 - Change to a reference to draft-ietf-ippm-ioam-deployment
> C2 - Keep (the section already uses formal requirements language) - and adjust the wording per Eric Vyncke's suggestions.
> C3 - Keep - and clarify the wording ("entities") per your note below
> C4, C5 - Refer to RFC 9197 and explicitly restate that IOAM only applies to limited domains per RFC8799
> C6 - Reword as a requirement.
> C7 - Expand/Clarify, pending the discussion on the applicability of the incremental trace option for IOAM with v6.
> 
> With that approach, we would also avoid the discussion and potential restatements of "how to ensure that a limited domain does not leak packets" in this document, i.e. the current topics C4 and C5. The document would rather make use of the already existing definition of a limited domain (in RFC8799) and its associated qualities.
> 
> Is this an acceptable approach? If so, we'll create a revision which also addresses your other comments (your comments on section 4 are really helpful - and can be woven into the next rev).
> 
> Thanks again, Frank
> 
>> 
>> ----------------------------------------------------------------------
>> DISCUSS:
>> ----------------------------------------------------------------------
>> 
>> # John Scudder, RTG AD, comments for draft-ietf-ippm-ioam-ipv6-options-09
>> CC @jgscudder
>> 
>> ## DISCUSS
>> 
>> As noted in https://urldefense.com/v3/__https://www.ietf.org/blog/handling-iesg-ballot-positions/__;!!NEt6yMaO-gk!HWtPKZFNvzKAvGGibIraT_FJ14meyU4ob82_XY1jCzk9LXQr32P_G_2jcU_QikNttNa1riQzmGyk0g$ , a
>> DISCUSS ballot is a request to have a discussion on the following topics:
>> 
>> ### Structure of the document; Section 5 is vague
>> 
>> The first part of the document, through Section 4, is unproblematic for me --
>> you're simply allocating some IPv6 extension header option types and defining
>> how to use them to transport fields you defined in RFC 9197. Fine.
>> 
>> But Section 5 is giving me a headache. For some reason, even though this is a
>> Standards Track document, you've structured it as some rather high-level
>> "considerations" (or are they "requirements"? It's unclear) and then some
>> generally-worded polite suggestions about how you could deploy it this way or
>> that way.
>> 
>> Is there some reason you've shied away from being prescriptive in Section 5? As
>> the document stands, I felt a bit like I'd been presented with a puzzle. "The
>> solution of this problem is left as an exercise for the student" is great in
>> textbooks, but not so wonderful in Standards Track documents.
>> 
>> ### Section 5.1, C5 is problematic
>> 
>> ```
>> An Autonomous System (AS) that inserts and leaks the IOAM data needs to be
>> easy
>> to identify for the purpose of troubleshooting, due to the high complexity in
>> identifying the source of the leak. Such a troubleshooting process might
>> require coordination between multiple operators, complex configuration
>> verification, packet capture analysis, etc. This requirement may require
>> additional option or fields to be defined to identify the domain that inserted
>> the IOAM data, this is out of the scope of this document. ```
>> 
>> First, just as in C4, the underlying assumption that it's OK if an AS "leaks
>> the IOAM data" appears problematic.
>> 
>> Second, how can you both say "this is a requirement" and in the same paragraph
>> "it's out of scope"? Surely, if this functionality is required, a finished spec
>> is required to support it. And if the spec isn't finished, we shouldn't be
>> advancing it, the WG should take it up and finish it, then send it back when
>> done.
>> 
>> Is it that this isn't truly a requirement? Or is the spec incomplete? If
>> neither, please help me understand.
>> 
>> 
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>> 
>> ## COMMENTS
>> 
>> These comments are non-blocking, but I'd still appreciate responses.
>> 
>> ### Section 4, a particular interface
>> 
>> ```
>> Unless a particular interface is explicitly enabled (i.e., explicitly
>> configured) for IOAM, a router MUST ignore IOAM Options. ```
>> 
>> I suppose what you mean is, the router MUST ignore IOAM Options on packets
>> received on that interface? If so, please say so. (If not, let's discuss.)
>> 
>> ### Section 4, reference for IOAM Type, nomenclature
>> 
>> ```
>> IOAM Type: 8-bit field as defined in section 7.1 in [RFC9197].
>> ```
>> 
>> Section 7.1 of RFC 9197 is the IOAM Option-Type Registry in the IANA
>> Considerations section. I wouldn't say an IANA registry "defines" anything, it
>> lists code points. I think a better reference would be Section 4 of 9197, which
>> does indeed define IOAM-Option-Types (in Section 4.1).
>> 
>> Also, it would be better to be consistent with your naming, in RFC 9197 you
>> don't call this the "IOAM Type" but rather the "IOAM-Option-Type" (34
>> occurrences in 9197) or "IOAM Option-Type" without the first hyphen (4
>> occurrences in 9197). I see why you don't want to use the full string from RFC
>> 9197 in your packet diagram (too many characters) but "IOAM-Opt-Type" would
>> fit
>> in the character budget.
>> 
>> ### Section 4, Trace-Type constraints
>> 
>> ```
>> In addition, to maintain IPv6 extension header 8-octet alignment and avoid the
>> need to add or remove padding at every hop, the Trace-Type for Incremental
>> Trace Option in IPv6 MUST be selected such that the IOAM node data length is a
>> multiple of 8-octets. ```
>> 
>> I spent some time trying to understand exactly what it means for the Trace-Type
>> to be selected such that, etc. I suppose this isn't too complicated in the end,
>> since all the types defined in RFC 9197 are either four- or eight-byte fields.
>> I don't see an explicit pad field being defined, though, so I wonder if it
>> would be a helpful hint to note that the Opaque State Snapshot with Length 0
>> and Schema ID 0xFFFFFF can be used as a four-octet pad if needed. I see you use
>> 0xFFFFFFFF as a pad in some places in RFC 9197, but there's no corresponding
>> Trace-Type and it doesn't seem prudent to just poach a currently undefined bit
>> to indicate the pad. I guess the other alternative would be to allocate a new
>> Trace-Type bit to explicitly indicate a four-byte pad field, but given you can
>> use Opaque State for this purpose, I don't see why you'd burn the bit.
>> 
>> Anyway, if I've missed some explicit way eight-byte alignment is supported in
>> RFC 9197, please point it out to me? Otherwise, I think you need to be clearer
>> about this, to discourage implementors from exercising "creativity".
>> 
>> ### Section 5.1, C3, entities that communicate the errors
>> 
>> ```
>> The entities that communicate the errors to devices outside of the IOAM domain
>> MUST remove any IOAM data from the packet included in the error message ```
>> 
>> This is quite non-specific. Who are “the entities“ in this context? The host or
>> router that originates the ICMP report? The router that forwards it outside the
>> domain? If the former, I guess that means every entity within the domain that
>> might originate an ICMP message has to know a priori if it's sending its
>> message to an internal or external recipient. If the latter, that's a notable
>> new requirement on border routers.
>> 
>> ### Section 5.1, C4 violates closed domain
>> 
>> ```
>> OAM data leaks can affect the forwarding behavior and state of network
>> elements
>> outside an IOAM domain. IOAM domains MUST provide a mechanism to prevent
>> data
>> leaks or be able to ensure that if a leak occurs, network elements outside the
>> domain are not affected (i.e., they continue to process other valid packets).
>> ```
>> 
>> I have a few difficulties with C4. Among them --
>> 
>> - The first sentence is quite nonspecific, I don't know if you have some actual
>> failure modes in mind but I suppose you must. Can you elucidate? - The second
>> sentence, starting with "or", appears to violate the limited domain assumption,
>> or at the least, it introduces a creative understanding of it ("go ahead and
>> leak as long as you're sure the leaks are fine"). - I don't understand how an
>> operator could possibly fulfill the "or" clause with confidence since by
>> definition whatever is happening outside the border of the limited domain is
>> not under the control of, or even necessarily known to, the operator.
>> 
>> My guess is that you're trying to motivate the ULA deployment option here,
>> without talking about it specifically. Is that right?
>> 
>> ### Section 5.1, C6 is inaccurate
>> 
>> ```
>> Compliance with [RFC8200] requires OAM data to be encapsulated instead of
>> header/option insertion directly into in-flight packets using the original IPv6
>> header. ```
>> 
>> I don't think you mean the OAM data (by which I think you mean IOAM data)
>> needs
>> to be encapsulated, but rather that it needs to be within an encapsulation
>> header? That's what's implied by your Deployment Options section, anyway. If
>> so, please say so, if not, let's discuss.
>> 
>> Assuming I've guessed correctly, a possible rewrite could look something like
>> 
>> ```
>> [RFC8200] precludes insertion of IOAM data directly into the original IPv6
>> header of in-flight packets. Therefore, in order to add IOAM data, in-flight
>> packets must be encapsulated in an outer header, and the IOAM data added to
>> that header. ```
>> 
>> ### Section 5.1, C7, wording
>> 
>> ```
>> Hence when the IOAM Incremental Trace Option-Type is used in the deployment
>> the
>> RemainingLen field of the option MUST follow the guidance in [RFC9197] and
>> must
>> be computed and set appropriately. ```
>> 
>> I assume you mean "used in deployment" and didn't intend the definite article
>> (which would imply you're talking about a specific deployment). But I think
>> instead of just dropping "the", you might as well go ahead and drop "in the
>> deployment" (where else is it going to be used after all, where this guidance
>> wouldn't apply?).
>> 
>> ### Section 5.1, C7, lack of specificity
>> 
>> ```
>> The IOAM Incremental Trace Option-Type expands the option length that may
>> affect the processing of extension headers and options that follow IOAM
>> options. Hence when the IOAM Incremental Trace Option-Type is used in the
>> deployment the RemainingLen field of the option MUST follow the guidance in
>> [RFC9197] and must be computed and set appropriately. ```
>> 
>> What guidance, exactly, is supposed to be followed? I dug around in RFC 9197
>> and the best guess I could come up with is
>> 
>> ```
>> The sender MAY calculate the value of the RemainingLen field by computing the
>> number of node data bytes allowed before exceeding the PMTU, given that the
>> PMTU is known to the sender. ```
>> 
>> I'm not very happy with that guess, because it doesn't seem to be responsive to
>> the problem you've posed. What I've quoted from RFC 9197 is a way to avoid
>> exceeding the PMTU, but in the present document you've posed the concern
>> that
>> if you shove too much data into the IOAM header, other extension headers may
>> be
>> ignored. That's not the same as PMTU, which relates to total packet size, not
>> header options size.
>> 
>> Whatever it is you're trying to do here, I think you need to be more specific
>> about it.
>> 
>> ### Section 5.2, encapsulation?
>> 
>> ```
>> For deployments where the IOAM domain is bounded by hosts, hosts will
>> perform
>> the operation of IOAM data field encapsulation and decapsulation. ```
>> 
>> Do you intend to imply that the hosts at the edge of the domain are sending
>> IP-in-IPv6 encapsulated data? It wouldn't seem to be required; since the hosts
>> are the source/sink of the packets, the problem described in C6 doesn't apply,
>> and the host can directly place the IOAM data in the header. (What would be the
>> "inner header" in an overlay solution.)
>> 
>> I suppose it's technically accurate to describe this as an "encapsulation and
>> decapsulation" operation, insofar as any data placed in any header might be
>> said to be encapsulated in that header -- but it's misleading. I think this
>> section needs to be rewritten to make the meaning plain. For example,
>> something
>> like "... hosts will place the IOAM data field directly in the IPv6 header."
>> (You could say "outer IPv6 header" since there's nothing precluding a host from
>> sending tunneled packets for some purpose.)
>> 
>> (I notice Éric Vyncke raises a similar concern about the nontraditional use of
>> the term "encapsulation" in his comments.)
>> 
>> ### Section 5.3, encapsulation again
>> 
>> This one is less misleading, but by analogy to 5.2 I suspect more clarity is
>> required here too.
>> 
>> ```
>> For deployments where the IOAM domain is bounded by network devices,
>> network
>> devices such as routers form the edge of an IOAM domain. Network devices will
>> perform the operation of IOAM data field encapsulation and decapsulation. ```
>> 
>> For example, a more straightforwardly understandable version of the last
>> sentence might be "Network devices will encapsulate in-flight packets in an
>> outer IPv6 header which carries the IOAM data field."
>> 
>> ### Section 5.4.1 infeasible requirement
>> 
>> ```
>> Addressing and routing in the IOAM Overlay Network are to be configured so
>> that
>> the IP-in-IPv6 encapsulated packets follow the same path as the original,
>> non-encapsulated packet would have taken. ```
>> 
>> This doesn't seem to be feasible in the face of ECMP.
>> 
>> ### Section 5.4.2, wording
>> 
>> ```
>> In this case IOAM can be encapsulated in as an extension header in the tunnel
>> (outer) IPv6 header. ```
>> 
>> Is the "as" unintended in the quoted text? If so, please remove it, if not, I
>> don't understand the meaning.
>> 
>> ## NITS
>> 
>> ### Section 5.1
>> 
>> "if IOAM is used in in transit devices"
>> 
>> s/in in/in/
>> 
>> ### Section 5.4
>> 
>> "This section lists out"
>> 
>> Just "lists", no "out".
>> 
>> ### 5.4.1
>> 
>> "almost close to zero"
>> 
>> I assume you mean either "almost zero" or "close to zero". Because as written,
>> I would parse "almost close to zero" as "not close to zero" which is probably
>> not what you're going for.
>> 
>> ## Notes
>> 
>> This review is in the ["IETF Comments" Markdown format][ICMF], You can use
>> the
>> [`ietf-comments` tool][ICT] to automatically convert this review into
>> individual GitHub issues.
>> 
>> [ICMF]: https://urldefense.com/v3/__https://github.com/mnot/ietf-comments/blob/main/format.md__;!!NEt6yMaO-gk!HWtPKZFNvzKAvGGibIraT_FJ14meyU4ob82_XY1jCzk9LXQr32P_G_2jcU_QikNttNa1riQo4wYhBw$
>> [ICT]: https://urldefense.com/v3/__https://github.com/mnot/ietf-comments__;!!NEt6yMaO-gk!HWtPKZFNvzKAvGGibIraT_FJ14meyU4ob82_XY1jCzk9LXQr32P_G_2jcU_QikNttNa1riSkUdWP9w$
>> 
>> 
>