Re: [IPv6] Comparing hbh-processing and eh-limits drafts with comments

[Tom Herbert <[e-mail]>]

On Thu, Oct 27, 2022 at 1:03 PM Brian E Carpenter
<[e-mail]> wrote:
>
> Tom, thanks for doing this! 6man would look silly if we advanced
> two documents simultaneously that were mutually inconsistent. A
> few comments below.
>
> On 28-Oct-22 06:57, Tom Herbert wrote:
> > Comparing draft-ietf-6man-hbh-processing-04 and draft-ietf-6man-eh-limits-01
> >
> >>From hbh-processing: "This document updates [RFC8200] to specify that
> > a node MUST process the first Option in the Hop-by-Hop Header at full
> > forwarding rate (e.g. on the router's Fast Path) and MAY process
> > additional Hop-by-Hop Options if configured to do so."
> >
> >>From eh-limits: "An intermediate node MAY limit the number of
> > non-padding Hop-by-Hop options that it processes.  If a limit is
> > exceeded, that is a packet contains more non-padding options than are
> > configured to process, the intermediate node SHOULD stop processing
> > the Hop-by-Hop Option and ignore any options in the chain beyond the
> > limit."
> >
> > Both drafts seem to allow an intermediate node to process some subset
> > of HBH options, but IMO eh-limits gives more guidance on how to handle
> > the case where there are more options in the packet then the node is
> > willing to process.
> >
> > hbh-processing requires that intermediate nodes process the first HBH
> > option in a packet. IMO, that proverbial ship has sailed. The data
> > show that there are a lot of deployed devices that don't process even
> > one HBH option, this is why the HBH options processing requirements
> > were relaxed in RFC8200.
>
> Yes, I think the requirement in hbh-processing is unrealistic. I can't
> imagine a world in which the highest speed routers in the backbone
> waste a nanosecond that way. Definitely a SHOULD, not a MUST. With a
> proviso such as "a node SHOULD process the first Option in the Hop-by-Hop
> Header at full forwarding rate *unless it inspects only the destination
> address*" or something like that.
>
> > Also, I believe it's unnecessary. Given that
> > HBH options aren't universally supported in the path, there's no
> > motivation to define HBH options where every node on the path needs to
> > process. Host stacks already assume that HBH options may or may not be
> > processed by any or all nodes. The trend seems to be to define HBH
> > options that are defined for limited domains or that are meant to be
> > processed by specific nodes such as the egress router in the local
> > network.
> >
> > In order for HBH options to work at all regardless of how many options
> > are present, intermediate nodes MUST NOT summarily drop packets just
> > because they contain the HBH extension header. Both drafts describe
> > this hard requirement although it's a bit more explicit in eh-limits.
>
> Right. The authors could perhaps figure out how to avoid any semblance
> of difference on this point.
>
> >
> > hbh-processing makes some nice, long changes needed for processing of
> > the two high order bits. However, considering that both the drafts
> > allow some or all HBH options to be ignored, effectively, any HBH
> > option can be treated like it has 00 in the two high order bits.
> > Perhaps it should be recommended that all new HBH options are defined
> > with 00 in the top bits (this could be a requirement in section 6).
> >
> >>From hbh-processing: "The size of an option SHOULD NOT extend beyond
> > what can be reasonably expected to be executed at full forwarding rate
> > (e.g., forwarded on a router's fast path).
> >
> >>From eh-limits: "An intermediate node MUST be able to correctly
> > forward packets that contain an IPv6 header chain of 104 or fewer
> > bytes" and "An intermediate node MAY set a limit on the maximum length
> > of Hop-by-Hop Options extension headers"
> >
> > Both drafts allude to the problem that intermediate nodes may not be
> > able to process variable length headers that are too long. IMO
> > eh-limits is more specific on the requirements around this.
> >
> > hbh-processing has some nice definitions for Fast Path and Slow Path,
> > but it's not clear to me that these are sufficient to be used in
> > normative requirements. Similarly, there are references to "full
> > forwarding rate", which I believe needs a definition in the
> > Terminology section, but also doesn't seem sufficiently understood to
> > be in normative requirements such as "Nodes MUST process all
> > Hop-by-Hop options at full forwarding rates." The fast/slow path split
> > is implementation dependent; what's considered to be a fast path in
> > one implementation
> > might be a slow path on another. Also, note that the adverse effect of
> > the slow/fast path split isn't necessarily the additional per packet
> > latency, it's that taking the slow path can cause OOO packets and high
> > variances in RTT which in turn wreaks havoc on congestion control and
> > retransmission algorithms in transport protocols. I suggest that
> > alternative might be to say that hop-by-hop options processing MUST
> > NOT cause packets to be processed or received out of order (in
> > practice that would mean that everything needs to be processed in the
> > "fast path")
>
> How would that look in an ECMP scenario?
>
Brian,

There's a general assumption in implementations that in order
processing and delivery needs to be maintained only for packets within
a flow, not necessarily maintaining order between packets in different
flows. ECMP does this by computing a hash over the flow identifying
information in a packet, e.g. the TCP five-tuple, and then using the
low order bits of the hash to select the outbound port. The algorithm
works well except that it requires intermediate devices to parse into
the transport layer to extract the port numbers for computing the flow
hash.

BTW, ECMP is one rationale why intermediate devices set an implicit
requirement that they must be able to parse into the transport layer,
and hence one of the motivations in eh-limits to require that
intermediate nodes doing DPI can correctly process packets with up to
sixty-four bytes of extension headers. The IPv6 flow label was
supposed to eliminate the need for DPI to do things like ECMP, but
unfortunately there still seems to be some lingering concerns that it
might not be sufficiently stable during the lifetime of a flow.

Tom

>      Brian
>
> >
> > Tom
> >
> > On Thu, Oct 27, 2022 at 10:40 AM Tom Herbert <[e-mail]> wrote:
> >>
> >> Oops, I just noticed I was looking at an older draft of hbh-options. Will resend
> >>
> >> Tom
> >>
> >> On Thu, Oct 27, 2022 at 10:32 AM Tom Herbert <[e-mail]> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Per Brian's feedback on draft-ietf-6man-eh-limits-01, I'm looking at
> >>> draft-hinden-6man-hbh-processing-01 (hbh-processing) and
> >>> draft-ietf-6man-eh-limits-01 (eh-limits) to compare and hopefully
> >>> rectify them to be consistent. Comparing the drafts and comments are
> >>> below.
> >>>
> >>>  From hbh-processing: "This document updates [RFC8200] that a node MUST
> >>> process the first Option in the Hop-by-Hop Header in the Fast Path and
> >>> MAY process additional Hop-by-Hop Options if configured to do so... If
> >>> there are more than one Hop-by-Hop options in the Hop-by-Hop Options
> >>> header, the node MAY skip the rest of the options"
> >>>
> >>>  From eh-limits: "An intermediate node MAY limit the number of
> >>> non-padding Hop-by-Hop options that it processes.  If a limit is
> >>> exceeded, that is a packet contains more non-padding options than are
> >>> configured to process, the intermediate node SHOULD stop processing
> >>> the Hop-by-Hop Option and ignore any options in the chain beyond the
> >>> limit."
> >>>
> >>> Both drafts seem to allow an intermediate node to process some subset
> >>> of HBH options, but IMO eh-limits gives more guidance on how to handle
> >>> the case where there are more options in the packet then the node is
> >>> willing to process.
> >>>
> >>> hbh-processing requires that intermediate nodes process the first HBH
> >>> option in a packet. IMO, that proverbial ship has sailed. The data
> >>> show that there are a lot of deployed devices that don't process even
> >>> one HBH option, this is why the HBH options processing requirements
> >>> were relaxed in RFC8200. Also, I believe it's unnecessary. Given that
> >>> HBH options aren't universally supported in the path, there's no
> >>> motivation to define HBH options where every node on the path needs to
> >>> process. Host stacks already assume that HBH options may or may not be
> >>> processed by any or all nodes. The trend seems to be to define HBH
> >>> options that are defined for limited domains or that are meant to be
> >>> processed by specific nodes such as the egress router in the local
> >>> network.
> >>>
> >>> In order for HBH options to work at all regardless of how many options
> >>> are present, intermediate nodes MUST NOT summarily drop packets just
> >>> because they contain the HBH extension header. Both drafts describe
> >>> this hard requirement although it's a bit more explicit in eh-limits.
> >>>
> >>> hbh-processing makes some nice, long changes needed for processing of
> >>> the two high order bits. However, considering that both the drafts
> >>> allow some or all HBH options to be ignored, effectively, any HBH
> >>> option can be treated like it has 00 in the two high order bits.
> >>> Perhaps it should be recommended that all new HBH options are defined
> >>> with 00 in the top bits (this could be a requirement in section 6).
> >>>
> >>>  From hbh-processing: "Fixed size in 8-octet units.  Specifically, any
> >>> new Hop-by-Hop options should not be variable size that could extend
> >>> beyond what  can be executed in the Fast Path."
> >>>
> >>>  From eh-limits: "An intermediate node MUST be able to correctly
> >>> forward packets that contain an IPv6 header chain of 104 or fewer
> >>> bytes" and "An intermediate node MAY set a limit on the maximum length
> >>> of Hop-by-Hop Options extension headers"
> >>>
> >>> Both drafts allude to the problem that intermediate nodes may not be
> >>> able to process variable length headers that are too long. IMO
> >>> eh-limits is more specific on the requirements around this.
> >>>
> >>> hbh-processing has some nice definitions for Fast Path and Slow Path,
> >>> but it's not clear to me that these are sufficient to be used in
> >>> normative requirements. For instance, hbh-processing states "Nodes
> >>> that implement a differentiation between a Fast Path and a Slow Path
> >>> MUST process all (with one exception noted below) Hop-by-Hop options
> >>> in the Fast Path." The fast/slow path split is implementation
> >>> dependent; what's considered to be a fast path in one implementation
> >>> might be a slow path on another. Also, note that the adverse effect of
> >>> the slow/fast path split isn't necessarily the additional per packet
> >>> latency, it's that taking the slow path can cause OOO packets and high
> >>> variances in RTT which in turn wreaks havoc on congestion control and
> >>> retransmission algorithms in transport protocols. I suggest that
> >>> alternative might be to say that hop-by-hop options processing MUST
> >>> NOT cause packets to be processed or received out of order (in
> >>> practice that would mean that everything needs to be processed in the
> >>> "fast path")
> >>>
> >>> Tom
> >
> > --------------------------------------------------------------------
> > IETF IPv6 working group mailing list
> > [e-mail]
> > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> > --------------------------------------------------------------------