Re: [IPv6] FW: I-D Action: draft-templin-6man-ipid-ext-00.txt

["Templin (US), Fred L" <[e-mail]>]

Tom, please see my charts from the intarea talk; they clearly show the performance benefits
for using larger transport layer segments. The question of congestion oriented loss is also a
tangential discussion, and one that is addressed by the PTB messaging service specified in
this draft.

The results of my studies with DTN LTP/UDP performance showed that sending fewer and
larger transport layer segments dramatically improved performance even when fragmentation
and reassembly were engaged whereas sending more and smaller transport layer segments
did not improve performance even when GSO/GRO were engaged.

I am getting ready to post a draft update that I hope will address many/most of the
comments from your original post - or, at least it was my intent to try to address them.
Please have another look if you are able.

Thank you - Fred

> -----Original Message-----
> From: Tom Herbert <tom=[e-mail]>
> Sent: Thursday, December 07, 2023 11:51 AM
> To: Templin (US), Fred L <[e-mail]>
> Cc: IPv6 List <[e-mail]>
> Subject: Re: [IPv6] FW: I-D Action: draft-templin-6man-ipid-ext-00.txt
> 
> On Thu, Dec 7, 2023 at 7:58 AM Templin (US), Fred L
> <Fred.L.Templin=[e-mail]> wrote:
> >
> > Tom, to the point on performance:
> >
> > > Please provide references to these studies. Also, note IP
> > > fragmentation is only one possibility, PMTUD and transport layer
> > > segmentation is another and that latter seems more prevalent.
> >
> > If by transport layer segmentation you mean GSO/GRO, it is not the same thing
> > as IP fragmentation at all. GSO/GRO provide a means for the application of the
> > source to transfer a block of data containing multiple MTU- or smaller-sized
> > segments to the kernel in a single system call, then the kernel breaks the
> > segments out into individual packets that are all no larger than the path MTU
> > and sends them to the destination. The destination kernel then gathers them
> > up and posts them to the local application in a reassembled buffer possibly
> > as large as that used by the original source. But, if some packets are lost,
> > the destination kernel instead sends up what it has gathered so far which
> > may be less than the block used by the original source.
> >
> > IP fragmentation is very different and operates on a single large transport
> > layer segment instead of multiple smaller ones. And, the studies I am referring
> > to show that performance was most positively affected by increasing the
> > segment size even to larger than the path MTU. I implemented GSO/GRO
> > in the ion-dtn LTP/UDP implementation and noted that the performance
> > increase I saw was very minor and related to more efficient packaging
> > and not a system call bottleneck. Conversely, when I increased the segment
> > sizes to larger than the path MTU and intentionally invoked IP fragmentation
> > the performance increase was dramatic. You can see this in the charts I
> > showed at IETF118 intarea here:
> >
> > https://datatracker.ietf.org/meeting/118/materials/slides-118-intarea-identification-extension-for-the-internet-protocol-00
> >
> > Again, GSO/GRO address performance limitations of the application/kernel
> > system call interface which seems to have a positive performance effect for
> > some applications. But, IP fragmentation addresses a performance limitation
> > of transport layer protocols in allowing the transport protocol to use larger
> > segment sizes and therefore have fewer segments to deal with.
> 
> Hi Fred,
> 
> Fewer segments, but NOT fewer packets. The net amount of work in the
> system is unchanged when sending larger segments instead of smaller so
> there won't be any material performance differences other than maybe
> implementation effects at the host and no effect at routers. Segments
> are the unit of congestion management and retransmission in a
> transport protocol, but fragments are transparent to the transport
> protocol-- this distinction can cause material issues in performance.
> 
> It's pretty easy to see why this is. Consider that the minimum number
> of segments for a connection would be to use 64K segments and fragment
> them. For a 1500 MTU one segment then would be sent in 43 fragments.
> The problem is that if just one fragment is dropped in a segment then
> the whole segment is retransmitted. Furthermore, the fragments
> themselves are likely to be the cause of the congestion at routers. So
> there is a high likelihood of creating congestion in the network and
> needing a lot of retransmissions. Even if CWND goes to one, each
> connection can still send 43 packets and SACKs don't help because
> there's no granularity at 64K segments so congestion control really
> wouldn't be effective. The net effect is likely to be very poor TCP
> performance.
> 
> While I think there might be some incidental positive performance
> effects in host implementation by using fragmentation, I really don't
> see how it addresses any fundamental performance limitation in a
> transport layer protocol like TCP. In fact, I don't see how IP
> fragmentation could possibly be better than doing PMTUD with SACKs
> especially on the Internet.
> 
> Tom
> 
> 
> 
> 
> > This latter
> > service showed significant performance increases for LTP/UDP and historically
> > provided better performance for services like NFS over UDP. GSO/GRO does
> > not show a similar performance increase for LTP/UDP.
> 
> 
> >
> > Fred
> >
> > > In a
> > > lossy network, transport layer segmentation will likely have better
> > > performance than IP fragmentation since we can do selective ACKs (i.e.
> > > if a fragment is dropped the whole chain needs to be retransmitted). I
> > > suggest removing this and just provide the need for a larger
> > > identification field in IP fragmentation (the second half of the
> > > paragraph).
> >
> > > -----Original Message-----
> > > From: Tom Herbert <tom=[e-mail]>
> > > Sent: Monday, December 04, 2023 2:30 PM
> > > To: Templin (US), Fred L <[e-mail]>
> > > Cc: IPv6 List <[e-mail]>
> > > Subject: Re: [IPv6] FW: I-D Action: draft-templin-6man-ipid-ext-00.txt
> > >
> > > On Tue, Nov 28, 2023 at 1:42 PM Templin (US), Fred L
> > > <Fred.L.Templin=[e-mail]> wrote:
> > > >
> > > > 6man, here is a -00 draft that I am moving over here from intarea where it saw some earlier
> > > > discussion while paring down to focus only on IPv6 aspects. The previous draft version was
> > > > presented at the intarea session at IETF118, per the minutes:
> > > >
> > > > https://datatracker.ietf.org/doc/minutes-118-intarea-202311091400/
> > > >
> > > > The draft is titled: "IPv6 Identification Extension", and its goal is to make IPv6 fragmentation
> > > > and reassembly robust at least within limited domain networks to start with but also for the
> > > > future of more general open IPv6 Internetworks. This is one aspect of a comprehensive set
> > > > of services that should support better Internetworking performance through packet size
> > > > diversity. The draft proposes to update RFC8900.
> > > >
> > > > Please review and send comments,
> > > >
> > >
> > > Hi Fred,
> > >
> > > Here are some comments.
> > >
> > > >From the draft:
> > > "Studies over many decades have shown that upper layer protocols often
> > > achieve greater performance by configuring segment sizes that exceed
> > > the path Maximum Transmission Unit (MTU). When the segment size
> > > exceeds the path MTU, IP fragmentation at some layer is a natural
> > > consequence."
> > >
> > > Please provide references to these studies. Also, note IP
> > > fragmentation is only one possibility, PMTUD and transport layer
> > > segmentation is another and that latter seems more prevalent. In a
> > > lossy network, transport layer segmentation will likely have better
> > > performance than IP fragmentation since we can do selective ACKs (i.e.
> > > if a fragment is dropped the whole chain needs to be retransmitted). I
> > > suggest removing this and just provide the need for a larger
> > > identification field in IP fragmentation (the second half of the
> > > paragraph).
> > >
> > > >From the draft: "A recent study [I-D.templin-dtn-ltpfrag] proved that
> > > configuring segment sizes that cause IPv4 packets to exceed the path
> > > MTU (thereby invoking IPv4 fragmentation and reassembly) provides
> > > substantial performance increases.. This contradicts decades of
> > > unfounded assertions to the contrary..."
> > >
> > > It's not clear to me what that draft proves, nor what the substantial
> > > performance increases are. But even if it did prove something and
> > > "contradicts decades of unfounded assertions to the contrary", I'm not
> > > sure it's very relevant since that means there's been decades of
> > > implementation and burned in deployment based on these assertions. IMO
> > > this paragraph could be removed since it seems to be more about
> > > generally promoting the use of IP fragmentation and not extending it
> > > which is the topic draft.
> > >
> > > >From the draft: "Following reassembly, if the Routing Header is
> > > present the destination resets the Routing Header Next Header field to
> > > the value cached in the Extended Fragment Header."
> > >
> > > I think it will be much simpler to just say that the fragmentation
> > > option cannot be used in Destination Options before a Routing Header.
> > >
> > > The Destination option is type is 00 which means skip if unknown. I
> > > believe this should be 01 to ensure that the destination host ensures
> > > that it sees the fragment and properly processes the packet.
> > >
> > > >From the draft: "This allows the source to test whether a destination
> > > recognizes the Extended Fragment Header by occasionally sending a
> > > fragmented "probe" packet using the option."
> > >
> > > It should be noted that IP is inherently a stateless protocol so
> > > probing like this is at most best effort-- it assumes bi-directional
> > > communications, doesn't work for multicast, or anycast, and has other
> > > failure cases.
> > >
> > > Section 6 "IPv6 Network Fragmentation" would be major protocol changes
> > > to IPv6. This would allow IPv6 routers to fragment, and also
> > > intermediate nodes would be processing Destination Options. I think
> > > there needs to be a very clear motivation for this and what the exact
> > > protocol normative requirements are.
> > >
> > > >From the draft: "Extended Fragment Header further MUST appear as the
> > > first option in the first Destination Options Header". Why is this
> > > requirement needed?
> > >
> > > >From the draft: "Rather than encourage timely course corrections,
> > > however, the IETF somehow forgot that IP fragmentation and reassembly
> > > still serve as essential internetworking functions." This seems
> > > subjective or potentially provocative. I suggest removing it and
> > > probably all of Section 9 can be removed without loss of content for
> > > the main purpose of the draft.
> > >
> > > >From the draft: "The option sets "act" to '00', "chg" to '1'". I'm not
> > > sure what it means to mark a Destination Options as modifiable in
> > > flight. I suppose this could be applied to Destination Options before
> > > the Routing Header, but I don't think that's the intent here. Maybe
> > > this is related to the idea of network node fragmentation?
> > >
> > > IMO, Multicast and Anycast requirements shouldn't be relegated to an appendix.
> > >
> > >
> > > Tom
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > > Fred
> > > >
> > > > -----Original Message-----
> > > > From: I-D-Announce <[e-mail]> On Behalf Of [e-mail]
> > > > Sent: Tuesday, November 28, 2023 1:26 PM
> > > > To: [e-mail]
> > > > Subject:  I-D Action: draft-templin-6man-ipid-ext-00.txt
> > > >
> > > > Internet-Draft draft-templin-6man-ipid-ext-00.txt is now available.
> > > >
> > > >    Title:   IPv6 Identification Extension
> > > >    Author:  Fred L. Templin
> > > >    Name:    draft-templin-6man-ipid-ext-00.txt
> > > >    Pages:   18
> > > >    Dates:   2023-11-28
> > > >
> > > > Abstract:
> > > >
> > > >    The Internet Protocol, version 4 (IPv4) header includes a 16-bit
> > > >    Identification field in all packets, but this length is too small to
> > > >    ensure reassembly integrity even at moderate data rates in modern
> > > >    networks.  Even for Internet Protocol, version 6 (IPv6), the 32-bit
> > > >    Identification field included when a Fragment Header is present may
> > > >    be smaller than desired for some applications.  This specification
> > > >    addresses these limitations by specifying an IPv6 Hop-by-Hop option
> > > >    for Identification Extension; it further provides a messaging service
> > > >    for fragmentation and reassembly congestion management.
> > > >
> > > > The IETF datatracker status page for this Internet-Draft is:
> > > > https://datatracker.ietf.org/doc/draft-templin-6man-ipid-ext/
> > > >
> > > > There is also an HTMLized version available at:
> > > > https://datatracker.ietf.org/doc/html/draft-templin-6man-ipid-ext-00
> > > >
> > > > Internet-Drafts are also available by rsync at:
> > > > rsync.ietf.org::internet-drafts
> > > >
> > > >
> > > > _______________________________________________
> > > > I-D-Announce mailing list
> > > > [e-mail]
> > > > https://www.ietf.org/mailman/listinfo/i-d-announce
> > > >
> > > > --------------------------------------------------------------------
> > > > IETF IPv6 working group mailing list
> > > > [e-mail]
> > > > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> > > > --------------------------------------------------------------------
> >