Re: [Int-area] [EXTERNAL] Re: Re: Call for WG adoption of draft-templin-intarea-parcels-10

Joel Halpern <jmh@joelhalpern.com> Tue, 12 July 2022 16:43 UTC

Return-Path: <jmh@joelhalpern.com>
X-Original-To: int-area@ietfa.amsl.com
Delivered-To: int-area@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0E4C0C14CF1B for <int-area@ietfa.amsl.com>; Tue, 12 Jul 2022 09:43:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.027
X-Spam-Level:
X-Spam-Status: No, score=-2.027 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=joelhalpern.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FT4ylyc7tfxp for <int-area@ietfa.amsl.com>; Tue, 12 Jul 2022 09:43:02 -0700 (PDT)
Received: from mailb2.tigertech.net (mailb2.tigertech.net [208.80.4.154]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 817CBC14CF0D for <int-area@ietf.org>; Tue, 12 Jul 2022 09:43:02 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by mailb2.tigertech.net (Postfix) with ESMTP id 4Lj64k0Yj7z1pXqR; Tue, 12 Jul 2022 09:43:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelhalpern.com; s=2.tigertech; t=1657644182; bh=D9TMZFq/uwIEeFvsp5tZzFyjgvDJuqxM39d0YiQsiRU=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=NulLOTGmxJzYK4VBWL0g1bVu00Bvk55b3i0LWUd6uiJl4jo2kYgseXOLums9akWej efoHyV93cdVPVE6goIWrR9sBIZFIah+Mj5RnwfjkCVufB5J2/3u61wruNH/Ct2HFD+ UeBVr+KEsBpMxu8vGl2S/JFkFvK/h8B4rRkjSVKk=
X-Quarantine-ID: <LNYDDCEsbqyI>
X-Virus-Scanned: Debian amavisd-new at b2.tigertech.net
Received: from [192.168.23.181] (unknown [50.233.136.230]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mailb2.tigertech.net (Postfix) with ESMTPSA id 4Lj64h426Gz1pXJm; Tue, 12 Jul 2022 09:42:59 -0700 (PDT)
Content-Type: multipart/alternative; boundary="------------NgAVOZaIp9TNfl6KiJUlp2JJ"
Message-ID: <bbf390dc-6376-3a47-b5b5-b36123c65f7b@joelhalpern.com>
Date: Tue, 12 Jul 2022 12:42:59 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0
Content-Language: en-US
To: "Templin (US), Fred L" <Fred.L.Templin@boeing.com>
Cc: "int-area@ietf.org" <int-area@ietf.org>
References: <889dd0e833374c41b51e10af98b2890e@boeing.com> <3bd3f91c-a9bd-cf9a-c787-85ec54267eca@joelhalpern.com> <bcd79787ebe34191bd98188067414271@boeing.com>
From: Joel Halpern <jmh@joelhalpern.com>
In-Reply-To: <bcd79787ebe34191bd98188067414271@boeing.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/int-area/zYY6l_eCi7SDQ8noEZ8c1q9DdVE>
Subject: Re: [Int-area] [EXTERNAL] Re: Re: Call for WG adoption of draft-templin-intarea-parcels-10
X-BeenThere: int-area@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: IETF Internet Area WG Mailing List <int-area.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/int-area>, <mailto:int-area-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/int-area/>
List-Post: <mailto:int-area@ietf.org>
List-Help: <mailto:int-area-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/int-area>, <mailto:int-area-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jul 2022 16:43:07 -0000

There are no "parcel-capable links"   You are trying to invent them.  
You are allowed to invent them.

But in the absence of a physical link that can actually handle parcels 
natively, I do not see any benefit in handling parcels (if needed at 
all) anywhere except the hosts.

At this point the chairs know my opinion.   I will leave it to them to 
conclude this adoption call and make the rough consensus decision.

Yours,

Joel

On 7/12/2022 12:38 PM, Templin (US), Fred L wrote:
>
> Joel,
>
> The path for an IP parcel traverses parcel-capable links. Some of 
> those links might
>
> be physical links (most likely links nearest the source and links 
> nearest the destination)
>
> while others might be virtual (most likely links in between). Only 
> forwarding nodes on
>
> those kinds of virtual links would deal with parcel packaging and 
> repackaging, and
>
> there would be very few of those. Routers that connect physical 
> parcel-capable links
>
> would either forward the parcel if it is not too big or drop it and 
> return a PTB otherwise.
>
> This is very much intarea territory.
>
> Fred
>
> *From:*Joel Halpern [mailto:jmh.direct@joelhalpern.com]
> *Sent:* Tuesday, July 12, 2022 9:20 AM
> *To:* Templin (US), Fred L <Fred.L.Templin@boeing.com>
> *Cc:* int-area@ietf.org
> *Subject:* [EXTERNAL] Re: [Int-area] Re: Call for WG adoption of 
> draft-templin-intarea-parcels-10
>
>
> 	
>
> EXT email: be mindful of links/attachments.
>
> If you document it purely as a host-to-host tunnel / shim, then it is 
> not even clear you need an RFC.  intarea does not own all tunnels or 
> overlays.
>
> But that is not what you currently have documented.  You ahve 
> documented a mechanism wherein some routers are expected to break up 
> parcels, and some routers are hoped to reassemble parcels.  That is 
> what I object to.
>
> Yours,
>
> Joel
>
> On 7/12/2022 12:02 PM, Templin (US), Fred L wrote:
>
>     Joel, the “shim” you are referring to is IPv6 encapsulation per
>     RFC2473. It forms an
>
>     adaptation layer below IP as the network layer and above IP as the
>     data link layer.
>
>     IP-in-IPv6-in-IP in other words. That seems like intarea.
>
>     Fred
>
>     *From:*Joel Halpern [mailto:jmh.direct@joelhalpern.com
>     <mailto:jmh.direct@joelhalpern.com>]
>     *Sent:* Tuesday, July 12, 2022 8:52 AM
>     *To:* Templin (US), Fred L <Fred.L.Templin@boeing.com>
>     <mailto:Fred.L.Templin@boeing.com>
>     *Cc:* int-area@ietf.org
>     *Subject:* Re: [Int-area] Re: Call for WG adoption of
>     draft-templin-intarea-parcels-10
>
>     If there are no links that can handle parcels bigger than 64K,
>     then as a step one deploy it only on the sending and receiving
>     hosts.  As a shim above the IP layer.  That would not fall into
>     the remit of 6man, nor any other IETF working group.
>
>     While I am skeptical of the long term value and applicability, you
>     are clearly free to do whatever you want inside your hosts
>     (witness the changes Tom has talked about in the Linux stacks.)
>
>     Yours,
>
>     Joel
>
>     On 7/12/2022 11:23 AM, Templin (US), Fred L wrote:
>
>         Joel, I can show you an orders-of-magnitude performance
>         speed-up when I send
>
>         large blocks of data using larger segment sizes that invoke
>         fragmentation and
>
>         reassembly. I can also show a significant speed-up when system
>         calls pass
>
>         multiple larger segments in a single system call instead of
>         one at a time. This
>
>         is on real systems with real data, and not in simulations.
>
>         About links with larger MTUs, I am specifically NOT saying
>         that we need to wait
>
>         until we have links with MTU>64K. What I am saying is that
>         parcels would pave
>
>         the way toward evolution of links with larger MTUs than what
>         we have in the
>
>         current practice allowing a path forward for future
>         innovation. But, parcels are
>
>         still good even for the smallish MTUs in widescale deployment
>         today.
>
>         Fred
>
>         *From:*Joel Halpern [mailto:jmh.direct@joelhalpern.com
>         <mailto:jmh.direct@joelhalpern.com>]
>         *Sent:* Tuesday, July 12, 2022 7:44 AM
>         *To:* Templin (US), Fred L <Fred.L.Templin@boeing.com>
>         <mailto:Fred.L.Templin@boeing.com>
>         *Cc:* int-area@ietf.org
>         *Subject:* [EXTERNAL] Re: [Int-area] Re: Call for WG adoption
>         of draft-templin-intarea-parcels-10
>
>
>         	
>
>         EXT email: be mindful of links/attachments.
>
>         Fred, I understood full well that you only envision a small
>         number of reassembly devices.  After all, on any given path
>         only one device will likely reassemble.  Still, that device
>         will be spending a lot of resources in a very expensive part
>         of the path (fast path forwarding) to provide a small benefit
>         to some hosts.
>
>         Fundamentally you are asking the archtiecture to spend those
>         resources for use case that you have not explained. "I have
>         proof" i snot relevant.  Without knowing the scenarios and the
>         assumptions, it does not help us to judge.  It is worse than
>         the case in the early days of the MANET working group where
>         the competing proposal repeatedly said "my simulation shows ..."
>
>         Fundamentally, it is not the network's job to reassemble
>         packets for a host.  If you want NICs to do that, as Tom has
>         said, that's fine.  It is a private matter between the host
>         and the NIC.  But you are asking for functionality in the
>         network.
>
>         I note also that you are assuming that hosts have links that
>         support actual MTUs larger than 64K.  I know of no link that
>         has those properties in current use.  (I am vaguely familiar
>         with HIPPI and FiberChannel.  Neither appears to be relevant.)
>
>         Yours,
>
>         Joel
>
>         On 7/12/2022 10:02 AM, Templin (US), Fred L wrote:
>
>             Joel, you are misunderstanding what nodes would be
>             involved in reassembly; this would
>
>             not be at every single IP layer router in the path. It
>             would only be at possibly 0, 1 or 2
>
>             adaptation layer middleboxes in the path from source to
>             destination. And, then most
>
>             likely only at a near-end middlebox very near the
>             destination that happens to know the
>
>             destination would prefer to receive larger parcels.
>
>             About segment size, I have proof that using segment sizes
>             significantly larger than the
>
>             path MTU can often produce dramatic performance increases
>             even when fragmentation
>
>             is intentionally invoked. I also have proof that packaging
>             multiple segments in the same
>
>             system call can drive performance even higher an without
>             reducing the segment size.
>
>             IP parcels takes it the logical next step of allowing
>             multiple segments to travel together
>
>             in the same packet, which may or may not be subject to
>             fragmentation and reassembly.
>
>             But, let’s not get so hung up on the middlebox question
>             that we forget the benefits
>
>             for end-to-end.
>
>             Fred
>
>             *From:*Joel Halpern [mailto:jmh@joelhalpern.com
>             <mailto:jmh@joelhalpern.com>]
>             *Sent:* Monday, July 11, 2022 4:02 PM
>             *To:* Templin (US), Fred L <Fred.L.Templin@boeing.com>
>             <mailto:Fred.L.Templin@boeing.com>
>             *Cc:* int-area@ietf.org
>             *Subject:* Re: [Int-area] Re: Call for WG adoption of
>             draft-templin-intarea-parcels-10
>
>
>             No, intermediate reassembly is not an optimization.
>
>             First, it is a bad idea.  It is very painful for routers
>             to perform reassembly.  They have to burn expensive
>             resources managing such attttempted reassesmbly.  It has
>             major cost even if the router decides to give up and
>             forward the pieces.
>
>             And second, unless one makes some unstated assumptions in
>             the absence of such reassembly the sending host will be
>             throttled to the receiving host rate.  So the benefit of
>             the entire system is markedly reduced.
>
>             Net: we should not adopt this draft.
>
>             Yours,
>
>             Joel
>
>             On 7/11/2022 6:41 PM, Templin (US), Fred L wrote:
>
>                 Tom,
>
>                 > Why would someone put six segments in a parcel if
>                 they already have a 9K link MTU?
>
>                 > Why not just send one segment in 9K?
>
>                 This is the mindset that we need to overcome. We have
>                 had it drilled into our heads
>
>                 that MSS must be the same as the path MTU, but it does
>                 not need to be that way.
>
>                 If the MSS is smaller than the path MTU, but we can
>                 send multiple segments in a
>
>                 single parcel that more closely approaches the size of
>                 the path MTU then
>
>                 amortization savings are possible.
>
>                 >The algorithm isn't the problem, it's supporting new
>                 protocols and multiple
>
>                 >checksums in a packet in hardware.
>
>                 But Tom, how hard can this be? Instead of running the
>                 Internet checksum 1 time
>
>                 over N octets of data simply run it M times over N/M
>                 octet chunks of the data in
>
>                 succession but still in a single pass. You spoke
>                 before of NICs adapting to support
>
>                 TCP jumbograms – if they can do that, why not a very
>                 straightforward application
>
>                 of Internet checksum? I haven’t looked at this in a
>                 long while, but isn’t this also
>
>                 similar to what UDP-lite did?
>
>                 > Either you're trivializing reassembly or maybe you're
>                 thinking of some new method that
>
>                 > somehow avoids all the pitfalls and problems we've
>                 had with reassembly over the years!
>
>                 Intermediate node parcel reassembly is really just an
>                 optimization to try to pass the
>
>                 largest possible parcels on to the next hop instead of
>                 passing many smaller ones. It is
>
>                 really just a concatenation of segments of sub-parcels
>                 belonging to the same original
>
>                 parcel. Reordering is unimportant – it is OK to
>                 concatenate sub-parcels 3,8,5,2 in that
>
>                 order and without even waiting for any other
>                 sub-parcels to show up. The application
>
>                 will simply perceive it as a case of network
>                 reordering and the upper layer protocol
>
>                 will do the correct thing with the sequence numbers.
>                 AFAICT, the only hard requirement
>
>                 is that the final sub-parcel must not be concatenated
>                 as an intermediate sub-parcel.
>
>                 This stuff will all work, and it will work for the
>                 betterment of the Internet.
>
>                 Fred
>
>                 *From:*Tom Herbert [mailto:tom@herbertland.com
>                 <mailto:tom@herbertland.com>]
>                 *Sent:* Monday, July 11, 2022 2:57 PM
>                 *To:* Templin (US), Fred L <Fred.L.Templin@boeing.com>
>                 <mailto:Fred.L.Templin@boeing.com>
>                 *Cc:* Richard Li <richard.li@futurewei.com>
>                 <mailto:richard.li@futurewei.com>; Juan Carlos Zuniga
>                 (juzuniga) <juzuniga=40cisco.com@dmarc.ietf.org>
>                 <mailto:juzuniga=40cisco.com@dmarc.ietf.org>;
>                 int-area@ietf.org
>                 *Subject:* Re: [EXTERNAL] Re: [Int-area] Call for WG
>                 adoption of draft-templin-intarea-parcels-10
>
>
>                 	
>
>                 EXT email: be mindful of links/attachments.
>
>                 On Mon, Jul 11, 2022 at 2:20 PM Templin (US), Fred L
>                 <Fred.L.Templin@boeing.com> wrote:
>
>                     Tom, some rejoinders:
>
>                     >Yes, I agree if the packet is fragmented by the
>                     network then this is a nice feature.
>
>                     >However, today we already have this from a host
>                     perspective property by just
>
>                     >sending "small" packets.
>
>                     It can be readily shown that some applications get
>                     much greater performance by
>
>                     sending larger packets that trigger
>                     fragmentation/reassembly than by sending
>
>                     smaller packets that do not. Multiple order of
>                     magnitude performance increases
>
>                     are indeed possible.
>
>                     >I'm not sure the savings qualify as significant. 9K
>                     MTUs are becoming common in data centers
>
>                     >and the standard TCP/IPv6 header is 80 bytes so
>                     that's already less than 1% overhead.
>
>                     I think 9K is only a starting point, and IP
>                     parcels pave the way to much larger link MTUs,
>
>                     possibly even in excess of 64KB. And, doing the
>                     math, even for just a 9K link sending a
>
>                     single parcel that contains 6x 1440 octet segments
>                     would save 5 * 60 == 300 octets in
>
>                 Why would someone put six segments in a parcel if they
>                 already have a 9K link MTU? Why not just send one
>                 segment in 9K?
>
>                     comparison with sending 6x  1500 octet packets
>                     with 60 octets of IP/TCP headers per
>
>                     packet. For links with larger MTUs, the savings
>                     for sending parcels with lots of segments
>
>                     (up to 64) becomes even greater.
>
>                     >As I already mentioned, this is addressed by the
>                     BiGTCP work (https://lwn.net/Articles/884104).
>
>                     >Sending or receiving multi-megabytes TCP segments in one
>                     system call is now feasible. Also, it's
>
>                     >inevitable that NIC vendors will apply this also to be able
>                     to offload TCP jumbo grams. Given this
>
>                     >is just software that doesn't require hardware
>                     change or on-the-wire protocols to change, it's
>
>                     >immediately deployable with just a softwar change which is a
>                     huge benefit to datacenter operators.
>
>                     As I have said, IP parcels has the same advantage
>                     within the host system-call (user-space
>
>                     to kernel-space) context. But, IP parcels goes a
>                     step further to provide efficient packaging
>
>                     over-the-wire, whereas the approach you are
>                     referring to opens the box inside the
>
>                     kernel and sends individual packets instead of
>                     aggregates.
>
>                     >All modern NIC HW can deal with offloading a single
>                     checksum per packet, it's going to be
>
>                     >a major effort for them to offload multiple
>                     checksum like IP parcels needs. Without checksum
>
>                     >offload, this would be a non-starter for a lot of deployments.
>
>                     Check the latest spec (now at -12 and likely to
>                     stay that way until IETF114. Any H/W checksum
>
>                     that can run over the first segment of a packet
>                     should be possible to make run over the N-1
>
>                     additional segments of the same packet (parcel) by
>                     applying the very familiar Internet
>
>                     checksum algorithm.
>
>                 The algorithm isn't the problem, it's supporting new
>                 protocols and multiple checksums in a packet in hardware.
>
>                     >I'm not convinced of that. For instance, I'm
>                     skeptical that intermediate devices trying to
>                     reassemble
>
>                     >packets that aren't addressed to themselves could ever be
>                     robust or efficient (i.e. complexity, non-work
>
>                     >conserving resource requirements, security issues with
>                     reassembly, multi-path that causes latency
>
>                     >increase, potential DoS vector, etc.). Can you comment on this?
>
>                     Perhaps what is confusing this matter is that the
>                     intermediate devices referred to
>
>                     here most certainly do not refer to all routers in
>                     the path. Instead, what is intended
>
>                     here is an OMNI intermediate device, of which
>                     there may be something on the order
>
>                     of 0, 1, or 2 of them on the path between the OMNI
>                     source and destination even
>
>                     though there may be many 10’s or even 100’s of
>                     ordinary IP routers on the path.
>
>                     And, again, this is not a strict reassembly case –
>                     instead, it is an opportunistic
>
>                     “combine if convenient; else forward” swift decision.
>
>                 Either you're trivializing reassembly or maybe you're
>                 thinking of some new method that somehow avoids all
>                 the pitfalls and problems we've had with reassembly
>                 over the years! Consider that many NIC vendors have
>                 tried, and largely failed, to get any sort of device
>                 reassembly widely deployed (e.g. IP reassembly, TCP
>                 segmentation reassembly, etc.). The reason they failed
>                 is because they can't give the host stack transparency
>                 and control over the reassembly process.
>
>                 In its nature reassembly can only be done with at
>                 least packets. That means a device performing
>                 reassembly has to receive one packet, hold it, and
>                 wait for the following packet to perform reassembly.
>                 That makes reassembly, unlike fragmentation, a
>                 non-work conserving process. Many issues and policies
>                 arise from this. For instance, what happens if a
>                 packet is held and the following packet is never seen?
>                 (usually implies a reassembly timer). What happens if
>                 a packet is received OOO and is already forwarded, but
>                 the preceding packet is then received, do we try to
>                 reassemble that one? (the solution here seems to be to
>                 maintain some sort of flow state)? What about
>                 overlapping fragments and the security issues around that?
>
>                 IMO, if the WG does pursue this, I believe a lot of
>                 the effort will be in specifying how reassembly in
>                 intermediate nodes works.
>
>                 Tom
>
>                     Thanks - Fred
>
>                     *From:*Tom Herbert [mailto:tom@herbertland.com]
>                     *Sent:* Monday, July 11, 2022 1:34 PM
>                     *To:* Templin (US), Fred L <Fred.L.Templin@boeing.com>
>                     *Cc:* Richard Li <richard.li@futurewei.com>; Juan
>                     Carlos Zuniga (juzuniga)
>                     <juzuniga=40cisco.com@dmarc.ietf.org>;
>                     int-area@ietf.org
>                     *Subject:* [EXTERNAL] Re: [Int-area] Call for WG
>                     adoption of draft-templin-intarea-parcels-10
>
>
>                     	
>
>                     EXT email: be mindful of links/attachments.
>
>                     On Mon, Jul 11, 2022 at 12:22 PM Templin (US),
>                     Fred L <Fred.L.Templin@boeing.com> wrote:
>
>                         Richard and others, thank you for these
>                         comments and for the ensuing discussion that
>
>                         took place over the time I was away on
>                         vacation. Strange how the timing hit when I
>
>                         was away from the office and off the grid - I
>                         was on a camping trip in Canada not far
>
>                         from where Steve Deering lives although I did
>                         not visit him.
>
>                         In any event, I was able to push out a new
>                         draft version ahead of the deadline that
>
>                         may address some (but likely not all) of your
>                         concerns:
>
>                         https://datatracker.ietf.org/doc/draft-templin-intarea-parcels/
>
>                         The major change is that the draft now talks
>                         about interactions with upper layer
>
>                         protocols including TCP and UDP, whereas the
>                         previous draft versions were silent
>
>                         regarding upper layer protocol framing.
>
>                         To others who have commented, I beg to differ
>                         and maintain that IP parcels do
>
>                         represent a significant improvement over the
>                         current state of affairs and over
>
>                         just regular IP jumbograms. In particular:
>
>                     Hi Fred, some comments in line.
>
>                         1) IP parcels make it so that the loss unit is
>                         a single segment instead of the entire
>
>                         packet/parcel, and loss of a segment often
>                         results in retransmission of just that
>
>                         segment instead of the entire packet/parcel.
>
>                     Yes, I agree if the packet is fragmented by the
>                     network then this is a nice feature. However,
>                     today we already have this from a host perspective
>                     property by just sending "small" packets.
>
>                         2) IP parcels are more efficient than sending
>                         a single segment per IP packet, since
>
>                         the parcel includes a single IP header plus
>                         single full {TCP,UDP} header for possibly
>
>                         many segments. This can result in significant
>                         savings in terms of bits over the wire
>
>                         for omitting unnecessary header bytes.
>
>                     I'm not sure the savings qualify as significant.
>                     9K MTUs are becoming common in data centers and
>                     the standard TCP/IPv6 header is 80 bytes so that's
>                     already less than 1% overhead.
>
>                         Consider the postal service analogy; when
>
>                         many items can be sent together in a single
>                         package/parcel there is a large savings
>
>                         in shippeing and handling costs than when each
>                         individual item is shipped separately.
>
>                     As I already mentioned, this is addressed by the
>                     BiGTCP work (https://lwn.net/Articles/884104).
>                     Sending or receiving multi-megabytes TCP segments
>                     in one system call is now feasible. Also, it's
>                     inevitable that NIC vendors will apply this also
>                     to be able to offload TCP jumbo grams. Given this
>                     is just software that doesn't require hardware
>                     change or on-the-wire protocols to change, it's
>                     immediately deployable with just a softwar change
>                     which is a huge benefit to datacenter operators.
>
>                         3) IP parcels improve large packet integrity
>                         by including a separate checksum for
>
>                         each segment instead of a single checksum for
>                         the entire packet.
>
>                     All modern NIC HW can deal with offloading a
>                     single checksum per packet, it's going to be a
>                     major effort for them to offload multiple checksum
>                     like IP parcels needs. Without checksum offload,
>                     this would be a non-starter for a lot of deployments.
>
>                         This means that
>
>                         large parcels (up to a few MB) can be sent in
>                         one piece over links with sufficiently
>
>                         large MTU without requiring the link itself to
>                         provide strong integrity checks over
>
>                         the entire length of the parcel. This means
>                         that link MTUs significantly larger than
>
>                         9KB are now safely possible.
>
>                         4) IP parcels offer all of the efficiency
>                         advantages to upper layers as are offered
>
>                         by GSO/GRO, etc. but also provide benefits 1)
>                         through 3) above that are not
>
>                         offered by GSO/GRO.
>
>                     Most of this is doable in GSO/GRO.
>
>                         5) Plus, the idea is just plain neat. Better
>                         packaging is good. More efficient
>
>                         handling is good. Reduced header overhead is
>                         good. SAFE larger MTUs are
>
>                         good. The idea itself is good.
>
>                     I'm not convinced of that. For instance, I'm
>                     skeptical that intermediate devices trying to
>                     reassemble packets that aren't addressed to
>                     themselves could ever be robust or efficient (i.e.
>                     complexity, non-work conserving
>                     resource requirements, security issues with
>                     reassembly, multi-path that causes latency
>                     increase, potential DoS vector, etc.). Can you
>                     comment on this?
>
>                     Tom
>
>                         Fred
>
>                         *From:*Int-area
>                         [mailto:int-area-bounces@ietf.org] *On Behalf
>                         Of *Richard Li
>                         *Sent:* Friday, July 01, 2022 3:11 PM
>                         *To:* Juan Carlos Zuniga (juzuniga)
>                         <juzuniga=40cisco.com@dmarc.ietf.org>
>                         *Cc:* int-area@ietf.org
>                         *Subject:* Re: [Int-area] Call for WG adoption
>                         of draft-templin-intarea-parcels-10
>
>                         Chairs and Authors,
>
>                         I always like every new idea and effort to
>                         improve the Internet performance, and thus I
>                         have read this draft with a great interest.
>                         The following are my
>                         observations/comments/questions. If they don’t
>                         make any sense to you, please accept my
>                         apology, and disregard them.
>
>                         1.The text “multiple upper layer protocol
>                         segments” is ambiguous. It seems that you
>                         really mean “multiple segments from ‘the same’
>                         upper layer protocol”, doesn’t it? It seems
>                         that multiple segments from different upper
>                         layer protocols are not allowed in your parcel.
>
>                         2.Is the following a fair statement? All
>                         segments in the same packet come from the same
>                         application identified by the 5-tupe (source
>                         address, destination address, source port,
>                         destination port, protocol number).
>
>                         3.Segment size
>
>                         You require that their sizes be the same
>                         except for the last one. Is this required for
>                         easy implementation or what? Do you require it
>                         for any other reasons?
>
>                         4.TTL issue
>
>                         You described how parcels are forwarded over
>                         the Internetwork, and in particular you
>                         described what the ingress/egress middlebox
>                         does about parcels. I understand that the
>                         ingress middlebox may break the parcel into
>                         smaller ones, which may rejoin at the egress
>                         middlebox. My question is about TTL. As
>                         different smaller parcels may traverse along
>                         different paths, as a result their TTLs may be
>                         different when they reach the egress middlebox
>                         . How does the egress middlebox set up the TTL
>                         value? Please provide more descriptions.
>
>                         5.Reordering at the egress middlebox
>
>                         The parcels would arrive one after another,
>                         and therefore the egress middlebox would
>                         “wait” for a little bit to identify and pick
>                         up enough parcels/packets for their rejoining
>                         and repackaging. A description of the egress
>                         middlebox behavior would be useful and
>                         helpful, in particular I would like to know
>                         more about the waiting time if any, and how
>                         you deal with the reordering and loss.
>
>                         6.IPv4 option
>
>                         Does IETF still allow to change/add IPv4
>                         option fields? I might be wrong, but aren’t
>                         they frozen? Also, do commercial routers still
>                         care about IPv4 options?
>
>                         7.IPv6 option
>
>                         This draft has defined a hop-by-hop option, it
>                         will require every intermediate IPv6 router to
>                         inspect this option. There have been some
>                         discussions on the pros/cons about Hop-by-Hop
>                         IPv6 Option. Is there any feedback from WG 6man?
>
>                         8.Parcel Path Qualification
>
>                         This draft has described a method for parcel
>                         path qualification probe from end to end. It
>                         is nice to have it, but it is unreliable
>                         simply for the following reason: a probe
>                         parcel goes along one specific path, and your
>                         real application parcels may take different
>                         paths.
>
>                         9.Integrity
>
>                         First paragraph of Section 7. More
>                         explanation/elaboration should be useful. I
>                         might have missed it in previous paragraphs,
>                         but if I do, please provide a reference to it
>                         such as “as described in …”.
>
>                         10.Implementation Status
>
>                         In section 10. TSO’s performance gain and
>                         Parcel’s gain should be regarded as two
>                         different things. Since this draft is adding a
>                         hop-by-hop option, every intermediate router
>                         is required to process the hop-by-hop option,
>                         which will, theoretically speaking, lead to
>                         performance downgrade. Of course, the whole
>                         performance would depend on many other
>                         factors, such as the total numbers of routing
>                         table lookups and number of segments.
>
>                         11.General observation
>
>                         This proposal essentially tries to solve a
>                         problem caused by MTU. If MTU be very big, one
>                         would simply put the whole data in a single
>                         packet. Since MTU is limited, a packet has to
>                         be cut into many smaller pieces (segments). In
>                         the existing specification, when an
>                         intermediate router sees a packet with its
>                         size larger than MTU, the router would be
>                         expected to fragment it so that the fragments
>                         could be forwarded. Here let me call it
>                         “fragmentation as needed”. In reality,
>                         however, some (if not all) commercial routers
>                         don’t do “fragmentation as needed”, instead of
>                         fragmenting the packet they simply discard it
>                         in order to achieve the wire-speed. This draft
>                         defines a new way to address the MTU issue:
>                         when a router sees a packet with its size
>                         larger than MTU, the router is asked to
>                         fragment it in a prescribed way (fragment it
>                         into pre-packaged segments). If I may, let me
>                         call it “fragmentation as prescribed”. Both
>                         “fragmentation as needed” and “fragmentation
>                         as prescribed” would require the support from
>                         intermediate routers. As the same as
>                         fragmentation as needed, fragmentation as
>                         prescribed may downgrade the performance of
>                         intermediate routers. What is more,
>                         intermediate routers/boxes may perform
>                         “rejoining and repackaging”, which will
>                         adversely impact the performance of the
>                         intermediate routers/boxes.
>
>                         Best regards,
>
>                         Richard
>
>                         *From:*Int-area <int-area-bounces@ietf.org>
>                         *On Behalf Of *Juan Carlos Zuniga (juzuniga)
>                         *Sent:* Wednesday, June 22, 2022 12:25 PM
>                         *To:* int-area@ietf.org
>                         *Subject:* [Int-area] Call for WG adoption of
>                         draft-templin-intarea-parcels-10
>
>                         Dear IntArea WG,
>
>                         We are starting a 2-week call for adoption of
>                         the IP-Parcels draft:
>
>                         https://www.ietf.org/archive/id/draft-templin-intarea-parcels-10.html
>                         <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Farchive%2Fid%2Fdraft-templin-intarea-parcels-10.html&data=05%7C01%7Crichard.li%40futurewei.com%7C715b5db213134932c70208da5484f702%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637915227299598680%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000%7C%7C%7C&sdata=w4G5ypaSRv%2FR31%2F%2B857XT2xUqHdEXv90ubD5GGjqBEQ%3D&reserved=0>
>
>
>                         The document has been discussed for some time
>                         and it has received multiple comments.
>
>                         If you have an opinion on whether this
>                         document should be adopted by the IntArea WG
>                         please indicate it on the list by the end of
>                         Wednesday July 6^th .
>
>                         Thanks,
>
>                         Juan-Carlos & Wassim
>
>                         (IntArea WG chairs)
>
>                         _______________________________________________
>                         Int-area mailing list
>                         Int-area@ietf.org
>                         https://www.ietf.org/mailman/listinfo/int-area
>
>
>
>
>
>
>                 _______________________________________________
>
>                 Int-area mailing list
>
>                 Int-area@ietf.org
>
>                 https://www.ietf.org/mailman/listinfo/int-area
>
>
> _______________________________________________
> Int-area mailing list
> Int-area@ietf.org
> https://www.ietf.org/mailman/listinfo/int-area