Re: [tsvwg] RDMA Support by UDP FRAG Option

[Joseph Touch <[e-mail]>]

Tom,

> On Jun 19, 2021, at 10:11 AM, Tom Herbert <[e-mail]> wrote:
> 
> ...
> There is another serious problem with transport checksums and use of
> the UDP surplus area. Most of this discussion has presumed that the
> UDP checksum is the one in the packet being offloaded, but that may
> not be the case. Consider the case where a sender sends a TCP packet
> that is encapsulated GRE/UDP or VXLAN (a very common use case in
> virtual networks where VMs send packets on their virtual networks).
> The stack will attempt to offload the innermost checksum which is the
> TCP checksum.

I’m assuming this means TCP in GRE/UDP (it seems ambiguous).

> The TCP checksum is the one offloaded regardless of
> whether or not the outer UDP checksum is zero (if it's non-zero then
> the stack would set it using local checksum offload (LCO)).

Can you explain this? Packets are both defined and processed from the outside in; to do anything else might not yield a meaningful result.

> The
> offloaded TCP checksum computation would start the computation at the
> first byte of the TCP header through the end of the whole packet.

If TCP interprets “end” to mean anything beyond what the UDP header indicates, it is that TCP stack that is broken.

(Presumably we’re talking about non-UDP fragmented packets; with UDP fragmentation, the TCP operation should never happen until the UDP fragments are reassembled).

> So
> unless the surplus area is properly checksummed then the computed TCP
> checksum will be invalid and the packet will be dropped at the
> receiver.

See above.

> This is not just a problem for offload for offload, I
> believe that this wouldn't work properly in existing software stacks
> without some major changes.

Same problem, same answer. 

> So to make all uses of transport checksum computation and offload
> reasonably robust, when the UDP surplus area is being used both the
> UDP checksum and checksum over the surplus area MUST always be set.

If protocol stacks that try to peek ahead in layers don’t follow the rules, there’s not much we can do, ever. 

> FYI, here is some nice background checksum offload is
> https://www.kernel.org/doc/html/latest/networking/checksum-offloads.html

There is an error in draft-herbert-remotecsumoffload in Section 2.1; it states that the UDP checksum is over the upper layer packet length (it is not). RFC768 defines the UDP checksum over the pseudoheader, UDP header, and data (not referring to the IP packet except for pseudo header info). In fact, it also refers to the pseudoheader as using the UDP Length, not the IP length (adjusted or not). 

The same error appears in the draft-herbert-vxlan-rco in the definition of packet_csum.

If either doc reflects how offloading is implemented, then those are bugs that should be fixed.

In TCPM we’ve identified a number of additional issues with TCP offload, notably regarding how they incorrectly coalesce packets with different TCP headers.  

If we’re not calling out these behaviors as the bugs they are, there’s little point in doing much of anything in the IETF.

Joe