Re: [tsvwg] RDMA Support by UDP FRAG Option

[Tom Herbert <[e-mail]>]

On Sat, Jun 19, 2021 at 4:40 PM C. M. Heard <[e-mail]> wrote:
>
> On Sat, Jun 19, 2021 at 11:09 AM Tom Herbert wrote:
>>
>> On Sat, Jun 19, 2021 at 11:01 AM C. M. Heard wrote:
>> > Is there a use case for employing UDP options with GRE/UDP? Or GUE?
>>
>> We can divide UDP encapsulations into two flavors. Those that are
>> extensible and contain their own TLVs or equivalent method to encode
>> ancillary data, and those that don't have that. GUE and Geneve are
>> examples of UDP encapsulation protocols that have their own
>> extensibility mechanisms and hence wouldn't be used with UDP options,
>> GRE/UDP and VXLAN don't have extensibility so UDP options could be
>> useful with those protocols.
>
>
> OK, thanks. To loop back to your original point:
>
>>
>>  The stack will attempt to offload the innermost checksum which is the
>>
>> TCP checksum. The TCP checksum is the one offloaded regardless of
>> whether or not the outer UDP checksum is zero (if it's non-zero then
>> the stack would set it using local checksum offload (LCO)). The
>> offloaded TCP checksum computation would start the computation at the
>> first byte of the TCP header through the end of the whole packet. So
>> unless the surplus area is properly checksummed then the computed TCP
>> checksum will be invalid and the packet will be dropped at the
>> receiver. This is not just a problem for offload for offload, I
>> believe that this wouldn't work properly in existing software stacks
>> without some major changes.
>>
>> So to make all uses of transport checksum computation and offload
>> reasonably robust, when the UDP surplus area is being used both the
>> UDP checksum and checksum over the surplus area MUST always be set.
>>
>> FYI, here is some nice background checksum offload is
>> https://www.kernel.org/doc/html/latest/networking/checksum-offloads.html
>>
>
> This was also helpful: https://www.ietf.org/proceedings/95/slides/slides-95-nvo3-3.pdf
>
> Even after looking at the referenced material, however, I must confess that I still need clarification on the following points:
>
> 1.) I do not understand why the UDP checksum (or anything prior to csum_start) would matter for the inner packet checksum offload.
>
It's relevant to checksum offload where devices can only offload one
checksum in the packet hence the best one to offload is the innermost
one since we can take advantage of techniques like LRO.

> 2.) Assuming that the trailer checksum (OCS) is present, it is not clear how offload for the inner packet checksum could be expected to work properly without change. Remember, OCS does not cause the data in the trailer to sum to zero. It causes it to sum to the ones-complement of the trailer length.
>
Regardless, of anything else, the hard requirement is that all
checksums in the packet must be correctly computed when transmitting.
If we are offloading a checksum in the packet, be it the outer UDP
checksum or an inner TCP checksum, the result must be correct. So if
there's bits in the surplus area and that the offload checksum covers
then the surplus area needs to sum to zero or whatever value is
necessary to ensure the offloaded transport checksum is correctly
computed.

> The modifications needed to handle point #2 are small -- specifically, adding the trailer length to whatever would normally be preloaded into the inner packet. The FRAG proposal in draft-ietf-tsvwg-udp-options-13#section-5.5, which proposes to sandwich the payload in between option headers, would make it a whole lot harder.
>
Protocol trailers are a fundamental problem. At this point it's not
clear to me that there is a way to make them work correctly in all use
cases with deployed HW.  Protocol headers, like those used per FRAG,
shouldn't be a problem,

Tom

> Looking forward to your reply.
>
> Mike