Re: [tsvwg] RDMA Support by UDP FRAG Option

[Tom Herbert <[e-mail]>]

On Sun, Jun 20, 2021 at 4:36 PM C. M. Heard <[e-mail]> wrote:
>
> On Sat, Jun 19, 2021 at 6:01 PM Tom Herbert wrote:
>>
>> On Sat, Jun 19, 2021 at 4:39 PM C. M. Heard wrote:
>>>
>>> 2.) Assuming that the trailer checksum (OCS) is present, it is not clear how offload for the inner packet checksum could be expected to work properly without change. Remember, OCS does not cause the data in the trailer to sum to zero. It causes it to sum to the ones-complement of the trailer length.
>>>
>>> The modifications needed to handle point #2 are small -- specifically, adding the trailer length to whatever would normally be preloaded into the inner packet. The FRAG proposal in draft-ietf-tsvwg-udp-options-13#section-5.5, which proposes to sandwich the payload in between option headers, would make it a whole lot harder.
>>
>>
>> Protocol trailers are a fundamental problem. At this point it's not
>> clear to me that there is a way to make them work correctly in all use
>> cases with deployed HW.  Protocol headers, like those used per FRAG,
>> shouldn't be a problem,
>
>
> I believe that fear is unjustified. In every case analyzed in detail so far -- including this one -- trailers involve only minor rework at most.  I believe that I proved my case conclusively for the usual case (UDP used as a straight transport, not as an encapsulation) in https://mailarchive.ietf.org/arch/msg/tsvwg/RZULHOKRgrSYIvsI-5Hg7sNtSS8/.

Mike,

Please consider the simple case where the UDP checksum is set. There
are two common methods to offload the checksum: protocol agnostic and
protocol specific. In the protocol agnostic case the stack provides
the starting offset of the checksum area (offset of the UDP header)
and the offset of the checksum field to write the result (UDP
checksum)-- these values are placed in the TX descriptor for a packet.
The checksum field is primed with the pseudo header. The end of the
checksum area is the end of the packet (trying to add a length of the
area in the hardware interface is a non-starter). If there is surplus
and that sums to zero then the algorithm works unchanged. If the
surplus area sums to non-zero then that requires an offsetting sum by
the host adding in the negative sum value of the surplus area into the
checksum field. The good news is that this works with any case of
checksum offload in a UDP packet including encapsulated checksums. In
any case, if the surplus area always sums to zero that would not
require any rework, a non-zero sum surplus area is going to require
work in the OS (I have not looked at what would be required, but it
might be viewed as significant in especially if we need to add a new
field to mbuf, skbuff, etc.)

Protocol specific transmit checksum offload is when the device parses
the protocol and does all the work to send a checksum. It is not the
preferred method, however it is still quite prevalent. We simply have
no idea how devices will do checksum offload done in the presence of
surplus area, I suspect implementations are all over the place. Some
might only checksum over the UDP length, some might checksum over the
surplus area also, some might use the wrong length in the pseudo
header like described in draft. If I had to take a guess on what gives
the best chances for this to work correctly for the most devices, it
would be to always ensure the surplus space sums to zero.

It's a similar story for receive checksum offload, there is a protocol
agnostic and protocol specific method. The protocol generic method
works in all cases, but does require that the host computes the
checksum over the surplus area regardless of whether a checksum in the
area was explicitly set. For protocol specific checksum offload, again
we really have no idea how various devices will deal with surplus
area. Like the send case, I believe that the greatest chance for
compatibility would be to ensure the surplus area sums to zero.

Tom

> Additionally, while an encapsulator that uses UDP options may need to be modified in order to use hardware checksum offload, I think it's clear that the modifications needed are bounded and small. Actually for the case of a TCP inner packet the pseudo-header will be OK to use with a trailer that has OCS included if instead of the actual RCP length the pseudo-header includes TCP + trailer length. The driver still has to remember to copy only UDP length - 8 bytes. Arguably, if it were written correctly in the first place, it would be ready now.
>
> The proposal in the -13 draft does have problems in this regard. I'll reply to that thread in the next day or so with an explanation.
>
> Thanks,
>
> Mike Heard
>