Re: [tsvwg] A review of draft-ietf-tsvwg-udp-options-12

[Joseph Touch <[e-mail]>]

> On Jun 14, 2021, at 10:57 AM, Tom Herbert <[e-mail]> wrote:
> 
> On Mon, Jun 14, 2021 at 10:44 AM Joseph Touch <[e-mail]> wrote:
>> 
>> Hi, Tom,
>> 
>> OCS has been required since -08 (Sept. 2019). Here’s the relevant text:
>> 
>>>> The OCS MUST be included when the UDP checksum is nonzero and UDP
>>   options are present.
> 
> Joe,
> 
> Yes, but in the case when that's not true the problem of corrupted
> option type still exists.

Not under the conditions where UDP checksum can be zero, as per RFC 8200. In that case, the entire UDP packet becomes a payload in another frame, so it would already be covered. The reason to allow this is to avoid double-protection effort for tunnels. This is no different.

> Also, this rule requires that the
> implementation needs to find the checksum option in a list of TLVS,

If we don’t need to reorder bytes for DMA (which we don’t now), we can require it to be first if present, preceded only by NOPs for alignment (we can remove that too if needed).

Joe

> that makes implementation harder and leads to some strange scenarios--
> like a receiver may process twenty options only to get to the 21st
> which is checksum and if the checksum cannot be validated then the
> node just wasted cycles processing TLVs for no reason. In TCP, for
> instance, we validate the checksum before processing any options so
> this scenario cannot occur.
> 
> Tom
> 
>> 
>> Joe
>> 
>> On Jun 14, 2021, at 10:32 AM, Tom Herbert <[e-mail]> wrote:
>> 
>> Joe,
>> 
>> I suggest that the UDP options should be preceded by a four byte
>> header consisting of one byte type, one byte length, and two byte
>> checksum. As I've mentioned previously, making the checksum optional
>> is inherently problematic because it cannot protect against a
>> corrupted type field for the optional checksum. e.g. a single bit flip
>> in the type field for the checksum could turn the checksum option into
>> some other type and there is no way to detect that.
>> 
>> Tom
>> 
>> Tom
>> 
>> 
>> On Mon, Jun 14, 2021 at 10:20 AM Joe Touch <[e-mail]> wrote:
>> 
>> 
>> Ps - we need an option length field to make fragments look like tcp. I can put that in - do we want that in OCS? Or independent?
>> 
>> On Jun 14, 2021, at 10:16 AM, Joe Touch <[e-mail]> wrote:
>> 
>> FYI that’s what fragments look like. We can’t do this for non fragments.
>> 
>> On Jun 14, 2021, at 10:03 AM, Tom Herbert <[e-mail]> wrote:
>> 
>> On Mon, Jun 14, 2021 at 9:31 AM Gorry Fairhurst <[e-mail]> wrote:
>> 
>> On 14/06/2021 17:17, Tom Herbert wrote:
>> On Sun, Jun 13, 2021 at 9:31 PM Joseph Touch <[e-mail]> wrote:
>> 
>> 
>> 
>> On Jun 13, 2021, at 7:20 PM, C. M. Heard <[e-mail]> wrote:
>> 
>> If we DO support zero-copy and thus want to allow non-terminal fragments to have post-fragoption options that operate on each fragment, then we would add THISFRAGLEN to the nonterminal format and issue different KIND numbers to nonterminal/terminal fragment.
>> 
>> 
>> 
>> I for one would appreciate further discussion of these last points. I admit that I have failed to grasp Joe's message on the RDMA thread, and I would appreciate some time to think about it.
>> 
>> 
>> Sure - here’s how it all works. Note that this is relevant mostly for long transfers with persistent UDP fragmentation; if that is assumed to be ‘adjusted’ at the app layer (as QUIC does), then we don’t need zero-copy support...
>> 
>> - right now, UDP data can be zero-copied when received into user space, starting with the user data
>> 
>> Only if the device supports header/data split where the headers are in
>> one buffer and UDP data is in aligned buffer.
>> 
>> - if we add options, UDP data can still be zero-copied because it hasn’t moved (it still begins the payload
>> - however, fragments are different because (esp given the merging of frag and lite) they don’t start at the beginning of data
>> - they always start after OCS (which I think we should make fit the uniform KIND/LEN/OCS format of 4 bytes)
>> - if the FRAG comes next, then we can move the frag content around a little and still support zero-copy
>> 
>> notably, we move the first 10 bytes of the fragment to the end
>> 4 for OCS
>> 6 for FRAG (assuming FRAG includes KIND/OPTLEN/FRAGOFFSET/ID/FRAGLEN)
>> that way we can zero-copy the frag packet into place, then just copy those last 8 bytes over OCS and the FRAG header
>> 
>> An obvious feature we'd want is NIC hardware to do UDP options
>> fragementation and reassembly, analogous to existing UDP Fragmentation
>> Offload (UFO) which performs IP fragmentation of UDP packets. The
>> impediment with supporting this is that hardware devices would need to
>> perform protocol processing on trailers as opposed to headers. Nearly
>> all hardware devices, including switches and NICs, are optimized to
>> process protocol headers and in modern devices they are quite
>> programmable in that regard. However, they typically rely on a parsing
>> buffer that holds the first N bytes of the packet and assume that all
>> the protocol headers lie within that. They wouldn't process data after
>> that header in the fast path at least, and almost certainly would have
>> capability to process protocol headers at that end of a large packet.
>> I am doubtful we'll ever see hardware support for trailer protocols,
>> and hence it's unlikely we'd see accelerations for UDP options like we
>> have for TCP.
>> 
>> Tom
>> 
>> 
>> OK.... Is there any way that we could design to enable this?
>> 
>> I'm "fishing" for ideas because I know you've talked about the various
>> offload methods.
>> 
>> 
>> Gorry,
>> 
>> My suggestion was to place UDP options after the UDP header. Instead
>> of just placing fragment header after the UDP header, place all the
>> UDP options there and then follow that by the Payload. So packet looks
>> like:
>> 
>> +-------------------+
>> |   UDP header  |
>> +-------------------+
>> |  UDP options  |
>> +-------------------+
>> |     Payload      |
>> +-------------------+
>> 
>> Now this looks a lot like a TCP packet and other variable length
>> headers which we know how to handle. For zero copy we can do
>> header/split by programming emerging smart devices to split through
>> UDP options in one buffer and payload in another thereby also
>> eliminating any need to move headers or data around.
>> 
>> Tom
>> 
>> So for options in the trailer, this is clearly an impediment.
>> 
>> For UDP-Opt fragmentation, I understand there is no standard UDP payload,
>> 
>> .... only an option containing a fragment, so the Fragment information
>> would actually be in the" first N bytes of the packet".
>> 
>> So, what do you think  could be most likely helpful to enable fastpath
>> accelleration for the fragments?
>> 
>> Gorry
>> 
>> This method assumes that we try to keep FRAG early in the packet - preferably right after OCS. The later it comes, the more additional bytes we need to move to “fix” the copy (beyond the 8 bytes noted above).
>> 
>> —
>> 
>> This method is the only reason we would want to allow options after non-terminal fragments - basically to keep the fragment toward the front of the packet, using the rule that post-noninitial frag options still operate on the fragment, rather than waiting for reassembly. The exception is the terminal fragment, where post-terminal fragment options operate on the reassembled packet.
>> 
>> Joe
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>