Re: [Tofoo] FW: I-D Action: draft-zhou-li-vxlan-soe-01.txt

[Joe Touch <[e-mail]>]

On 5/20/2014 7:17 PM, Zhou, Han wrote:
> Hi Joe,
>
> This is an interesting topic.
>
>> TCP offloading is fine when the OS hands off user data, and the offload
>> engine creates the entire segment.
 >
> Existing TSO/GSO mechanisms deliver full (large) TCP segment to
> "offload engine", which then create smaller segments according to
> physical MTU, and recalculates checksums. This is the case even
> without overlay considered. So I suppose the problem you pointed out
> is not related to my change, but a general limitation for TSO/GSO,
> right?

It depends on what part of TCP happens in the guest OS vs. the 
underlying engine. If you expose the TCP API to the guest OS (the API 
spec'd in RFC793), and hand "Send" call data down to the engine, that's 
fine.

However, what I think is happening is this:

	- the guest OS receives the "Send" call and creates a TCP
	segment, including TCP header and TCP options

	- the guest OS hands the TCP segment to the engine

	- the engine parses that TCP segment to create multiple
	outgoing segments, typically by copying the passed segment's
	header and options, and recalculating the fields it
	thinks it needs to

Simply put, that's as bad as having any middlebox re-calculating TCP 
segments, and is guaranteed to create problems (even if the 'typical' 
case doesn't trip over them).

The problem is that the engine's TCP interpreter may not understand all 
TCP header options - when (not if) that happens, what does it do?

RFC793 is clear on this - when a SYN arrives with an option that isn't 
understood, the receiver MUST silently ignore that option.

So the engine ought to have stripped out all options it doesn't 
understand from the first SYN sent*. But I suspect that's not what it 
thinks it should do - I suspect it thinks it's OK to merely copy - or 
pass through - options it doesn't understand.

What should happen is that the engine interface should NEVER be a TCP 
segment formed by the guest OS. If what you want is to offload 
segmentation, you ought to pass the user data and TCP header (and its 
options) as separate parameters.

(* this is why a correctly-written engine ends up reducing TCP 
functionality, because a connection can support only what is supported 
by the endpoints AND the engine [on each end]). Any option the engine 
doesn't support should never be allowed on the connection.

> For my understanding the TCP implementation should decide whether to
> use offloading or not according to the feature/options required by a TCP
> connection. If the option required (such as MD5) is not supported by
> offloading, the TCP stack should do the segmentation by itself instead
> of utilizing offloading.

That works if unknown options are assumed NOT SUPPORTED.

But I still don't quite understand why you want the segmentation 
happening in the VM - why not pass the MTU info to the virtual interface 
in the guest OS and let it handle things?

> In fact, the proposal in this draft should be able to alleviate the
> limitation for TCP connections between VMs behind same gateways, because
> in this case there is no real TCP segmentation performed by "offload
> engine".
>
> Let me know if you have more concerns, or maybe an example of how an
> option is broken by TSO/GSO, then we can check what's the current
> solution in kernel.

See above - and thanks,

Joe