Re: [iccrg] [tsvwg] New Version Notification for draft-welzl-irtf-iccrg-tcp-in-udp-00.txt

[Tom Herbert <[e-mail]>]

On Fri, Mar 25, 2016 at 10:29 AM, Michael Welzl <[e-mail]> wrote:
> Hi!
>
>
>> On 25. mar. 2016, at 18.00, Tom Herbert <[e-mail]> wrote:
>>
>> Hi Michael,
>>
>> For encapsulation format I suggest that you look at GUE
>> (https://tools.ietf.org/html/draft-ietf-nvo3-gue-02). GUE can
>> encapsulate TCP in UDP and inserts a GUE header that allows extra meta
>> data.  The connection ID could be moved to the GUE header so that no
>> special modification of the encapsulated TCP header would be needed.
>> The connection ID would be a new field in the GUE header. There is
>> already a session ID option defined, but that is 96 bits which is
>> probably overkill.
>
> I have, as I wrote this draft. It struck me as something that isn’t really necessary just for the purpose of what TCP-in-UDP is trying to achieve. In particular, other than GUE (if I got it correctly), TCP-in-UDP strictly assumes end-to-end operation, and requires changing the sender-side TCP code too because the main point is to combine congestion controls. This makes it possible to do whatever we want with the encapsulated TCP header.
>
Then you are redefining TCP. You can do that, but this substantially
reduces the possibility of deployment. In real networks we need
tcpdump, netflow, diagnostics, other debugging tools. Encapsulation
works best when it does not require changes to the encapsulated
packet. Besides that if your willing to change TCP for this use, why
not just go a little farther and use SCTP/UDP which I believe already
has a concept of shared congestion window amongst sub-flows?

>
>> Another consideration is the UDP checksum. This must be set in IPV6
>> (excepting that the requirements in RFC6935 and RFC6936 are met) and
>> is recommended for encapsulation any way. This means that there are
>> two checksums (TCP and UDP) per packet which becomes a performance
>> issue since most NICs can offload at most one checksum and often that
>> is restricted to only the checksum in a plain TCP or UDP packet with
>> no encapsulation. We solved this in GUE with remote checksum offload
>> (https://tools.ietf.org/html/draft-herbert-remotecsumoffload-02) which
>> should work fine with TCP over UDP case.
>
> We (and the preceding proposals that are cited in the draft) removed the redundant TCP checksum from the encapsulated TCP header. Because it’s end-to-end and changes TCP anyway, we can do that.
>
That will break our ability to use NIC offload with TCP/UDP
(segmentation offload for instance). Everything, including many
performance optimizations, is already designed around the requirement
that the TCP checksum must be set. Trying to undo that requirement is
actually a complication, not a simplification in practice.

Tom

>
>> There are some other end system issues in dealing with UDP
>> encapsulation that we considered in Linux (load balancing, checksum,
>> segmentation offload). Please look at
>> http://people.netfilter.org/pablo/netdev0.1/papers/UDP-Encapsulation-in-Linux.pdf.
>
> Thanks for the pointer!  But from quickly looking at it, the TCP-in-UDP case really seems simpler than most of this. E.g., in fig.1, a host kernel has to handle packets coming from a guest kernel - it’s not changing the guest kernel’s TCP code.
>
> BTW I’m not at all insisting on my encapsulation method, just explaining the design rationale. Anyway I’m beginning to think that the next version of the draft should make a cleaner split between the two elements: 1) congestion control coupling and 2) encapsulation (ensuring that TCP packets take the same path). The latter could be a separate method to support 2), where various options could be described: for v6, using the flow label as you suggested; for v4, the existing UDP encapsulation or something else if there’s a good reason (first I’ll need to read some more encapsulation stuff more carefully to fully appreciate the reasons).
>
> Cheers,
> Michael
>