[multipathtcp] Options or Payload: pros and cons

[Costin Raiciu <[e-mail]>]

Hi

I tried to assemble a pros and cons list for the two approaches.  
Please add bullets to both lists until we get something reasonably  
exhaustive, and then we can make an informed decision.

At a high level, we need to use options on every subflow at least in  
the syn exchange. Otherwise we can't attach the subflow to the  
existing connection, or can't negotiate mptcp in the first place.
Ideally you'd like fate sharing: if the syn/ack exchange passes, the  
subflow should work.

Options
+ nice with tcp
+ single subflow mptcp if friendly with IDSes, and traffic  
normalizers (i.e. data flow is what the app receives).
- reliability is more difficult (how often do you retransmit?) drop  
packets? what's the failure mode when options don't get through?
- boxes may crash when parsing unknown stuff. The hope is that on SYN  
processing is more robust with unknown options. ECN story is  
frightening :). Murari (Microsoft)
says that even changing position of padding bits in options crashes  
some middleboxes; I've heard a similar story from the Linux kernel  
experience. However, we can't (cleanly) avoid sending options on SYNs.

Payload
+ option space limits no longer matter; can do funkier stuff later,  
like negotiate keys, etc.
+ easier to get reliability; retransmit of control data linked to  
data retransmit
+ easier to deal with middleboxes that resegment (it just works!)
+ new options on syns may be allowed, but not on data (is this true)?
- feels like a hack, not the traditional way to evolve tcp
- implies encoding of any data sent MUST  be ASCII; middleboxes (ids,  
traffic normalizers) that look at content and see binary stuff will  
probably drop the data; finally, if we used binary stuff, there is no  
guarantee for fate sharing. . Text encoding is less space efficient;  
does it matter?

Any more?
Costin