Re: [multipathtcp] High-level design decisions /architecture

[Joe Touch <[e-mail]>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



William Herrin wrote:
> On Mon, Nov 9, 2009 at 10:27 PM, Joe Touch <[e-mail]> wrote:
>> That requires that options are processed on the data after ordered, as
>> you note. However, TCP is an unordered protocol.
>>
>>> My first thought would be a three-part handshake. Request option X,
>>> option X accepted, option X in effect. Requires that the receiver
>>> understand that order is important after sending X accepted though.
>> Right. How do you bury that handshake in an unordered, unreliable stream
>> without halting it, though?
> 
> Hi Joe,
> 
> How about a 4-step handshake like this:
> 
> Request option X, option X accepted, option X in effect (1), option X
> in effect (2).
> 
> When you receive X accepted, attach X1 to *every* outgoing packet
> until you get an ack for everything up to and including the first X1
> packet.

What happens to retransmitted packets that were first sent before you
decided to set X?

> Then start sending X2 in every outgoing packet until you get
> an ack for any X2 packet. Then stop sending the X option on any packet
> that isn't a retransmission of one where X1 or X2 was sent.
> 
> At the receiver end: until you receive a packet with X2, only packets
> with X1 or X2 have option X in effect.

So now you have a problem. Some packets are handled as if option X is in
effect, and some are not. These could arrive in any order. What if the
option has something to do with the order in which the packets are ACKd?
Or the order in which they're sent to the app? You now have essentially
two separate data streams, so you need to reimplement buffering to
handle the old stream as it was, and the new stream treats the option
exchange as a SYN-SYN/ACK-ACK exchange.

The issue is that the receiver cannot act on X until the handshake
completes (it gets X2). None of the data with X1 in it can be processed
until X2 is received. But this "processing" could mean ACKing too - you
can't even ACK the data because the option could mean "ACK every four"
or something like that.

> After you see any packet with
> X2, all subsequent packets have option X in effect regardless of the
> presence of X1 or X2.

Not packets sent before X was set at the source. Again, you have two
separate streams now - the one before X, and the one after X2. The first
is your current connection, which is essentially draining out. The
second is your new connection - with X set - that is basically starting
anew.

At that point, it's going to be a LOT simpler to just treat this as a
CLOSE and subsequent OPEN than to do this all inside a single connection.

> Nobody pauses.

Disagree - see above.

> Options are processed in the order received.

Disagree - see above. They can't be.

> And you're
> dealing with pretty trivial state changes on each end.

Disagree - you now need another 11-state machine to talk about all the
possible ways in which the "set X" operation can fail, and to avoid
trying to negotiate another option while the first negotiation is going on.

> One concern that jumps to mind is if the receiver gets a late copy of
> an already received and acked segment where option X is not in effect.
> What will the receiver do with the late packet?  Let me narrow that a
> bit: assuming option X does not affect the use of the TCP sequence
> number in any way, what will the receiver do with a late copy of an
> already received packet where it thinks X is in effect but in reality
> X was not requested?

I think at best that there would need to be a LOT of constraints on what
the option could do. But I'm not sure there's a useful way even if
that's true to make this work.

> On the other hand, if option X has some other intrinsic value (such
> has holding the multipath TCP connection ID for middleboxes to decode)
> then you can just keep sending it in every packet which uses it for
> the remainder of the connection instead of the fancy handshake.

Options can add asynchronous information - which is how they're used
now. They simply cannot add synchronized info - which is why they can't
be used to negotiate new options - AFAICT.

Joe
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)

iEYEARECAAYFAkr510cACgkQE5f5cImnZrtlpgCg7q4Dm/RtIcnNOfTkpPWlzini
JVcAn0zJpg/7Nxgx+IPTucWKqr9R7llX
=20o5
-----END PGP SIGNATURE-----