Re: [multipathtcp] Proposed changes to the protocol draft

[Costin Raiciu <[e-mail]>]

Hi,

> First some comments on "opening new subflows in fallback mode":
> 
> In case checksums are used, I agree with your inclination to restrict fallback mode to regular TCP. If not checksums are used, however, there are no payload-rewriting middleboxes (by definition). Under such circumstances, DSS with infinity setting could be interpreted as "use this mapping until updated via a new DSS". The next DSS can be sent on any packet and on any subflow.
> 
> I strongly advocate such "temporary fallback solution in absence of checksums" 

If checksums are not used, MPTCP should not go into fallback mode - failed checksums were the only ways MPTCP would go into fallback mode. 
Since fallback cannot really happen (or will happen very rarely, if the spec is updated), there is no point in allowing  mobility in fallback mode. 

Fallback mode was designed as a way to signal to endpoints running MPTCP to revert to TCP from a certain sequence number onwards, in the middle of the connection. We expect this to be very rare in practice, so there is no need to add unnecessary complexity for this fallback mode.

MPTCP can revert to TCP in the beginning of the connection also; after this it never comes back to MPTCP mode. Why should we allow MPTCP to revert back from TCP in fallback mode (i.e. mid connection downgrade), and not allow it to revert back in some cases when the connection has been downgraded in the beginning (for instance because options did not get thourgh on data segments)?

Adding complexity to an already pretty complex protocol is probably the wrong thing to do. I think we should keep things simple and mandate that: 

"Once MPTCP reverts to TCP, it MUST NOT revert back to MPTCP aftewards". 


> Here are some comments on your ideas about the transparent proxy. If I understand you right, you offer 3 potential solutions:
> 
> a) Add a byte to ADD_ADDRESS for the P flag.
> 
> b) Use one bit of the address-id for the P flag.
> 
> c) Use address-id = 0 in ADD_ADDRESS (or MP_JOIN), which is usually the default for the initial subflow.
> 
> I have a preference for solution c). Some issues:
> - Q: What happens to the address-id of the interface used for the initial subflow (whose value was 0)? A: This address cannot be used anymore. The initial subflow, however, does not have to be torn down. Once torn down, it cannot be re-established since the address is gone. Is there any problem here?
> - Q: How to distinguish "do join" from "I'm a proxy"? A: Address-id = 0 means "I'm a proxy". Address > 0 means "do join". Is that right?

Actually, the only usable solutions in my mind are a) and b)
They would allow a proxy to send an add_address with a new address_id and a proxy flag, telling the endpoint that the advertised address belongs to a proxy.

However, this by itself does not allow the endpoint to tell if it is talking to a proxy on its initial subflow, or any other subflows for which is has not received information via ADD_ADDRESS with the P flag set and the corresponding address ID.

I was using solution (c) for this exact purpose. Once an address is first signalled implicitly (via subflow creation with MP_CAPABLE (address_id = 0) or MP_JOIN (address_id = x)), the proxy can also send an ADD_ADDRESS with the same address_id (0 or x), thus informing the endpoint that is a proxy address.

We thus need to change the rules on accepting ADD_ADDRESS options: when duplicate options are received for the same address ID, they should be processed nevertheless. This shouldn't introduce new vulnerabilities because subflow setup is the thing that matters and that uses crypto.

Cheers,
Costin




> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Costin Raiciu
> Sent: Thursday, September 08, 2011 3:40 PM
> To: Alan Ford; philip.eardley Eardley; [e-mail]; Yoshifumi Nishida; [e-mail] List
> Subject: [multipathtcp] Proposed changes to the protocol draft
> 
> Hi all,
> 
> As you know, we are in the stage of finalizing the MPTCP protocol draft. There are two major changes still to be made. Having re-read recently all the emails that have gone back and forth, here are the resolutions I propose.
> 
> The first question relates to the ability of opening new subflows in fallback mode (break before make in fallback is a specialization of this case).
> 
> My inclination would be to mandate the following:
> 
> - when MPTCP goes into fallback mode, it effectively becomes equivalent to a regular TCP session; no further subflows can be setup; teardown happens only with regular FIN messages, etc.
> 
> This does prevent mobility in certain cases that would require a lot of complexity to handle; it seems reasonable given my hope that payload changing middleboxes are not that ubiquitous.
> 
> The second question regards adding support for transparent proxies. In particular, each endpoint should know, for each address of their peer, if it is talking to a proxy or the destination. This would allow a smooth transition from
> an initial proxy based deployment in cellular networks to worldwide support for MPTCP in servers. In particular, MPTCP could connect to the proxy
> 
> The proposal is to add the P flag to ADD_ADDRESS, to allow middleboxes to say they are on path.
> 
> Currently ADD_ADDRESS does not have flag space;
> - we could add another octet for flag space (but this is not ideal since this option is already big enough for IPv6, as Georg pointed out)
> - or we could reduce the number of bits in the address ID by one. 256 addresses seem like a lot of interfaces to have at the same time; 128 should suffice. We should explicitly allow the number to wrap, though, to support long term mobility where a mobile's Wifi interface comes up and down hundreds of times. If we do this, we also free one bit on the MP_JOIN.
> 
> My inclination is to go for the second option.
> 
> When I first thought of it, it seemed to me that adding the P flag only to add_address is not sufficient; the host does not know upon directly opening a connection/subflow whether it is talking directly to the destination or a proxy.
> It turns out that using add address the proxy can make itself known for the initial path. This path will have address ID 0; the proxy will just send an add address with ID 0 and it's real address. The receiver now compares this address with what he thinks the remote endpoint is, and can infer it is talking to a proxy on that path. The same applies if a new subflow is opened.
> 
> What do people think? Are these reasonable solutions? If people agree, I will go ahead and rev the draft accordingly.
> 
> Cheers,
> Costin
>> 
>> On 6 Sep 2011, at 18:31, Alan Ford wrote:
>> 
>>> Folks,
>>> 
>>> Does anybody agree/disagree with my comments below? Can anybody contribute text for (2) and (5)? These are the main ones that I don't know enough about what was discussed and was intended in order to write anything myself.
>>> 
>>> Thanks,
>>> Alan
>>> 
>>> On 22 August 2011 22:08, Alan Ford <[e-mail]> wrote:
>>> Apologies for the delay in replying.
>>> 
>>> (1) - This won't work, unfortunately - we don't have enough option space.
>>> 
>>> (2) - I'm afraid I'm out of touch on what the conclusion here was. When I last thought about this, the only way I could see it working was if a subflow was clearly terminated before a new one was started, relying on the timeout you talk about in (3) to keep the MPTCP state alive. Was there an alternative way?
>>> 
>>> (3) - Indeed this is heuristics, although more so than for regular TCP applications may make assumptions about session survivability. This is one of those issues that we need to say needs thought in the protocol draft, and then in the API draft talk about its actual impact.
>>> 
>>> (4) - ok
>>> 
>>> (5) - Once again I'm out of touch of what was discussed at Quebec so am not sure how this would be defined. I remain uneasy about this flag - embedding proxies as a fundamental part of the protocol - but am willing to bow to consensus at Quebec.
>>> 
>>> Cheers,
>>> Alan
>>> 
>>> 
>>> On 19 August 2011 08:15, <[e-mail]> wrote:
>>> Hi,
>>> 
>>> Just a reminder, please can we try and close these issues
>>> 
>>> (1)    no-one complained at Mark's suggestion
>>> 
>>> (2)    Needs some slightly revised text
>>> 
>>> (3)    May need some tweaked text?
>>> 
>>> (4)    Needs no action
>>> 
>>> (5)    Is the main point requiring a detailed proposal
>>> 
>>> Thanks
>>> 
>>> phil
>>> 
>>> 
>>> 
>>> From: Eardley,PL,Philip,DES8 R
>>> Sent: 28 July 2011 00:01
>>> To: Eardley,PL,Philip,DES8 R; [e-mail]
>>> Subject: Open /closed issues from WG meeting
>>> 
>>> 
>>> 
>>> From the WG meeting I think we have a proposal on various Issues that seemed open before the meeting.
>>> 
>>> This email is to check consensus, especially amongst the WG people who aren't in Quebec or didn't listen in.
>>> 
>>> 
>>> 
>>> Of the issues, (1) was raised in the meeting, (2) & (3) are from Mark's slide 12, (4) and (5) are from Georg's slide 34.
>>> 
>>> 
>>> 
>>> (1) Ack of MP_CAPABLE.
>>> 
>>> See Mark's email to the list a short while ago, which explains & proposes a resolution.
>>> 
>>> 
>>> 
>>> (2) Mobility in fallback
>>> 
>>> The doc needs to correctly describe the case - when it works OK & when it doesn't (ie what the limitation is). We accept the limitation (it isn't too clear how to do a solution but seems it would be complex)
>>> 
>>> 
>>> 
>>> (3) Teardown of state when all subflows fail
>>> 
>>> This is a heuristics issue rather than a protocol issue.
>>> 
>>> (Hopefully in the future we can write a doc with lessons learnt from experiments, with suggestions how to deal with all sorts of heuristics issues)
>>> 
>>> 
>>> 
>>> (4)  Add Bulk_transfer_optimisation flag to MP-Capable?
>>> 
>>> Don't add, seems like extra complexity for not much gain
>>> 
>>> 
>>> 
>>> (5) Add support for transparent proxy?
>>> 
>>> Rough consensus in the room in Quebec to add an "I'm a proxy" flag to ADD_ADDRESS.
>>> 
>>> Need the precise text to add to the draft.
>>> 
>>> 
>>> 
>>> We'd like to resolve all these issues within the next week or so - hoping that proves possible!
>>> 
>>> 
>>> 
>>> We will WG last call as soon as there's an updated draft that deals with these issues.
>>> 
>>> 
>>> 
>>> If I got something wrong, please shout.
>>> 
>>> 
>>> 
>>> thanks to everyone for the efforts to try and resolve the hopefully last remaining issues with the protocol spec.
>>> 
>>> Best wishes,
>>> 
>>> phil
>>> 
>>> From: [e-mail] [[e-mail]] On Behalf Of [e-mail] [[e-mail]]
>>> Sent: 27 July 2011 23:34
>>> To: [e-mail]
>>> Subject: [multipathtcp] Draft minutes of WG meeting
>>> 
>>> thanks to Andrew for taking notes.
>>> 
>>> 
>>> 
>>> Here are the draft minutes of the meeting, please send corrections /improvements
>>> 
>>> thanks
>>> 
>>> phil/
>>> 
>>> 
>>> 
>>> Minutes of MPTCP WG Meeting
>>> 
>>> WEDNESDAY, July 27, 2011
>>> 0900-1130  Morning Session I
>>> 
>>> Main aim:
>>> Conclude on all protocol issues. So that we can WG last call protocol (& API) docs immediately after IETF.
>>> 
>>> 
>>> 
>>> 1. Agenda bashing, note well, Milestones - Chairs
>>> 
>>> 
>>> 
>>> 2. API update - Michael Scharf
>>> 
>>> Doc ready for last call, no comments.
>>> 
>>> 
>>> 
>>> 3. Congestion control update - Mark Handley
>>> Done. Minor comments from IESG and David Black have been resolved.
>>> 
>>> 
>>> 
>>> 4. Protocol update & discussion of open issues - Mark Handley
>>> 
>>> Reviewers could review section 2 (overview)
>>> 
>>> Small changes to protocol
>>> 
>>> 
>>> 
>>> Discussion over sending key-A on the third Ack; there is an issue over the reliable delivery of the key.
>>> 
>>> Jana: B could echo back the options.
>>> 
>>> Mark: The issue is how to trigger stopping option sends.  We will double-check if the issue is covered and resolve if required.
>>> 
>>> 
>>> 
>>> Discussion on mobility in fallback mode
>>> 
>>> It doesn't seem that it is possible to reliably use mobility in fallback mode.
>>> 
>>> Andrew: Isn't that OK?  Can't you just say, if the ACKs haven't arrived, oh well, RST it.
>>> 
>>> Georg: The draft says that there's no more multipath signalling in fallback.  So you can't get here.
>>> 
>>> Mark: So, the only resolution seems to be that if you end up with un-ACKed data, you lose.
>>> 
>>> Costin: You have signalling in the SYNs
>>> 
>>> Mark: Which gives you a way out, if that is possible.
>>> 
>>> Georg: I have a proposal for how to cover this later.
>>> 
>>> Mark: It probably isn't worth trying to optimise this case.
>>> 
>>> What to do if all subflows are in time out?
>>> 
>>> Phil: You're saying that is an implementation specific heuristic?
>>> 
>>> Andrew: What say no interfaces are up?
>>> 
>>> Mark: Yes, doesn't need specified.
>>> 
>>> 
>>> Phil: Plan is, get these three issues resolved on the list, give it a week or so, and WGLC it.
>>> 
>>> 
>>> 
>>> 
>>> Experiments on "Is MPTCP deployable?"
>>> 
>>> Georg: You can't name service providers or enterprises, to what extent are the harmful boxes installed in providers vs enterprises?
>>> 
>>> Mark: Cellular providers have more, but no monopoly.  We can't claim this to be completely representative.
>>> 
>>> 
>>> 5. (new draft) Enhancements to Improve the Applicability of Multipath TCP to Wireless Access Networks - Georg Hampel
>>> 
>>> Mark: Best path probably has lowest energy per byte
>>> 
>>> Georg: If you have money or energy to burn, you might not care
>>> 
>>> Mark: Proxy.  One part of me is revolted by supporting transparent proxies explicitly, and we should support them.  Calling the flag I_AM_A_PROXY is more explicit.
>>> 
>>> Andrew: I agree there are always proxy devices "there"
>>> Andrew: ... there are good features here (is it important to tag this as "min" complexity, images are getting big for this part of the stack)
>>> Georg: How about recv buffer reduction?
>>> Andrew: good idea
>>> 
>>> Mark: I'm not totally convinced that buffer reduction is real.
>>> 
>>> Costin: Receive buffer reduction is orthogonal to implementation complexity
>>> 
>>> Mark: The other question, if you're in path selection mode, do you get the right behaviour over mobile networks.  Assert: you should be able to tell from signal strength etc. if it is a good link... not really.  I can't easily predict from wireless if it's decent, I have to actually try it.
>>> 
>>> Georg: The signal is not necessarily indicative, could have other variables
>>> 
>>> Andrew: Wifi links have to train up, which can take a few hundred packets
>>> 
>>> David: I have in mind a satellite PEP setup, where RTT could be radically different.
>>> 
>>> Mark: Our intuition is that you should only use one path most of the time, but you want to transition make-before-break and do a soft transition.  You need to get the links adequately probed.
>>> 
>>> Georg: Both of our points are based on speculation, we need to know how well you do if you shorten the overlap
>>> 
>>> Costin: that paper assumed there is an oracle that knows the best path that chooses the best path yes: 15% compared to an optimal path selection algorithm that can't be achieved in practice
>>> 
>>> Costin: you can also have redundant data if you want to probe and avoid hol blocking
>>> 
>>> Mark: The spec does not prohibit you sending data on multiple paths
>>> 
>>> Phil: We've had quite a lot of discussion now on heuristic stuff, how you might make this work in practice.  It would be nice to have more experiments.  My impression is that BTO doesn't seem to gain very much
>>> 
>>> Andrew: Suggest not doing bulk transfer optimisation, it could be really expensive for little gain.
>>> 
>>> Andrew: Very much in favour of proxy flags, they're everywhere
>>> 
>>> Mark: I am in favour of adding a bit to support transparent proxies, 1) address of proxy 2) remote supports mptcp - could decide based on this
>>> 
>>> Andrew: It can be a content inspecting firewall that just wants to do its job
>>> 
>>> Georg: Proxy is not optimisation, it's a feature
>>> 
>>> Michio: No such middle box
>>> 
>>> Mark: If we can preserve the benefits in the presence of boxes that want to be on path, then we should.  If we don't want the content inspected, we need to encrypt.
>>> 
>>> Georg: 3G providers still get to bill it.
>>> 
>>> Gorry: There are proxies that need to inform L2 what is going on.
>>> 
>>> Andrew: Could have network architectures that wouldn't otherwise work that way.
>>> 
>>> Mark: We should probably try to address this before we move the protocol forward.
>>> 
>>> Phil: Would like to see people get together in the remainder of the week.
>>> 
>>> Andrew: I think the bottom three lines of the slide are OK, up to naming.
>>> 
>>> Costin: +1
>>> 
>>> 
>>> 
>>> 6. Call for news on Implementations
>>> 
>>> Linux version being updated as http://mptcpinfo.ucl.ac.be, also for Nokia N900 & Android Nexus-one
>>> 
>>> Tim: Implementation is not SMP capable, does anyone know if it is fixed?
>>> 
>>> <apparently not, docs still say no SMP>
>>> 
>>> Georg Hampel - in progress on 'lightweight' implementation
>>> 
>>> Sebastien Roy: Oracle are working on a prototype in Solaris.  Don't know if it can be released yet.
>>> 
>>> 
>>> 
>>> 7. Discussion on future of WG - Chairs
>>> 
>>> 
>>> 
>>> Discussion last time (roughly) concluded that we pause for experiments & usage: WG hears about implementations, heuristics etc; hears about investigations on possible extensions; not do any protocol standards work. Re-charter in ~ 1year when understand better what the needs are & where the energy is
>>> No comments - Chairs will work with AD on re-chartering text, when the current charter is done.
>>> 
>>> 
>> 
> 
> _______________________________________________
> multipathtcp mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/multipathtcp