Re: [multipathtcp] A proposal for MPTCP Robust session Establishment (MPTCP RobE)

[<[e-mail]>]

Dear Yoshi,

see inline

> -----Original Message-----
> From: Yoshifumi Nishida [mailto:[e-mail]]
> Sent: Mittwoch, 4. Oktober 2017 09:53
> To: Amend, Markus <[e-mail]>
> Cc: multipathtcp <[e-mail]>
> Subject: Re: [multipathtcp] A proposal for MPTCP Robust session
> Establishment (MPTCP RobE)
> 
> Hi Markus,
> 
> I still need to think about it a bit more, but here's my current thoughts.. Sorry
> for slow response.
> 
> 1: I am guessing we cannot use this approach with TFO yet. But, if TFO cannot
> be used, it might affect the latency of some apps.
> 
We assume, RobE can incorporate TFO. Therefore we have different solutions in mind. But first we need an aligned understanding of TFO. There is one aspect, which isn't fully clear for me or respectively from my view not specified explicitly in RFC7413.
TFO allows payload in a SYN and in case of a valid TFO cookie, this is directly delivered to receivers application. Now the question:  Is the sender allowed to proceed directly after sending the SYN with further data packets or is a SYN/ACK response required?
By investigating current Linux TFO implementation it seems, sender first wait for the SYN/ACK before sending further data packets. But what shall we assume? At the end this is pretty important for a possible RobE<->TFO integration.

> 2: Although I am not sure as I'm not an implementor, but I feel it could be a
> bit tricky to implement MP_JOIN_CAP as servers cannot be sure if this
> subflow becomes primary or secondary until several packets are exchanged.
> (also, I'm not sure how to merge connections when TFO is used.)
> 
until MP_JOIN_CAP or third MP_CAPABLE is received all connections are potential master.
The path receiving MP_JOIN_CAP, then will be converted to a subflow.
That are not several exchanges but only one round-trip., compared to RFC6824(bis) the same or even better because MP_JOIN isn't necessary.
From an implementation point of view this is rather simple.

> 3: I guess we'll need another step after SA6.2 to verify the response from
> host B.

no, we don't. data can be sent together with the MP_JOIN_CAP and immediately afterwards. if it is not successful, either we receive a RST (standard case) or we run into a timeout (error case). In both cases a retransmit over the first path is done.

>     Also, MP_JOIN_CAP will require extra RTTs when the other end doesn't
> support it. (because it will need to send RST and MP_JOIN to establish a
> subflow)
> 
yes, indeed this is a problem. there are several possibilities to address that problem:
a) modify the MP_CAPABLE, adding an extra bit in syn/syn-ack. (e.g. one of 
   the A-H bits).
   however this violates the standard, and thus will be incompatible with
   existing implementations - unless it is introduced together with
   ver. 1 - which already is incompatible, so it doesn't matter.
b) if introduced together with ver. 1 - we know that the server supports
   it if it supports ver. 1 - however it still might be deactivated.
c) add an extra MPTCP-option. 
   in version 0, this can exhaust the tcp option space.
   in version 1, (due to missing key info) this is feasible, if TFO
   is not active. with TFO we already have that info, an extra option
   is not necessary. 
d) cache that information, so only the first try adds an extra rtt.
   later on (until cache expiration) we don't try fast-join again.
e) sending an MP_JOIN_CAP + sending an MP_JOIN (note the join is a
   new connection request (SYN), but can be associated with the
   MP_JOIN_CAP by the key/token).
   - In case the MP_JOIN_CAP succeeds, the MP_JOIN is ignored.
   - In case it fails, the join is initiated before the RST arrives
     on sender site. Avoiding the extra RTT, with the cost of
     extra packets on the wire.
   - The problem with reordering on the wire (MP_JOIN arrives before
     MP_JOIN_CAP), can be avoided using one of the three reserved bits
     in the MP_JOIN. These should be ignored by current implementations.
f) a combination of those above.

> 4: I tend to prefer just sending MP_JOIN after SA6.1 for simplicity so far. We
> might be able to do everything in a shim layer.
> 
Of course it is possible to do it in two steps. First only robustness and later on extend it with the fast-join approach.
But why a shim-layer, do you suspect some incompatibility issues? Or do you just want to keep it out of MPTCP? 
But then note: Even that shim-layer needs a path-management. Either MPTCP has to provide a standard API to access the path-management from the outside world.
Or a new path-management has to be developed, including address-add and address-remove. Thus you need to define a new protocol (either extra TCP-options or some new ICMP-types...). Once implemented the path-management outside of MPTCP, it doesn't make sense to have a second one in MPTCP itself.
So you can throw it out, and use the path-management from the outside world.
For sure an interesting approach. However it is everything but simple!

Further having RobE outside of MPTCP makes it impossible to extend it with the fast-join, or you have to provide some extra APIs in MPTCP to enable the fast-join from outside!

BR

Markus
> --
> Yoshi
> 
> On Tue, Sep 19, 2017 at 3:00 AM, <mailto:[e-mail]>
> wrote:
> Dear all,
> 
> any comments to my proposal from two weeks ago?
> 
> BR
> 
> Markus
> 
> > -----Original Message-----
> > From: Amend, Markus
> > Sent: Montag, 4. September 2017 14:17
> > To: mailto:[e-mail]
> > Subject: RE: [multipathtcp] A proposal for MPTCP Robust session
> > Establishment (MPTCP RobE)
> >
> > Dear all,
> >
> > Based on feedback at IETF99, regarding MPTCP RobE proposal
> > (https://datatracker.ietf.org/meeting/99/materials/slides-99-mptcp-a-
> > proposal-for-mptcp-robust-session-establishment-mptcp-robe), I
> completely
> > reconsider the original approach, I presented. Remember, the
> "Downgrade"-
> > approach (see slides) was preferred from my side, because it combines
> > robustness and a huge latency reduction.
> >
> > The main feedback I got was:
> >
> >   (1) Relying on SYN-Key-A as identifier to combine potential initial flows,
> raise
> > several issues
> >     (1.1) not available in RFC6824bis anymore
> >     (1.2) security, because
> >         (1.2.1) Key-A is transmitted many times
> >         (1.2.2) can be abused by accident/intend
> >   (2) Separate robustness and latency functionality
> >   (3) To much computing resources needed on receiver side to detect
> > duplicate Key-A
> >   (4) Do it in the application layer library (build a shim layer); not in the
> > MPTCP standard and/or implementation to reduce complexity
> >
> > With the following reworked proposal (based on the "Downgrade"
> approach,
> > see slides), I try to address all of the points above.
> >
> > With Fig. 1, the flow diagram for the new proposal is depicted. It is referred
> > to MPTCP version 0, but should be equivalent to probable version 1
> > (RFC6284bis), just without a SYN-MP_CAPABLE containing Key-A.
> > To reach robustness, several separate MPTCP initial flows are initiated at
> the
> > beginning, in the following denoted as "potential initial flows". In step SA1
> > and SA2, respectively SB3 and SB4, standard MPTCP initial flow procedure is
> > applied, so no change so far. With occurring SA3/SA4 the requester "Host
> A"
> > has to decide which potential initial flow will be used further as initial flow
> > (e.g. the first which returns) and what shall happen with the other
> potential
> > flows. Either it can break all but one (e.g. sending RST at SA6.1) OR recycle
> > them (step SA6.2). Thereby, it is a full client side decision to use robustness
> > only or benefit from latency reduction as well and can be toggled
> separately
> > by e.g. sysctl globally and/or socket option.
> > Until SA6.1, it doesn't require any MPTCP standard modification!
> >
> >               Host A                                  Host B
> >      ------------------------                       ----------
> >      Address A1    Address A2                       Address B1
> >      ----------    ----------                       ----------
> >          |             |                                |
> >          |            SYN + MP_CAPABLE(Key-A)           |
> >  (SA1)   |--------------------------------------------->|  (SB1)
> >          |             |    SYN + MP_CAPABLE(Key-A')    |
> >  (SA2)   |             |------------------------------->|  (SB2)
> >          |             |                                |
> >  (SA3)   |<---------------------------------------------|  (SB3)
> >          |          SYN/ACK + MP_CAPABLE(Key-B)         |
> >  (SA4)   |             |<-------------------------------|  (SB4)
> >          |             |   SYN/ACK + MP_CAPABLE(Key-B') |
> >          |             |                                |
> >          |        ACK + MP_CAPABLE(Key-A, Key-B)        |
> >  (SA5)   |--------------------------------------------->|  (SB5)
> >          |             |             RST                |
> >  (SA6.1) |             |------------------------------->|  (SB6.1)
> > robust   |             |                                |
> > ness     |             |                                |
> > -------------------------------------------------------------------
> > latency  |             |                                |
> > red.     |             | ACK + MP_JOIN_CAP(Key-A, HMAC) |
> >  (SA6.2) |             |------------------------------->|  (SB6.2)
> >
> >    Key-B_fast_hash = crc16(Key-B) & 0x3FF               -> (10bit)
> >    HMAC_keys =  HMAC(Key-A+Key-B+Key-B')                -> (>=96bit)
> >    HMAC =  (HMAC_keys & ~0x3FF) | Key-B_fast_hash       -> (size same as
> > HMAC_keys)
> >
> > Figure 1: Example Use of MPTCP RobE (for MPTCP version 0)
> >
> > For not wasting resources by breaking all but one and benefit in addition
> > from a huge latency reduction (see MPTCP RobE proposal, p.12,14+15,17),
> > one can make use of a new proposed MPTCP option "MP_JOIN_CAP",
> > illustrated in Fig. 2. According to Fig. 1, A1<->B1 is assumed as initial flow at
> > SA3 and A2<->B1 shall be recycled at SA6.2 (SA6.1 is skipped therefore).
> > Instead of sending an MP_CAPABLE in the third ACK, an MP_JOIN_CAP is
> > sent, which includes Key-A of the selected initial flow A1<->B1 and an
> HMAC,
> > coding Key-A, Key-B, Key-B' in a determined way (see Fig. 1 bottom).
> >
> >                    1                   2                   3
> >  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> > +---------------+---------------+-------+-------+---------------+
> > |     Kind      |    Length     |Subtype|       |    ADDR_ID    |
> > +---------------+---------------+-------+-------+---------------+
> > |                    Sender's Key-A (64 bits)                   |
> > |                                                               |
> > |                                                               |
> > +---------------------------------------------------------------+
> > |                        HMAC (>=96 bits)                       |
> > |                                                               |
> > |                                                               |
> > :                                                               :
> > +---------------------------------------------------------------+
> > Figure 2: Multipath Join Capable (MP_JOIN_CAP) Option (for Third
> > MP_CAPABLE ACK)
> >
> > For the MP_CAPABLE, ADDR_ID is set to zero (like RFC6824) and for
> > MP_JOIN_CAP it is sent with.
> >
> > Which boundary conditions can happen:
> >
> > 1. Like Fig. 1, A1<->B1 is assumed as initial flow and MP_CAPABLE receives
> > first at Host B (SB5), A2<->B1 shall be recycled and ACK is sent with
> > MP_JOIN_CAP
> >    If MP_JOIN_CAP is received it has to iterate once (like MP_JOIN) the
> > connection list and check for Key-A availability. If a Key-A connection is
> > found, it is validated against HMAC value. The validation has two reasons,
> > firstly several Key-A can exist, because different Hosts choose same Key-A
> by
> > accident. Furthermore no one can join a connection by just
> recording/brute-
> > forcing Key-A and duplicate the request. (solves point (1) from the
> feedback)
> >
> > 2. Like 1., but MP_JOIN_CAP arrives before the MP_CAPABLE at Host B
> >
> >     2.1 draft-ietf-mptcp-rfc6824bis-09
> >         Based on Key-A, it will iterate over the connection list, but it will not
> > match, because Key-A from formerly selected initial flow hasn't arrived yet.
> > So it will continue with a fast iterate only over the connections, which are
> still
> > in establishment phase, using the 10bit Key-B_fast_hash (crc16(Key-B) &
> > 0x3FF). If it matches against a (precomputed) existing Key-B_fast_hash in
> the
> > connection list, it will validate with the HMAC(Key-A+Key-B+Key-B') again
> to
> > ensure legitimation. If successful, both, the initial flow and the
> MP_JOIN_CAP
> > flow, can be immediately established. This is true, because without the
> > knowledge of Key-B, Host A couldn't calculate the HMAC. So it is clear, that
> > host A had received the SYN/ACK (SB3).
> > This also mitigates the requirement of a reliable ACK (draft-ietf-mptcp-
> > rfc6824bis-09, p.15) exchange, because only one
> > MP_CAPABLE/MP_JOIN_CAP needs to reach receiver side.
> >
> >     2.2 MPTCP version 0 (RFC6824)
> >         Can match based on Key-A, like 1., same effort as for a MP_JOIN.
> >
> > 3. A2<->B1 is selected as initial flow, because according SYN/ACK returns
> > earlier at Host A.
> >     It is the same as 1. and 2., just the other way round :-)
> >
> > With these boundary conditions it has been shown, that no additional
> > computing resources are required (see feedback (3)), compared to todays
> > MPTCP, with a minimal exception of 2.1.
> >
> > With such a simple approach, and by adding an inconspicuous simple
> MPTCP
> > option MP_JOIN_CAP, MPTCP would be extended towards robustness and
> if
> > desired, latency reduction can be enabled on top, but obviously requires
> > receiver support. Negotiation is inherently given by the new
> MP_JOIN_CAP
> > which is either accepted or not. If not, the standard MP_JOIN process can
> be
> > applied.
> >
> > Because of such a simplicity and the massive benefit by optional latency
> > reduction, I still believe it fits best in the MPTCP layer itself and not in an
> > application shim layer (see feedback (4)).
> >
> > Due to the separation of robustness and latency reduction functionality, it
> > offers the chance to implement "robustness" in plain TCP as well. It could
> > look the same like in Fig. 1, (SA1)-(SA6.1), just without MP_CAPABLE
> option.
> > But be aware, it needs some kind of path management, which is already
> > present in MPTCP, but not in TCP. Furthermore I assume "robustness" is a
> > topic more related to taps working group, whereas "latency reduction" is
> > related to MPTCP obviously and depends on "robustness"!
> >
> > BR
> >
> > Markus
> _______________________________________________
> multipathtcp mailing list
> mailto:[e-mail]
> https://www.ietf.org/mailman/listinfo/multipathtcp