Re: [multipathtcp] AD Review: draft-ietf-mptcp-architecture-03

["Ford, Alan" <[e-mail]>]

Thanks for your comments, Lars. A new version addressing them is
available at:

http://www.ietf.org/id/draft-ietf-mptcp-architecture-04.txt

Details of changes inline...

> -----Original Message-----
> From: [e-mail]
[mailto:[e-mail]] On
> Behalf Of Lars Eggert
> Sent: 17 December 2010 12:02
> To: [e-mail] Mailing List
> Subject: [multipathtcp] AD Review: draft-ietf-mptcp-architecture-03
> 
> Summary: good to go.
> 
> I'll start the IETF last call. Please take the comments below as
last-call
> comments. I will make this last call longer than normal, due to the
holiday
> season.
> 
> Lars
> 
> 
> Section 1., paragraph 2:
> >    By the application of resource pooling[3], these available
resources
> 
>   Nit: s/pooling[3]/pooling [3]/  (also check other reference
citations
>   for a missing space before them)

All fixed I think.

> Section 1.2., paragraph 0:
> > 1.2.  Terminology
> 
>   I'd be somewhat useful to reorder the terms such that definitions
>   appear before they are used, as much as possible.

This is slightly tricky due to interdependence, but "Multipath TCP"
moved to the top.

> Section 1.2., paragraph 2:
> >    Path Identifier:  Within the context of a multi-addressed
multipath
> >       TCP, a path is defined by the source and destination (address,
> >       port) pairs (i.e. a 4-tuple).
> 
>   Why is a Path identified by something that includes port numbers?
>   Because of ECMP?

Yes, this is explained in Section 5.5.

Quite why this definition was still there I don't know, I've removed it
- it adds nothing to clarity being this early.

> Section 1.2., paragraph 6:
> >    Subflow:  A flow of TCP packets operating over an individual
path,
> >       which forms part of a larger Multipath TCP connection.
> 
>   s/packet/segment/ (please check this throughout the document,
"packet"
>   is often used where "segment" is meant)

All changed where appropriate.

> Section 1.3., paragraph 5:
> >    Furthermore, the paths through
> >    the Internet may be interrupted by any number of middleboxes
> >    including NATs and Firewalls.
> 
>   Middleboxes don't really "interrupt" a path. Maybe rephrase with
"are
>   present"?

Difficult wording to get right... text changed to:

   Furthermore, the paths through the Internet often do not
   provide a pure end-to-end service, and instead may be affected by
   middleboxes such as NATs and Firewalls.

> Section 1.3., paragraph 6:
> >    Finally, although the diagram refers
> >    to the Internet, Multipath TCP may be used over any network where
> >    there are multiple paths that could be used concurrently.
> 
>   Not sure if there is much value in mentioning this. (Plus, we do
>   assume an _IP_ network.)

Removed.

> Section 2.2.1., paragraph 3:
> >    A multipath-capable equivalent of TCP SHOULD retain backward
> >    compatibility with existing TCP APIs, so that existing
applications
> >    can use the newer transport merely by upgrading the operating
systems
> >    of the end-hosts.
> 
>   Any reason why this is not a MUST? I think this is critical;
otherwise
>   we'd already have seen widespread use of SCTP.

Only compatibility with the most commonly used features is really
required. Anyway, text changed.

> Section 2.2.2., paragraph 1:
> >    In the traditional Internet architecture, network devices operate
at
> >    the network layer and lower layers, with the layers above the
network
> >    layer instantiated only at the end-hosts.  While this
architecture,
> >    shown in Figure 2, was initially largely adhered to, this
layering no
> >    longer reflects the "ground truth" in the Internet with the
> >    proliferation of middleboxes[8].  Middleboxes routinely interpose
on
> >    the transport layer; sometimes even completely terminating
transport
> >    connections, thus leaving the application layer as the first real
> >    end-to-end layer, as shown in Figure 3.
> 
>   And even application level protocols often define roles for
>   intermediaries, e.g., SIP and HTTP.

These are application-level intermediaries, however, which is a bit out
of scope for MPTCP. The section is concentrating on middleboxes that
exist at the transport layer. I couldn't find an appropriate way of
inserting this into this section without going off on a
non-MPTCP-relevant tangent; I hope you're ok with it still left out.

> Section 5.2., paragraph 6:
> >    Therefore, to provide a fully robust multipath TCP solution,
MPTCP
> >    SHOULD feature explicit connection-level acknowledgements, in
> 
>   After all of this argumentation in the previous paragraphs, why is
>   this not a MUST? Esp. since in the first paragraph you already say
>   "MPTCP features..." which in essence is a MUST.

It doesn't need to be a MUST if the implementation can be sure of
behaviour of the network and endpoints. So I've changed it to: "MPTCP
for use on the public Internet MUST feature explicit connection-level
acknowledgements".

> Section 5.3., paragraph 6:
> >    The resulting buffer size should be small enough for practical
use.
> >    However, there may be extreme cases where fast, high throughput
paths
> >    (e.g. 100Mb/s, 10ms RTT) are used in conjunction with slow paths
> >    (e.g. 1Mb/s, 1000ms RTT).  In that case the required receive
buffer
> >    would be 12.5MB, which is likely too big.  In these cases a
Multipath
> >    TCP scheduler SHOULD use only the fast path, potentially falling
back
> >    to the slow path if the fast path fails.
> 
>   Please rephrase the last sentence so that it not specific to the
>   example given, since we're trying to state a general rule here.
Maybe:
>   "it may be prudent to only use some of the fastest available paths
for
>   the MPTCP connection" or something like that

Now reads:

   In extreme cases such as
   this example, it may be prudent to only use some of the fastest
   available paths for the MPTCP connection, potentially using the slow
   path(s) for backup only.

> Section 5.4., paragraph 3:
> >    The MPTCP protocol design will, however, use TCP Options for this
> >    additional signalling.  This has been chosen as the mechanism
most
> >    fitting in with the goals as specified in Section 2.
> 
>   Aside: Instead of using several different TCP Option numbers, I
wonder
>   if we can use one TCP Option Number for MPTCP and disambiguate based
>   on the Length field what kind of MPTCP option we have? (Assuming all
>   our different option kinds have different lengths...)

Seb's already answered that one - it's a bit limiting. However, to save
on option numbers, a sub-code could be used (at the cost of an
additional octet). But this is out of scope for the architecture, anyway
(it only says that Options will be used, not how they'll be used).

> Section 5.5., paragraph 1:
> >    Currently, the network does not expose multiple paths between
hosts.
> 
>   This isn't really capturing it. The network *does* expose this, *if*
>   the paths begin and end at different IP addresses. I think you mean
>   that the network doesn't expose the availability of path diversity
*in
>   the network* for paths between the same two IP addresses.

Now reads:

   Currently, the network does not expose path diversity between pairs
   of IP addresses.  In order to achieve path diversity from today's IP
   networks, in the typical case MPTCP uses multiple addresses at one or
   both hosts to infer different paths across the network.

> Section 5.5., paragraph 4:
> >    Multiple different (source, destination) address pairs will thus
be
> >    used as path selectors in most cases.  Each path will be
identified
> >    by a TCP 4-tuple (i.e. source address, destination address,
source
> >    port, destination port), however, which can allow the extension
of
> >    MPTCP to use such 4-tuples as path selectors.  This will allow
hosts
> >    to use MPTCP to load balance to different ports, for example if
the
> >    network routes different ports over different paths (which may be
the
> >    case with technologies such as Equal Cost MultiPath (ECMP)
routing
> >    [16]).
> 
>   So actually, you use a five-tuple, it's just that the protocol
number
>   component is always 6 (TCP) in our case. There are ECMP schemes
where
>   the resulting paths are different when only the protocol number
>   component is different. I'd prefer if we could talk about
five-tuples
>   and path IDs, since paths are not usually something that is specific
>   to one transport protocol. (And you actually do already talk about
>   five-tuples in the next section...)

Fair enough... All changed to "five-tuple".

> Section 5.5., paragraph 6:
> >    The modularity of path management will permit alternative
mechanisms
> >    to be employed if appropriate in the future.
> 
>   What modularity of path management? Nothing in this section so far
>   talks about modularity. Suggest to drop this paragraph.

I was going to drop it, but then thought it makes an important point,
just badly worded. Now reads:

   The path management is a separate function from the packet
   scheduling, subflow interface, and congestion control functions of
   MPTCP, as documented in Section 4.  As such it would be feasible to
   replace this IP-address-based design with an alternative path
   selection mechanism in the future, with no significant changes to the
   other functional components.

> Section 5.8., paragraph 11:
> 
> >    o  The desire to support a 'break-before-make' (as well as a
'make-
> >       before-break') approach to adding subflows implies that a host
> >       cannot rely on using a pre-existing subflow to support the
> >       addition of a new one.
> 
>   But I don't think we want to support "unlimited" break-before-make,
>   i.e., just because we had the connection up yesterday doesn't mean I
>   should still be able to join a new first flow into it today. In
other
>   words, we want to support break-before-make for some limited amount
of
>   break time only.

I've added "time-limited", but of course it's entirely up to
implementations, just like retaining TCP state today.

However long it is, it doesn't change the implication that's stated.

> Section 12.2., paragraph 7:
> >    [9]   Carpenter, B., "Internet Transparency", RFC 2775,
> >          February 2000.
> 
>   Unused Reference: '9' is defined on line 1108, but no explicit
>   reference was found in the text

I think the nits tool is broken :)

(Adding spaces fixed that one, it's always been there on Page 8).

I've also addressed Cullen's two minor comments by adding the following
to "Network Compatibility":

   Middleboxes may also cause some TCP features to be able to exist on
   one subflow but not another.  Typically these will be at the subflow
   level (such as SACK [10]) and thus do not affect the connection-level
   behaviour.  In the future, any proposed TCP connection-level
   extensions should consider how they can co-exist with MPTCP.

And the following to Interactions with Applications:

   TCP features the ability to send out-of-band ("Urgent") data.
   Although the use of Urgent data is not recommended (see [20]), MPTCP
   SHOULD still support this feature if requested by an application.  An
   MPTCP implementation SHOULD send the Urgent data on one subflow of
   the MPTCP connection; it MAY choose this to be the best performing
   subflow.

Thanks,
Alan