Re: [multipathtcp] Multipath TCP Address advertisement 2/5 - Reliability

Yoshifumi Nishida <> Mon, 08 August 2016 08:07 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C693F12D0CE for <>; Mon, 8 Aug 2016 01:07:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.147
X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id UOeRSuRVNIPn for <>; Mon, 8 Aug 2016 01:07:08 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 2B88D128874 for <>; Mon, 8 Aug 2016 01:07:07 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTPSA id F2161278544 for <>; Mon, 8 Aug 2016 17:07:05 +0900 (JST)
Received: by with SMTP id 97so6944797uav.3 for <>; Mon, 08 Aug 2016 01:07:05 -0700 (PDT)
X-Gm-Message-State: AEkoouv4mtqxXgaBjUt+ZGOle5oXkJJIKb9cfbF0Fg5wyYsEnch5cg05mYpQbD0dJT3Yo/prRl1aBUzpJZ8U0g==
X-Received: by with SMTP id q204mr41839997vkf.51.1470643624455; Mon, 08 Aug 2016 01:07:04 -0700 (PDT)
MIME-Version: 1.0
Received: by with HTTP; Mon, 8 Aug 2016 01:07:03 -0700 (PDT)
In-Reply-To: <>
References: <> <> <> <> <>
From: Yoshifumi Nishida <>
Date: Mon, 8 Aug 2016 01:07:03 -0700
X-Gmail-Original-Message-ID: <>
Message-ID: <>
To: Alan Ford <>
Content-Type: multipart/alternative; boundary=001a11438460d3b69705398ae653
Archived-At: <>
Cc: "" <>
Subject: Re: [multipathtcp] Multipath TCP Address advertisement 2/5 - Reliability
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Multi-path extensions for TCP <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 08 Aug 2016 08:07:11 -0000

Hi Alan,

On Sat, Aug 6, 2016 at 1:35 PM, Alan Ford <> wrote:

> Hi Yoshi,
> Inline...
> On Friday, 5 August 2016, Yoshifumi Nishida <>
> wrote:
>> Hi Alan,
>> On Thu, Aug 4, 2016 at 12:57 AM, Alan Ford <> wrote:
>>> Hi Yoshi,
>>> Inline…
>>> On 4 Aug 2016, at 07:36, Yoshifumi Nishida <>
>>> wrote:
>>> Hi Fabien,
>>> On Wed, Aug 3, 2016 at 8:47 AM, Fabien Duchêne <fabien.duchene@uclouv
>>>> wrote:
>>>> Hello,
>>>> As agreed in Berlin during IETF96, I'm sending a series of emails to
>>>> discuss the different contributions proposed
>>>> in
>>>> This is the part 2/5 : reliability.
>>>> In RFC6824, ADD_ADDR options can be attached to segments carrying data
>>>> or pure acknowledgements.
>>>> In practice, notably given the length of ADD_ADDR with IPv6 addresses
>>>> and the HMAC, it is very likely that they will be often sent as pure
>>>> acknowledgements.
>>>> This implies that ADD_ADDR are sent unreliably, which could be
>>>> problematic when the ADD_ADDR is required to allow the establishment of
>>>> additional subflows, as in load balancing scenarios.
>>>> We propose to rely on the "E" (Echo) flag in the ADD_ADDR option.
>>>> This echo flag is used to acknolwedge a received ADD_ADDR by echoing it.
>>>> If the acknowledgement is not received, the ADD_ADDR option will be
>>>> retransmitted up to N times.
>>> I have several comments on this. Please let me know if I miss something.
>>> 1: Do we really need ADD_ADDR reliability in all cases? An end node
>>> might want to send ADD_ADDR for 'just in case' rather than for "I want you
>>> to use this"
>>> BTW, I'm not sure if the draft tries to replace the current ADD_ADDR or
>>> to propose an additional ADD_ADDR option.
>>> The argument is that by making it reliable, you only need to send it
>>> once to know it’s been received, even if the far end does not use it.
>> 2: The sender cannot be sure whether the info in ADD_ADDR will be used or
>>> not. It depends on the peer's decision.
>>> I might prefer re-transmitting ADD_ADDR up to N times when the sender
>>> doesn't receive MP_JOIN to the address for a certain amount of time.
>>> As above, by making it reliable you only need to send it once to know
>>> it’s been received, even if the far end chooses not to use it.
>> Hmm. if a sender wants the peer to use specific address (such as a load
>> balancer case), I can understand that we want to make it reliable. But, if
>> the sender doesn't care, I'm not very sure the reason to make it reliable
>> or send ADD_ADDR frequently.
> If local policy means the subflow won't be immediately established,
> there's no way of knowing if it's been received.
> Similarly, if the subflow can't be created due to e.g. firewall/network
> issues, it would be useful to know if the ADD_ADDR had nevertheless been
> received, so you could stop sending it.

Right. But, let's say we send ADD_ADDR at once in 3 mins, is it such
serious waste of resources?  If we don't care if the peer uses the address
or not, we might not need to send it so frequently and the logic can be
simpler. Also, if the peer uses the address in 3 mins, all we need to send
is one segment.

>> I personally think this is a trade-off point. We are trying to introduce
>> some complexities in the protocol to send TCP options reliably.  I might
>> want to see some benefits to compensate it.
>>> I think I’m sold on this idea now, given the additional recent comments.
>>> I support this.
>>> 3: "the receiving host MUST return the exact option.." Although it is
>>> MUST, can we always do this? There might be a situation where there is no
>>> option space or no segment to send.
>>> I would assume the same rules as ADD_ADDR are followed, so it’s sent on
>>> a duplicate ACK in that case.
>> Hmm. If we apply the same rules to this, it may need to send a dupack for
>> a dupack and may be repeated N times. I still need to think about it, but
>> it doesn't look pretty..
> OK, we wouldn't need to retransmit the echo, unless a new ADD_ADDR comes
> in. You raise a good point though about rate limiting however, an
> implementation couldn't just keep on retransmitting for ever, and needs to
> assume the options are being lost and stop sending them.

I think we can prepare some rules here to avoid potential issues and corner
if we can make them simple enough, then we are good. But, I'm just not very
sure yet.