Re: [multipathtcp] Multipath TCP Address advertisement 2/5 - Reliability

Alan Ford <> Sat, 06 August 2016 20:35 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id D883112D0F9 for <>; Sat, 6 Aug 2016 13:35:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 75VJOgAWkLBX for <>; Sat, 6 Aug 2016 13:35:54 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4002:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 75C4A12D0E5 for <>; Sat, 6 Aug 2016 13:35:54 -0700 (PDT)
Received: by with SMTP id z8so285535675ywa.1 for <>; Sat, 06 Aug 2016 13:35:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=yqIXcwPIGdZMR8Jr/g4Gyan/Zmq8GAmaO9AZbEgN9UU=; b=qz/pkU9tkYF5JAL7M+7ETuX2TbqnXNV3/B2VSkev+htVlWw9yW5w9HEFvysv71r7J8 QPMtSfWFtXgnclbPdMVQDRv4pqFySWY6m9UWF7+RBO/M2GIkXOeeNZ9Lp+w7GCzlzwgP GKB1yenfxHRVoQFR3YruvUaiI73STlkxkA+Mj6nvznORE1czsg6qjZCuljJznuo2+QPO f+TRiwO7juQWdhAR3/HBs3g30mPDpYWngd6Mh7P+OgCBisRLLgw7x+K35VGxjxj/5g28 zqv3qxAJumcSLgB6iXkbVNujlmq2VsvoRXyQqBztL61LuPNc8yNmf1Hx8n17j24dLBew 0TOA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=yqIXcwPIGdZMR8Jr/g4Gyan/Zmq8GAmaO9AZbEgN9UU=; b=Lmwq/uy6IeolWBn3lJMpE3jafnUSaEF6xkdzFrxSyRK4iWcsx2JrUl+Vq0o+h+0XAW 16K+UDGptzPnqPjs+uv7ZUEYF7XDG6JE7KVkBRtsFfw7eGoXgH+bvMBPceNM4uCufe1W GSB4kN7QjViBQZXMPDKjT3Rpu3SDO1rJrzNtyc6oKKZS0DM0AJD8KxDIOhJeNUyDYr2N xMkTJvpkvj6Spem/hLfRjD05U92YoCLi9eh++yERqbnQ2pPdPQDp15lGnsclqRk4oWyo y6sykwpjn2hJJPGfob1h/Y+n602R7CIUSJq42/dgPBTI15YrVxr1MX/SeNbIQBcmBvbg HDbg==
X-Gm-Message-State: AEkoousmncONo9RT5E+EdFUmjoIoex+aIz9/zYvzvYQBzCYgIfdfbCjJIWwNFh5jjrSMmJSAemzjoo4tZ4sHBQ==
X-Received: by with SMTP id k137mr62440951ywe.318.1470515753637; Sat, 06 Aug 2016 13:35:53 -0700 (PDT)
MIME-Version: 1.0
Received: by with HTTP; Sat, 6 Aug 2016 13:35:53 -0700 (PDT)
In-Reply-To: <>
References: <> <> <> <>
From: Alan Ford <>
Date: Sat, 6 Aug 2016 21:35:53 +0100
Message-ID: <>
To: Yoshifumi Nishida <>
Content-Type: multipart/alternative; boundary=94eb2c07747421de7905396d2120
Archived-At: <>
Cc: "" <>
Subject: Re: [multipathtcp] Multipath TCP Address advertisement 2/5 - Reliability
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Multi-path extensions for TCP <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 06 Aug 2016 20:35:57 -0000

Hi Yoshi,


On Friday, 5 August 2016, Yoshifumi Nishida <> wrote:

> Hi Alan,
> On Thu, Aug 4, 2016 at 12:57 AM, Alan Ford <
> <javascript:_e(%7B%7D,'cvml','');>> wrote:
>> Hi Yoshi,
>> Inline…
>> On 4 Aug 2016, at 07:36, Yoshifumi Nishida <
>> <javascript:_e(%7B%7D,'cvml','');>> wrote:
>> Hi Fabien,
>> On Wed, Aug 3, 2016 at 8:47 AM, Fabien Duchêne <fabien.duchene@uclouv
>> <javascript:_e(%7B%7D,'cvml','');>>
>> wrote:
>>> Hello,
>>> As agreed in Berlin during IETF96, I'm sending a series of emails to
>>> discuss the different contributions proposed
>>> in
>>> This is the part 2/5 : reliability.
>>> In RFC6824, ADD_ADDR options can be attached to segments carrying data
>>> or pure acknowledgements.
>>> In practice, notably given the length of ADD_ADDR with IPv6 addresses
>>> and the HMAC, it is very likely that they will be often sent as pure
>>> acknowledgements.
>>> This implies that ADD_ADDR are sent unreliably, which could be
>>> problematic when the ADD_ADDR is required to allow the establishment of
>>> additional subflows, as in load balancing scenarios.
>>> We propose to rely on the "E" (Echo) flag in the ADD_ADDR option.
>>> This echo flag is used to acknolwedge a received ADD_ADDR by echoing it.
>>> If the acknowledgement is not received, the ADD_ADDR option will be
>>> retransmitted up to N times.
>> I have several comments on this. Please let me know if I miss something.
>> 1: Do we really need ADD_ADDR reliability in all cases? An end node might
>> want to send ADD_ADDR for 'just in case' rather than for "I want you to use
>> this"
>> BTW, I'm not sure if the draft tries to replace the current ADD_ADDR or
>> to propose an additional ADD_ADDR option.
>> The argument is that by making it reliable, you only need to send it once
>> to know it’s been received, even if the far end does not use it.
> 2: The sender cannot be sure whether the info in ADD_ADDR will be used or
>> not. It depends on the peer's decision.
>> I might prefer re-transmitting ADD_ADDR up to N times when the sender
>> doesn't receive MP_JOIN to the address for a certain amount of time.
>> As above, by making it reliable you only need to send it once to know
>> it’s been received, even if the far end chooses not to use it.
> Hmm. if a sender wants the peer to use specific address (such as a load
> balancer case), I can understand that we want to make it reliable. But, if
> the sender doesn't care, I'm not very sure the reason to make it reliable
> or send ADD_ADDR frequently.

If local policy means the subflow won't be immediately established, there's
no way of knowing if it's been received.

Similarly, if the subflow can't be created due to e.g. firewall/network
issues, it would be useful to know if the ADD_ADDR had nevertheless been
received, so you could stop sending it.

> I personally think this is a trade-off point. We are trying to introduce
> some complexities in the protocol to send TCP options reliably.  I might
> want to see some benefits to compensate it.
>> I think I’m sold on this idea now, given the additional recent comments.
>> I support this.
>> 3: "the receiving host MUST return the exact option.." Although it is
>> MUST, can we always do this? There might be a situation where there is no
>> option space or no segment to send.
>> I would assume the same rules as ADD_ADDR are followed, so it’s sent on a
>> duplicate ACK in that case.
> Hmm. If we apply the same rules to this, it may need to send a dupack for
> a dupack and may be repeated N times. I still need to think about it, but
> it doesn't look pretty..

OK, we wouldn't need to retransmit the echo, unless a new ADD_ADDR comes
in. You raise a good point though about rate limiting however, an
implementation couldn't just keep on retransmitting for ever, and needs to
assume the options are being lost and stop sending them.