Re: [multipathtcp] A question related to MPTCP control overhead

Olivier Bonaventure <> Fri, 14 April 2017 09:50 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id BF822126C22 for <>; Fri, 14 Apr 2017 02:50:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.321
X-Spam-Status: No, score=-4.321 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 8J-3lO6Tmzf5 for <>; Fri, 14 Apr 2017 02:50:50 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 79DD11289C3 for <>; Fri, 14 Apr 2017 02:50:50 -0700 (PDT)
Received: from [] (unknown []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: by (Postfix) with ESMTPSA id 29CF567DFD6; Fri, 14 Apr 2017 11:50:40 +0200 (CEST)
DKIM-Filter: OpenDKIM Filter v2.9.2 29CF567DFD6
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;; s=selucl; t=1492163440; bh=oFkTE3D4qGOjQ+EnWdi9aZwdoWMEPF8OG7EquuzWSvA=; h=Reply-To:Subject:References:To:Cc:From:Date:In-Reply-To; b=Z0038AWKXlJP+wrzZo/8A3kzTyh87D1JVhPL2Zmf2yS/OVbyevKJkWnVvb3hrt13e 3NeHF0r37ETweIrS7vxGclNfUCGCei1h4lKtyfwemF/DGVA04b25y7RhkfmzGm1Hp9 UO5xhkDFF8+mO2/m5bka5EpQ2I9bXzH97rNMpP/w=
X-Virus-Status: Clean
X-Virus-Scanned: clamav-milter 0.99.2 at smtp-2
References: <5C068B455EB58047BBD492DA2B0829FA3763ADD5@blreml501-mbs> <> <5C068B455EB58047BBD492DA2B0829FA3763CB68@blreml501-mbs> <> <5C068B455EB58047BBD492DA2B0829FA3763D266@blreml501-mbs> <> <5C068B455EB58047BBD492DA2B0829FA3763D85B@blreml501-mbs> <> <5C068B455EB58047BBD492DA2B0829FA3763DA5D@blreml501-mbs>
To: Sayee Kompalli Chakravartula <>, "Sargent, Matthew T. (GRC-LCA0)[Peerless Technologies]" <>
Cc: "" <>
From: Olivier Bonaventure <>
Message-ID: <>
Date: Fri, 14 Apr 2017 11:50:44 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <5C068B455EB58047BBD492DA2B0829FA3763DA5D@blreml501-mbs>
Content-Type: text/plain; charset="windows-1252"; format="flowed"
Content-Transfer-Encoding: base64
X-Sgsi-Spamcheck: SASL authenticated,
X-SGSI-MailScanner-ID: 29CF567DFD6.A4A90
X-SGSI-MailScanner: Found to be clean
X-SGSI-Spam-Status: No
Archived-At: <>
Subject: Re: [multipathtcp] A question related to MPTCP control overhead
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Multi-path extensions for TCP <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 14 Apr 2017 09:50:53 -0000


>> If the connection is closed using the FIN bit: Let us represent a SN as x_{i,j} where the subscript i denotes the type of sequence space (SSN or DSN) and the subscript j gives the subflow ID. Let x_{DSN, SF2} be the DSN of the last byte transmitted before FIN is sent on SF2 and that x_{DSN, SF1} be the last DSN mapped to SF1 before the DSS Option is disabled by the sender. In this context we need to consider two possibilities: x_{DSN, SF1} < x_{DSN, SF2} and x_{DSN, SF1} >= x_{DSN, SF2}. Now consider the case x_{DSN, SF1} < x_{DSN, SF2}. When a data byte is received on SF1 that is not covered by DSN we know where to place that data byte with respect to the data byte whose DSN is x_{DSN, SF1} using subflow sequence number of the data byte. Because, eventually, data bytes need to be ordered based on DSN before releasing to the application layer we still need to decide where to place this data byte at the connection-level with respect to x_{DSN, SF2}, and this is where the protocol will run into ambiguity: whether to place the data byte before or after x_{DSN, SF2}. This will lead to protocol dead-lock. Next, we consider the possibility x_{DSN, SF1} >= x_{DSN, SF2}. Here we do not have the ambiguity present in the previous context. The received data byte on SF1 will have to be placed to the right of both x_{DSN, SF1} as well as x_{DSN, SF2} and the actual placement on SF1 will be determined by the SSN of the data byte.
>> The above analysis helps us to decide how and when DSS Option can be disabled. After sending the FIN on SF2, continue sending the DSS option on SF1 until the largest DSN used on SF1 is at least as large as x_{DSN, SF2} and the state variable NUM_SUBFLOW has transitioned to 1, and then disable the DSS Option. To describe the receiver behaviour, the receiver expects to continue to receive data bytes covered with DS mapping on SF1 until the largest DSN mapped to SF1 is at least as large as x_{DSN, SF2}. After then, starting with the first received byte that is not covered with DSN, the receiver will understand that the sender has disabled DS mapping on SF1.
>> At this point, I like to believe that defining a new Option Subtype or utilizing an unused bit in the DSS option will be fruitful.
>> Except in some specific scenarios, like a data center, I don't think a device can know in advance if second interface becomes available during the connection. So if the users goes with TCP then he is at disadvantage because he cannot utilize additional interfaces when they become available. If he opens MPTCP connection hoping that new interfaces may become available in the future, until then the connection has to incur control overhead due to redundant DSS Option.
>> As a first-cut solution I think we should allow a MPTCP connection not to use the DSS option until it opens the second subflow. To keep things simple, we may say that once DSS Option is enabled it will be enforced until the MPTCP connection is closed.
> On smartphones a solution would need to support break-before-make to be useful. This means that we cannot terminate the subflow with a FIN before deciding to switch to the utilisation of DSS. This fact, coupled with middleboxes that can transparently add/remove data from the bytestream make the problem difficult to solve
> <<Sayee>> Can you please elaborate on this scenario, as I do not seems to get what you were implying?

A very annoying corner case is the following one :

A smartphone creates an MPTCP connection over WiFi. The 
three-way-handshake is done correctly. After a few seconds, the 
smartphone sends some data but it has moved out of reach of the WiFi 
network and it does not receive acknowledgements for this data. From the 
information exchanged over the WiFi, the smartphone cannot know whether 
the data has been received by the server or not.

The smartphone then opens a subflow over the LTE interface. Once the 
subflow is established, it can be used to send data with the DSS option. 
Note that during the three-way handshake for the initial subflow, the 
client and the server have agreed on the initial sequence numbers for 
the two directions of the transfer. This means that when the first data 
arrives on the LTE subflow with a DSS option, the server can determine 
whether this is the first data of the bytestream or other data and then 
it would need to wait for the first data before delivering it to its 

If you remove the DSS on the initial subflow, then I do not see any way 
for the server to cope correctly with this scenario. Note that sequence 
number randomisation affects the sequence numbers that the smartphones 
and the server see.