Re: [multipathtcp] Regarding rate control at a subflow level

<> Thu, 20 June 2019 15:14 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 644DD1200DE for <>; Thu, 20 Jun 2019 08:14:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id sHLeb6Hjax7a for <>; Thu, 20 Jun 2019 08:14:30 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 7FF3D1200A3 for <>; Thu, 20 Jun 2019 08:14:29 -0700 (PDT)
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1713.5; Thu, 20 Jun 2019 16:14:26 +0100
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1395.4; Thu, 20 Jun 2019 16:14:26 +0100
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Frontend Transport; Thu, 20 Jun 2019 16:14:26 +0100
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1713.5; Thu, 20 Jun 2019 16:14:25 +0100
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+1iGFH/vGxXdL0dCmpjUsL06Phy++SVjSRyxH0TFS+o=; b=TSqseEr0Q/hfOzWVFqsjjmev5DF3NyNHnNREQid9KyNCmYUX3dck7D5HKyDkyDTiZNZGn8z2bCVXabBk0NFKhOiYWqyzTBJ5E64+8MbJZV8BmJS5sTwbZaVP4dg6673s3Dp1KQbge21SUEZCDd/Og9lF4v9wUO/Hwo/T+m/1cUA=
Received: from LNXP123MB2587.GBRP123.PROD.OUTLOOK.COM ( by LNXP123MB2233.GBRP123.PROD.OUTLOOK.COM ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1987.11; Thu, 20 Jun 2019 15:14:25 +0000
Received: from LNXP123MB2587.GBRP123.PROD.OUTLOOK.COM ([fe80::d0d4:e85d:d101:97f0]) by LNXP123MB2587.GBRP123.PROD.OUTLOOK.COM ([fe80::d0d4:e85d:d101:97f0%7]) with mapi id 15.20.1987.014; Thu, 20 Jun 2019 15:14:25 +0000
Thread-Topic: [multipathtcp] Regarding rate control at a subflow level
Date: Thu, 20 Jun 2019 15:14:24 +0000
Message-ID: <LNXP123MB258799012B1C3F610A4D3FB3EBE40@LNXP123MB2587.GBRP123.PROD.OUTLOOK.COM>
References: <> <> <20190520135014.GG41806@MacBook-Pro-64.local> <LO2P123MB1965EA246F2652E9D5C47731EB180@LO2P123MB1965.GBRP123.PROD.OUTLOOK.COM> <> <LO2P123MB1965E1FF05CB40AC39F8B36AEB140@LO2P123MB1965.GBRP123.PROD.OUTLOOK.COM> <> <LNXP123MB2587FB78BF5AC2BB256E4A66EBEA0@LNXP123MB2587.GBRP123.PROD.OUTLOOK.COM> <>
In-Reply-To: <>
Accept-Language: en-GB, en-US
Content-Language: en-US
authentication-results: spf=none (sender IP is );
x-originating-ip: []
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4c58efbd-57f8-4181-707b-08d6f591fb38
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:LNXP123MB2233;
x-ms-traffictypediagnostic: LNXP123MB2233:
x-ms-exchange-purlcount: 1
x-antispam-2: 1
x-microsoft-antispam-prvs: <LNXP123MB223305D62870BF792C6C6583EBE40@LNXP123MB2233.GBRP123.PROD.OUTLOOK.COM>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-forefront-prvs: 0074BBE012
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(376002)(39860400002)(366004)(396003)(136003)(199004)(13464003)(189003)(53936002)(478600001)(73956011)(8676002)(4326008)(966005)(68736007)(25786009)(76176011)(81156014)(71190400001)(74316002)(6116002)(53546011)(86362001)(7696005)(6506007)(102836004)(229853002)(81166006)(486006)(256004)(52536014)(14444005)(3846002)(110136005)(66066001)(33656002)(476003)(54906003)(2501003)(8936002)(71200400001)(76116006)(2906002)(14454004)(316002)(66946007)(66446008)(99286004)(55016002)(64756008)(66556008)(6436002)(6306002)(446003)(7736002)(9686003)(11346002)(305945005)(26005)(6246003)(66476007)(5660300002)(186003)(20673002); DIR:OUT; SFP:1101; SCL:1; SRVR:LNXP123MB2233; H:LNXP123MB2587.GBRP123.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1;
received-spf: None ( does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: F4KDn20dyPM698T0nbsGb2N8ibj+WD9IkbDXONB3hp6OV7RlVQeWTjHc4XVtfRSl5e/bMv9PAgUVryBwKlTXqRMn5yQjjoz1liupkuzXjQT2ruMIvBH1x2xJoiFWeU+Q0n4anYT8js3MGUVQXslnDS1pBb8tM8mb9JI3butfZY6DDNlYnlBipIICt2oFNi8BIzjkeS4WIBhrV+xZ/cZ675FL/a/E262z0A1jsFB0xl6phkrww0yCuTkJSMdGDrJzfkRQnuW1U0ShMWZkvusePAgWaNhc6NQUewx2ko8U4Sk0CjZyv6AmZQbMoXcVgWtqaJCQkQ4qjJXDLwmfY+zg/8yny1OOctlWc8WOsCH87478FshvHVy3fSApds2cEYLxvNKUDbl60FiucYizAe4N75xR5Kfejd+ZwHVIZ2WfULo=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 4c58efbd-57f8-4181-707b-08d6f591fb38
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jun 2019 15:14:25.2187 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a7f35688-9c00-4d5e-ba41-29f146377ab0
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LNXP123MB2233
X-NAI-Spam-Flag: NO
X-NAI-Spam-Threshold: 5
X-NAI-Spam-Score: 0.1
X-NAI-Spam-Report: 4 Rules triggered * 0.1 -- GEN_SPAM_FEATRE * 0 -- EDT_SDHA_ADR_FRG * 0 -- EDT_SDHA_DMN_FRG * 0 -- RV6573
X-NAI-Spam-Version: : core <6573> : inlines <7107> : streams <1825047> : uri <2858230>
Archived-At: <>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Multi-path extensions for TCP <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 20 Jun 2019 15:14:35 -0000

For upstream, isn’t the most likely case that the client side knows the available bandwidth on the multiple paths? so it can control the rate at which it sends traffic upstream on the different paths. (ie the server doesn’t need to tell the client side to throttle speed, because the client side can do this on its own)


-----Original Message-----
From: Nagesh shamnur [] 
Sent: 20 June 2019 06:28
To: Eardley,PL,Philip,TUD1 R <>;;
Cc:; Ashutosh prakash <>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	Generally for the downlink traffic, server tries to share the available bandwidth among all the available connections (flows and subflows) and this would be achieved by Congestion control algorithm. Any algorithm would try to scale up to pipe limit. If server needs to throttle the speed then it wouldn't be possible with congestion control algorithm alone and server can realize the same with many ways and one of them would be to ask application on the server side to slow down the packet sending rate.
	How will server achieve the same to tell client side to throttle speed? Dropping packets on server side implicitly or explicitly can achieve it, but IMO it is not the best way. 
Nagesh S

-----Original Message-----
From: []
Sent: 18 June 2019 15:15
To: Nagesh shamnur <>;;
Cc:; Ashutosh prakash <>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level


Thanks, interesting to hear about an upstream use case. Would be good to explore the scenario in a bit more detail. 
As you say, the congestion control algorithm can be used to stop one connection starving others (ie server / network-side drop packets to slow down a connection). I think you're hinting that this may not be good enough in all scenarios (<< but there can be cases where the application decides to limit the bandwidth to avoid possibly overload kind of scenarios.>>) - would you explain this a bit more please?
I'm trying to understand what benefit we might gain from some extra signalling (whether of preferences, a rate to limit to, or whatever) and therefore whether it's something we should think about doing. 


ps Sorry for delay, been away.
-----Original Message-----
From: Nagesh shamnur []
Sent: 03 June 2019 14:07
To: Eardley,PL,Philip,TUD1 R <>;;
Cc:; Ashutosh prakash <>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	The idea is about controlling the rate on the uplink rather than the downlink and the decision to limit the bandwidth is made at the server since, server can control the downlink traffic rate to all the clients connected to it, but the uplink limit needs to be pushed to the client. 	

Nagesh S

-----Original Message-----
From: []
Sent: 03 June 2019 15:46
To: Nagesh shamnur <>;;
Cc:; Ashutosh prakash <>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Could you explain your scenario a bit more please? Not sure I get it right.  Is this about uplink? Is the application decision to limit the bandwidth made at the client or at the server, and does the network have a role (either in the decision making or supplying info)?

-----Original Message-----
From: Nagesh shamnur []
Sent: 31 May 2019 06:19
To: Eardley,PL,Philip,TUD1 R <>;;
Cc:; Ashutosh prakash <>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

Hi Phil,
	I am glad the problem is being discussed upon.  One more possible usecase is for load balancing to achieve one or a set of clients doesn't hog the entire bandwidth available on the server side. Though the links to server supports the bandwidth but still chooses to limit the bandwidth. I agree, this is left best to be dealt with congestion control algorithm to achieve the bandwidth sharing, but there can be cases where the application decides to limit the bandwidth to avoid possibly overload kind of scenarios.

Nagesh S

-----Original Message-----
From: []
Sent: 30 May 2019 23:23
Cc:; Ashutosh prakash <>; Nagesh shamnur <>
Subject: RE: [multipathtcp] Regarding rate control at a subflow level

I think the use case a hybrid operator could be interested in is the following. Similar to Alexander's use case.

We want to provide a particular customer with broadband access that is faster than their DSL alone can provide, for instance to meet some minimum rate to meet a target of say 20Mbps. The rate can't quite be met by DSL, therefore cellular or even satellite is used as a top-up. DSL capacity is much cheaper than cellular /satellite, therefore the ideal scheduler would favour DSL. It would react reasonably quickly to increased total load above the DSL rate (but not so fast that the instantaneous rate from a variable rate source 'kicks in' the cellular when the rate over a slightly longer time can be met by DSL). Note that the DSL rate is not completely static.  It supports deployments with a proxy in the home gateway and in the network (under the control of the operator), and deployments where there's only the proxy in the network (ie MPTCP is in the multi-interface phone). Possible to ensure that an individual flow goes over only one access. Downward direction (from network to house) more important than upward.  In a long-running session using a lot of bandwidth, because the amount of other traffic varies, it may be that this session sometimes uses just the DSL and sometimes is spread over both accesses. On a minor point, ideally a speed test should measure the rate correctly (I don’t mean the scheduler identifies a speed test and favours it; rather, ideally the speed test shouldn’t eg measure just the DSL rate - alternatively re-define the speed test). 

So in terms of Olivier's question below, I think this means that the limit is about the total and not per subflow. 

In terms of mechanism, I'm open to whatever is most suitable. I'm very interested if the best method would mean that ideally some new functionality is added to the MPTCP standard (and by implication to MP-QUIC).

Best wishes,

-----Original Message-----
From: multipathtcp [] On Behalf Of Christoph Paasch
Sent: 20 May 2019 14:50
To: Olivier Bonaventure <>
Cc:; Ashutosh prakash <>; Nagesh shamnur <>
Subject: Re: [multipathtcp] Regarding rate control at a subflow level


On 17/05/19 - 09:59:54, Olivier Bonaventure wrote:
> Dear Nagesh,
> > 
> >                  Greetings. In case of Mobile deployments of MPTCP, 
> > though the data rates are getting cheaper, still it would be wise 
> > not to run the cellular path to full limit but to throttle to a 
> > certain extent considering cost in mind or if server wants to limit 
> > the client at subflow level, then I couldn’t find the support for 
> > the same in the specification.
> We have developed several prototypes that include this capability in 
> the Linux kernel.
> > So, while going through the discussion archives, could only find 
> > that, the peer(server) can throttle the speed for the entire 
> > connection by publishing a smaller receiver window rather than for a 
> > particular subflow. I feel, it would be a good idea if the peers can 
> > exchange this information using the control packets.
> We could imagine an MPTCP option that provides the maximum rate on a 
> per-subflow level, but I was wondering whether the use case is not to 
> limit the bandwidth on the smartphone at the link level (i.e. multiple 
> tcp connections or udp flows) and not at the subflow level. Could you 
> precise your use case for the subflow level ?

another use-case for rate-control I see is when a client wants to tell a sender to gracefully close a subflow.

- Sending a TCP-RST results in packet-loss of the in-flight data.
- Reducing the window is going to stall the whole connection because the window is shared.
- Setting MP_PRIO also won't work because none might want to drain a secondary
  subflow even if the "primary" subflow is having severe packet-loss.

Thus, a "maximum-rate" option on a per-subflow level would allow to send the rate to 0, which would drain the subflow.


multipathtcp mailing list