Re: [multipathtcp] MPTCP Schedulers

Olivier Bonaventure <> Sun, 24 March 2019 17:41 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 34FEE12000E for <>; Sun, 24 Mar 2019 10:41:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id BnjvFsytBJb2 for <>; Sun, 24 Mar 2019 10:41:43 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 39723120005 for <>; Sun, 24 Mar 2019 10:41:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oBRzECe8AhZ1LwOpeX+S/Lbj9MMCWpCDT/DQai8dBTQ=; b=RMY/+AWDaukcr2Jp3kb8ugBBRu12QFOZ1KK7itqoZJzzgCxZ3wJVwhtOrXKPXlD4YFPmOb3tzwwRTNZEKFnGPptEIzMoHW3dB2OiFwHjMrgSR9CpZlN6056tNYX611m2ZBe/cyMpbYAbVdpx5vR6mFbfwq528fvE9+O1E30Zwsw=
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.16; Sun, 24 Mar 2019 17:41:39 +0000
Received: from ([fe80::f464:9089:cad4:1a01]) by ([fe80::f464:9089:cad4:1a01%2]) with mapi id 15.20.1730.019; Sun, 24 Mar 2019 17:41:39 +0000
From: Olivier Bonaventure <>
To: Zhen Cao <>
CC: "" <>, "" <>, "" <>, "" <>, "" <>
Thread-Topic: [multipathtcp] MPTCP Schedulers
Date: Sun, 24 Mar 2019 17:41:39 +0000
Message-ID: <>
References: <> <20190215180722.GR1880@MacBook-Pro-19.local> <> <> <LO2P123MB17926F324140C813F236CA48EB630@LO2P123MB1792.GBRP123.PROD.OUTLOOK.COM> <> <>
In-Reply-To: <>
Reply-To: Olivier Bonaventure <>
Accept-Language: en-US
Content-Language: en-US
x-clientproxiedby: (2603:10a6:803:50::18) To (2603:10a6:10:e0::20)
authentication-results: spf=none (sender IP is );
x-ms-exchange-messagesentrepresentingtype: 1
x-originating-ip: [2001:67c:1232:144:4177:a315:efa2:c031]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4fd0c2dd-3031-491c-625d-08d6b07ff843
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020); SRVR:DB8PR03MB5673;
x-ms-traffictypediagnostic: DB8PR03MB5673:
x-microsoft-antispam-prvs: <>
x-forefront-prvs: 09860C2161
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(366004)(376002)(346002)(396003)(39850400004)(136003)(55674003)(199004)(189003)(2906002)(74482002)(6512007)(11346002)(14454004)(76176011)(52116002)(8676002)(6916009)(6486002)(229853002)(478600001)(6246003)(81156014)(186003)(81166006)(97736004)(6116002)(53936002)(3450700001)(99286004)(386003)(6506007)(36756003)(786003)(316002)(31686004)(43066004)(46003)(8936002)(54906003)(25786009)(4326008)(31696002)(305945005)(71200400001)(71190400001)(105586002)(106356001)(7736002)(102836004)(86362001)(93886005)(446003)(486006)(6436002)(256004)(14444005)(68736007)(2616005)(476003)(5660300002); DIR:OUT; SFP:1102; SCL:1; SRVR:DB8PR03MB5673;; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1;
received-spf: None ( does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: a4rnSWzPwc82gLH6vOrDib0uEUtQV7U1r94j07zjaCOcEitD3i+1ENkZNRZ7n9V+ciO0E332VLt3SqUpi3GlpvSYiqtc9zgCdXIvyJnAvRbA7wYke5IEKLCWPxOuCoaEhPEGJjxfsDa42welMWqhDeEzaJsJuhbYcQfg3egYwvK4ZFQ3fBSZHFqa7eg4gvSOm5Z+KbjHiowvPU0zOx554SHZIFwjxMIbyiW0/m4udYGSq55RVN0pCgpDpeqhEGYzgvfjH8piLV1lzzA6xMABuelkooZXPuQH/LlJbi6RXRFG+S+1V3UdJKHpeNo0fHF9oMtsW2xDbczt04Pbfu1BRMuUBwiwpGxzics4bE1L7a4UlrOzXUmgJjh/GT/++cFrS5TDYEcAjxTFziGqT30f6UkB5m91Fygj3uGh05OFLQE=
Content-Type: text/plain; charset="utf-8"
Content-ID: <>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 4fd0c2dd-3031-491c-625d-08d6b07ff843
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Mar 2019 17:41:39.8419 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 7ab090d4-fa2e-4ecf-bc7c-4127b4d582ec
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR03MB5673
Archived-At: <>
Subject: Re: [multipathtcp] MPTCP Schedulers
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Multi-path extensions for TCP <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 24 Mar 2019 17:41:46 -0000

>> Yes. In the hybrid access networks use case, the schedulers are
>> optimised to first use the DSL network and then overlfow over the LTE
>> network when the DSL is full.
> We also have a similar case where the server would like to saturate
> WLAN and then overflow to LTE, where another scheduler will be needed.
> But  we at least have two design choices here:
> 1) first one, as you said, the fine-grained remote control of the
> schedulers used on the peer.

This is a possibility, my fear is that it will be difficult for the IETF 
to clearly specify a reference scheduler. Maybe we can think about 
simple models to represent packets schedulers and then instantiate them.

You also need to take into account the path manager, i.e. who creates 
subflows and when. Most deployments today assume that only clients 
create subflows, mainly because of NATs, but in pure IPv6, we might be 
able to have server triggered subflows in some cases.

> 2) extension of the spare space before the Backup bit in
> MP_JOIN/MP_PRIO.  That's, keep the current Backup bit as it is( use
> the backup flow only in the event failure).  And in the meantime use
> the spare bits as the ‘Saturate and Overflow(S&O)’ priority bit; The
> server who receives sub flows marked with S&O, will first saturate the
> prioritized subflow and then overflow to the others.

We probably need more than a single bit to represent the importance of a 
flow. Note that this importance can change over time and thus should be 
exchanged by using an MPTCP option. The viewpoint of the server is not 
necessarily the same as the one of the client.

> I am in favor of the second one if the cost based prioritizing is only
> case we want to solve, and the priority can be expressed as binary.
> This does not need new options space for MPTCP anyway.

I'd suggest to briefly list all the requirements that operators have in 
controlling MPTCP schedulers. There are many deployments for very 
different use cases and we should leverage this