[tcpm] BBR and ECN/drop

"De Schepper, Koen (Nokia - BE/Antwerp)" <koen.de_schepper@nokia-bell-labs.com> Thu, 20 July 2017 23:38 UTC

Return-Path: <koen.de_schepper@nokia-bell-labs.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9AA8E12EB43 for <tcpm@ietfa.amsl.com>; Thu, 20 Jul 2017 16:38:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.922
X-Spam-Level:
X-Spam-Status: No, score=-1.922 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1vMVgltfS47v for <tcpm@ietfa.amsl.com>; Thu, 20 Jul 2017 16:38:06 -0700 (PDT)
Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0135.outbound.protection.outlook.com [104.47.0.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 43D351296C6 for <tcpm@ietf.org>; Thu, 20 Jul 2017 16:38:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector2-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=rZrYaxGOmvLVzC/vG8o+7Gr77xI9sjtm+NRWO7+XJHo=; b=Ct6eb2mhQK9ih9vFErp8b4OryeJUailgvUht/quBRjxhx6IsnL9BHb+WClGYrz7glevH5WvzhdjFIYzvWNYlrKKKH8KBCurE0aH0Q7OvOzesm7m3zuqc036vQHrRK4PUbQ16kEdp2WtVNo0kLkSeZictVV4+bQRxmYL47OHkSl4=
Received: from VI1PR0701MB2126.eurprd07.prod.outlook.com (10.169.137.7) by VI1PR0701MB2462.eurprd07.prod.outlook.com (10.168.139.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1282.4; Thu, 20 Jul 2017 23:38:03 +0000
Received: from VI1PR0701MB2126.eurprd07.prod.outlook.com ([fe80::1c36:e69e:a2b8:c323]) by VI1PR0701MB2126.eurprd07.prod.outlook.com ([fe80::1c36:e69e:a2b8:c323%16]) with mapi id 15.01.1282.011; Thu, 20 Jul 2017 23:38:03 +0000
From: "De Schepper, Koen (Nokia - BE/Antwerp)" <koen.de_schepper@nokia-bell-labs.com>
To: Neal Cardwell <ncardwell@google.com>, "Gengxuesong (Geng Xuesong)" <gengxuesong@huawei.com>
CC: "tcpm@ietf.org" <tcpm@ietf.org>
Thread-Topic: BBR and ECN/drop
Thread-Index: AdMBp6J0BjT+g6JDTRSMLkDMotwMKA==
Date: Thu, 20 Jul 2017 23:38:02 +0000
Message-ID: <VI1PR0701MB2126DDD3FF46E4F1DF038A4FB9A70@VI1PR0701MB2126.eurprd07.prod.outlook.com>
Accept-Language: nl-BE, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=koen.de_schepper@nokia-bell-labs.com;
x-originating-ip: [31.133.158.165]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; VI1PR0701MB2462; 7:ihRy0xqG6acVIeh2wNNk3TKag0TzeTdiOJt6gvIm7i18UX7eJHqXDcrkGipcJO2KiBbtFMr3AbHop760FHP1lX3GxPdan1MgOySuQjZI5W92flKRhNFv45hU9UGpS4Chkoig4ENdraOIxU93lyf4VasnRKo1Nqpi6Y4iL/4Tb/W5Yx735BlwtEb3XF22dCQ2xIQv/dvEoX4U5I4/sKg8I3daig5DXqiGy5qYqz7FNJo6Kssmv596dzD4s+ks++G7P2oq1BS2bhlNja/67cFy82gHyNnQWtS+8qNqWD8FgxJ7F9l92RcpeR4vMzfxp4UzHTrFgolqolC+EEICZFfJES6fS7/dJiPuY3KvQp0jkD35uHcjF5G9G1b5bcymafG4lJmdZcXHNKViVR9F7As0osE3mO9zIFqQPMNZzpq3AWPK0FV0eq57BBzdu5j3Yl9PynCPwTfc1E90m5+EsYq1H24W4FeKss18sn8xlynEmWERY3n1qHjYn7zF1jmlC/8JGTQrZhB68M0kI3HBv+B/gx6+JfS6cqeb408bOYAHhKFMWbuyiSK/zGmQnJnjlYRpI9BaLKi+0AX3koCb7B8LD4CEM66ZjkTSt/B9+C/U9SJZmkg9lbbXJ0ciaTgjP+ZiUnKWYrHcv3fZfCqF9SIIH+NyLF48cGVba1NvwJx/HLaS/O7hpBmUJ5v6s7xZHeic6euraEuKqXamIr22GYs6GQb/AfHH+7FNGmb8koLbj/Vck/NEgJHfCAUhIw5Gh5dmXeXMyvCeOL9ZiUBD8eEDq26y2JZxDZ02pOG6okxuzT8=
x-ms-office365-filtering-correlation-id: 5512d752-0a9d-4ff2-1d33-08d4cfc85d43
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(300000503095)(300135400095)(48565401081)(2017052603031)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:VI1PR0701MB2462;
x-ms-traffictypediagnostic: VI1PR0701MB2462:
x-exchange-antispam-report-test: UriScan:(50582790962513)(17755550239193);
x-microsoft-antispam-prvs: <VI1PR0701MB2462DC6944F8FA0D5B03CC08B9A70@VI1PR0701MB2462.eurprd07.prod.outlook.com>
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123555025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123562025)(20161123558100)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:VI1PR0701MB2462; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:VI1PR0701MB2462;
x-forefront-prvs: 0374433C81
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(39850400002)(39860400002)(39400400002)(39410400002)(39450400003)(39840400002)(199003)(13464003)(24454002)(189002)(377454003)(53546010)(7696004)(81166006)(4326008)(2900100001)(6436002)(8936002)(478600001)(5250100002)(6506006)(8676002)(305945005)(50986999)(55016002)(99286003)(105586002)(966005)(7736002)(9686003)(97736004)(81156014)(53936002)(6306002)(14454004)(102836003)(106356001)(33656002)(561944003)(38730400002)(54356999)(2906002)(25786009)(6116002)(3846002)(3660700001)(66066001)(189998001)(74316002)(3280700002)(68736007)(101416001)(5660300001)(86362001)(90052001); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0701MB2462; H:VI1PR0701MB2126.eurprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:0; MX:1; LANG:en;
received-spf: None (protection.outlook.com: nokia-bell-labs.com does not designate permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nokia-bell-labs.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jul 2017 23:38:02.9139 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0701MB2462
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/ov-bv1P8X1zaOZP9vWpiELO4-hU>
Subject: [tcpm] BBR and ECN/drop
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jul 2017 23:38:10 -0000

Hi,

I renamed the subject to start that new thread.

As BBR is not controlling its window based on a drop or mark signal, it will be difficult to define a reduction of the window, as that window is later overwritten by a value calculated from other non-related parameters. What I think could work is a cap on the max(BtlBw) rate that is used in combination with the min(RTT) to calculate the window. The BtlBw could be min-ed with a Bw derived from the average drop/mark probability over some (sliding) time-interval (order of max to support RTT, 100ms?). The exact drop_bw = f(p) function f needs to be determined and is preferably improved over how Reno and Cubic respond to a similar probability but then based on its AIMD. 

For Cubic in Reno mode: drop_bw = 1.68/sqrt(p)/RTT.

For small BDPs this will be very aggressive, and a "big" queue will usually limit the drop probability (in this case also Cubic falls back to the Reno behavior, though with a higher constant, due to its reduction of 30% only).

For high BDPs drop probability will be too low (even for Cubic) and a high drop probability due to bursty traffic on top of a standing queue or due to competing traffic with a small RTT, will reduce the throughput for the long RTT flows.

BBR uses the other extreme, by not responding to drop/ECN signal unless it persists for a long time above 20%. This has many consequences, and I think better would be to respond just more aggressive to it, with a corrected f(p) equation.

The discussion should be what a good equation could be. Preferably the same equation could be used for all use cases (both inside datacenters, DC interconnect and internet).

I presented in ICCRG a first proposal for this equation, that could be used to start the discussion whether it would cover all problem cases sufficiently, or is too aggressive:
Drop_bw = min( 1.68 / sqrt(p) , x / p ) / max( RTT , RTT_ref )
with:
x = 1.68 sqrt(p_o) = 0.168
p_o = 1%
RTT_ref = 5ms

If p < p_o the CC becomes proportional to 1/p (scalable) instead of 1/sqrt(p).
If RTT > RTT_ref, the CC becomes RTT independent.
This will also help to define TCP-Prague behavior.

Any alternatives for this equation or the cut-over parameters?

Koen.

> -----Original Message-----
> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Neal Cardwell
> Sent: donderdag 20 juli 2017 17:12
> To: Gengxuesong (Geng Xuesong) <gengxuesong@huawei.com>
> Cc: tcpm@ietf.org
> Subject: Re: [tcpm] ECN++: Congestion response to CE on Pure ACK
> 
> On Thu, Jul 20, 2017 at 2:17 AM, Gengxuesong (Geng Xuesong)
> <gengxuesong@huawei.com> wrote:
> > Hi Neal
> >
> > I think it is a very interesting discussion about the interaction between BBR
> and ECN.
> > You mentioned that " since ECN schemes typically mandate a particular
> cwnd response from a sender ". Do you mean that the conflict between ECN
> and BBR is the conflict between cwnd mode and pacing mode?
> > In other word, do you think if ECN is used as a signal of modifying pacing
> rate, these two mechanisms may work together well?
> 
> I didn't really mean the distinction between control via cwnd vs
> control via pacing rate. I meant the deeper question of whether a
> congestion control algorithm should simply have a fixed back-off
> response to losses (Reno, CUBIC) or ECN marks (RFC 3168, DCTCP), or
> whether the algorithm should try to make an independent estimate of
> the capacity available to the flow (BBR).
> 
> But if someone wants an extended discussion of BBR and ECN, I'd
> suggest that this should move to a separate thread, so that this
> thread can stay focused on the original issues Bob raised in this
> thread.
> 
> thanks,
> neal
> 
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm