Re: [aqm] [Bloat] TCP BBR paper is now generally available

Eric Dumazet <eric.dumazet@gmail.com> Fri, 02 December 2016 23:32 UTC

Return-Path: <eric.dumazet@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC5CC129478 for <aqm@ietfa.amsl.com>; Fri, 2 Dec 2016 15:32:30 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.7
X-Spam-Level:
X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I4K14S7dZI1m for <aqm@ietfa.amsl.com>; Fri, 2 Dec 2016 15:32:28 -0800 (PST)
Received: from mail-pf0-x243.google.com (mail-pf0-x243.google.com [IPv6:2607:f8b0:400e:c00::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 09E8012945F for <aqm@ietf.org>; Fri, 2 Dec 2016 15:32:27 -0800 (PST)
Received: by mail-pf0-x243.google.com with SMTP id 144so14193648pfv.0 for <aqm@ietf.org>; Fri, 02 Dec 2016 15:32:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=6PdI3E9zpyhugi7nmQXDwciegDKofKFicESl7yyogU0=; b=rg5AROICO3f7rM9cizARv9MN0xp/KQS6A+WdCKZkTjH5ieaaeDnqzMsAOUSy8kP/CL 2l+f6l6C0VsdIxeCsgWnJuPau9JgzBxxO7gMFehA4St4elDppCvsKzZaXh954m1UZ0LB GYP+tM2kuIr9f+KIFfY45+xagu5iU0AsBe7mJB0m2+tRmUHrBogLoTjQ7HWsiom8Q4m/ INV8lYVSBUwJJgqG3iu9IOrjN2rpZlGFatoMmrg4/1K3LQXS2UxxgVeUN4wbrjeKcNqI pUMjmZ+CPRG9lGrx0GdDrv9/N7BThw8z6iSJCzJgfl0GFxjRVSllD1tEWqNRMgOZoOT+ QUUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=6PdI3E9zpyhugi7nmQXDwciegDKofKFicESl7yyogU0=; b=euhUUiPsIMvB8YaasSmgTfEK9bbsnwqN0Acpd1UZzClyVHD6Juwbo5GYWSUrS5iSxl cjCAjxlV27XW6eu6w/79iLOA+nZ0uhCpxVxKdhFXnnQIJ92wu0aR4QryijqBHFaD5dFT KUlZ4H4Zu+0sQq25IH2TispyvME5dO5W/BRdbckfOClFWtCcN9yYvPPv11ldCQyz/qfq saRaJkcKHa06JmkZcgW3VjnPGGKhXhZFk8IBrLaj+j8jlclYkyEJACXz7YXw019juL4r 9eYP22t4kFtMNyfVmymjtvzUY+BNKEFYlaaQ37jpYlOep3759yDMMCE9tZvN5oQjC45w vRdA==
X-Gm-Message-State: AKaTC024EShOYZIZIvVrXvypD23WoxL22vP8hQsndo/HIXgEiJb6ysBOlj9Ow9H/Yd7C8A==
X-Received: by 10.84.218.136 with SMTP id r8mr101667566pli.23.1480721547450; Fri, 02 Dec 2016 15:32:27 -0800 (PST)
Received: from [172.19.22.200] ([172.19.22.200]) by smtp.googlemail.com with ESMTPSA id q27sm10120839pfd.49.2016.12.02.15.32.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Dec 2016 15:32:27 -0800 (PST)
Message-ID: <1480721486.18162.392.camel@edumazet-glaptop3.roam.corp.google.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
To: "Steinar H. Gunderson" <sgunderson@bigfoot.com>
Date: Fri, 02 Dec 2016 15:31:26 -0800
In-Reply-To: <20161202224006.GA5065@sesse.net>
References: <CAA93jw7DfMY4qHnbxYDUN8hfpgY_aNxa1LcyPKd6pa93qXe2Kw@mail.gmail.com> <CALQXh-Pr+RNux5w6phqaw4kKifbB2j38JWBjCVBEog1GCYBafw@mail.gmail.com> <56F6A3AB-3A47-4178-BEFF-04E3DC23B039@gmail.com> <CADVnQymCmQ_MWSRcd+Y4=pgf3Shqnw5SfXrAkjonj+UFqtBrdA@mail.gmail.com> <20161202224006.GA5065@sesse.net>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.10.4-0ubuntu2
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/aqm/WvarQcovn8DhdlIsEYhVPYYZuqE>
Cc: Jonathan Morton <chromatix99@gmail.com>, Neal Cardwell <ncardwell@google.com>, "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>
Subject: Re: [aqm] [Bloat] TCP BBR paper is now generally available
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Dec 2016 23:32:31 -0000

On Fri, 2016-12-02 at 23:40 +0100, Steinar H. Gunderson wrote:
> On Fri, Dec 02, 2016 at 05:22:23PM -0500, Neal Cardwell wrote:
> > Of course, if we find important use cases that don't work with BBR, we will
> > see what we can do to make BBR work well with them.
> 
> I have one thing that I _wonder_ if could be BBR's fault: I run backup over
> SSH. (That would be tar + gzip + ssh.) The first full backup after I rolled
> out BBR on the server (the one sending the data) suddenly was very slow
> (~50 Mbit/sec); there was plenty of free I/O, and neither tar nor gzip
> (well, pigz) used a full core. My only remaining explanation would be that
> somehow, BBR didn't deal well with the irregular stream of data coming from
> tar. (A wget between the same machines at the same time gave 6-700 Mbit/sec.)
> 
> I will not really blame BBR here, since I didn't take a tcpdump or have time
> to otherwise debug properly (short of eliminating the other things I already
> mentioned); most likely, it's something else. But if you've ever heard of
> others with similar issues, consider this a second report. :-)
> 
> /* Steinar */

It would be interesting to get the chrono stats for the TCP flow, with
an updated ss/iproute2 command and the kernel patches :

efd90174167530c67a54273fd5d8369c87f9bd32 tcp: export sender limits chronographs to TCP_INFO
b0f71bd3e190df827d25d7f19bf09037567f14b7 tcp: instrument how long TCP is limited by insufficient send buffer
5615f88614a47d2b802e1d14d31b623696109276 tcp: instrument how long TCP is limited by receive window
0f87230d1a6c253681550c6064715d06a32be73d tcp: instrument how long TCP is busy sending
05b055e89121394058c75dc354e9a46e1e765579 tcp: instrument tcp sender limits chronographs