IETF Logo Mail Archive

Re: [aqm] [Bloat] TCP BBR paper is now generally available

Eric Dumazet <eric.dumazet@gmail.com> Sat, 03 December 2016 20:20 UTC

Return-Path: <eric.dumazet@gmail.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D786D1299C2 for <aqm@ietfa.amsl.com>; Sat, 3 Dec 2016 12:20:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level:
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CRLnxsYIKnvI for <aqm@ietfa.amsl.com>; Sat, 3 Dec 2016 12:20:18 -0800 (PST)
Received: from mail-pg0-x244.google.com (mail-pg0-x244.google.com [IPv6:2607:f8b0:400e:c05::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 050521289C4 for <aqm@ietf.org>; Sat, 3 Dec 2016 12:20:18 -0800 (PST)
Received: by mail-pg0-x244.google.com with SMTP id x23so11577815pgx.3 for <aqm@ietf.org>; Sat, 03 Dec 2016 12:20:18 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=c6QfGFOVBHCo3e4RZuSI5c7kpoz0imOeEi12I8mnYBU=; b=P83mRIVLm3XdG1uZ6z8X4aTmUN3idnWD3ZCdiEzCMXESLvObXGBIZ97rhAsk2FHcK9 LdzAWtvRYc8Oqm7ehOljvz9LR9wqSwBEyJAy7dPgUgnApWfe+O5kD5BV5xS6+d7Ue0q6 uC9jTeERO5w5PZTDTwdDRdpBDgz4OPCqhJzcxV+mJvAUfYBwg48rLV9EsXaIZj12XQZ4 F2kNvOHSWYvSPaDfosBcrWg9lTk7KR9BKPabKfwcHD4qxCMG6dmFEKTM1CNozdMsmyj7 C8j+yzdMomTFcJa1FSZu9+TeAhaeznFz01qllee5yyFU/kmuj7SZzIz+oVFKMDdTnAX+ jq6Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=c6QfGFOVBHCo3e4RZuSI5c7kpoz0imOeEi12I8mnYBU=; b=U+gDtHVCvXBwGHXPzBF8qVHMlTb62Hgv8xx4Xh8IFL8jbW9Rx/gJzKD5DkGDDk1IWS AMMJ/349mMyoF/kS/1Sln7tI/gTLB4nBlA+01XZao7ZOIpc0LMWcrLKJfA50fnoHZ5FV 2MseZzMoz7dOX7m5Joi+6lNeoL9zOLAF2gCXCdFFww+N/CZN35lBrMKY69kbQci4VByN iGGnRnkykxTI1C9fGyWt5ZcjUrXxbVnfmWZH+u4TD6V79jfJvPQdVJ+oojaa1Bw/BsIy 3dItJw94g+hI7s2mRGpMbKRVM9qjDo6gOGLMiMHFQOBJQWdUk6d9jc4heS+ou1cisDEy mbNg==
X-Gm-Message-State: AKaTC03LsQG6i70exSmKgnvlYosKs9bBlJn3C4D3TVRLE/1HpPsWThk2wyKYICBH3yoZKg==
X-Received: by 10.84.129.99 with SMTP id 90mr111575799plb.154.1480796417623; Sat, 03 Dec 2016 12:20:17 -0800 (PST)
Received: from [172.19.250.57] ([172.19.250.57]) by smtp.googlemail.com with ESMTPSA id y6sm16806763pge.16.2016.12.03.12.20.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 03 Dec 2016 12:20:16 -0800 (PST)
Message-ID: <1480796415.18162.408.camel@edumazet-glaptop3.roam.corp.google.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
To: "Steinar H. Gunderson" <sgunderson@bigfoot.com>
Date: Sat, 03 Dec 2016 12:20:15 -0800
In-Reply-To: <20161203191353.GA972@sesse.net>
References: <CAA93jw7DfMY4qHnbxYDUN8hfpgY_aNxa1LcyPKd6pa93qXe2Kw@mail.gmail.com> <CALQXh-Pr+RNux5w6phqaw4kKifbB2j38JWBjCVBEog1GCYBafw@mail.gmail.com> <56F6A3AB-3A47-4178-BEFF-04E3DC23B039@gmail.com> <CADVnQymCmQ_MWSRcd+Y4=pgf3Shqnw5SfXrAkjonj+UFqtBrdA@mail.gmail.com> <20161202224006.GA5065@sesse.net> <1480721486.18162.392.camel@edumazet-glaptop3.roam.corp.google.com> <CADVnQym9iPJ+GR7BN9fPRe3on_j=OxUD0D83DS6Dzf1xLKvtnA@mail.gmail.com> <20161203191353.GA972@sesse.net>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.10.4-0ubuntu2
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/aqm/DgEVP9C2CxNqEpFQ-Ijf8k6k9FE>
Cc: Jonathan Morton <chromatix99@gmail.com>, Neal Cardwell <ncardwell@google.com>, "aqm@ietf.org" <aqm@ietf.org>, bloat <bloat@lists.bufferbloat.net>
Subject: Re: [aqm] [Bloat] TCP BBR paper is now generally available
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 03 Dec 2016 20:20:21 -0000

On Sat, 2016-12-03 at 20:13 +0100, Steinar H. Gunderson wrote:
> On Sat, Dec 03, 2016 at 08:03:50AM -0500, Neal Cardwell wrote:
> > Thanks for the report, Steinar. This is the first report we've had
> > like this, but it would be interesting to find out what's going on.
> > 
> > Even if you don't have time to apply the patches Eric mentions, it
> > would be hugely useful if the next time you have a slow transfer like
> > that you could post a link to a tcpdump packet capture (headers only
> > is best, say -s 120). Ideally the trace would capture a whole
> > connection, so we can see the wscale on the SYN exchange.
> 
> I tried reproducing it now. I can't get as far down as 50 Mbit/sec,
> but it stopped around 100 Mbit/sec, still without any clear bottlenecks.
> cubic was just as bad, though.
> 
> I've taken two tcpdumps as requested; I can't reboot this server easily
> right now, unfortunately. They are:
> 
>   http://storage.sesse.net/bbr.pcap -- ssh+tar+gnupg
>   http://storage.sesse.net/bbr2.pcap -- wget between same hosts
> 
> /* Steinar */

Hi Steinar

Huge ACK decimation it seems.

11:13:28.762772 IP6 S > C: Flags [.], seq 777672:779100, ack 3432, win 179, options [nop,nop,TS val 3498278784 ecr 864447699], length 1428
11:13:28.763145 IP6 S > C: Flags [.], seq 779100:783384, ack 3432, win 179, options [nop,nop,TS val 3498278784 ecr 864447699], length 4284
11:13:28.763190 IP6 S > C: Flags [.], seq 783384:789096, ack 3432, win 179, options [nop,nop,TS val 3498278784 ecr 864447699], length 5712
11:13:28.763239 IP6 S > C: Flags [.], seq 789096:790524, ack 3432, win 179, options [nop,nop,TS val 3498278784 ecr 864447699], length 1428
11:13:28.763334 IP6 S > C: Flags [.], seq 790524:791952, ack 3432, win 179, options [nop,nop,TS val 3498278784 ecr 864447699], length 1428
11:13:28.764109 IP6 S > C: Flags [.], seq 791952:794808, ack 3432, win 179, options [nop,nop,TS val 3498278785 ecr 864447699], length 2856
11:13:28.764138 IP6 S > C: Flags [.], seq 794808:800520, ack 3432, win 179, options [nop,nop,TS val 3498278785 ecr 864447699], length 5712
11:13:28.764189 IP6 S > C: Flags [.], seq 800520:804804, ack 3432, win 179, options [nop,nop,TS val 3498278785 ecr 864447699], length 4284
11:13:28.764980 IP6 S > C: Flags [.], seq 804804:806232, ack 3432, win 179, options [nop,nop,TS val 3498278787 ecr 864447700], length 1428
11:13:28.765034 IP6 S > C: Flags [.], seq 806232:811944, ack 3432, win 179, options [nop,nop,TS val 3498278787 ecr 864447700], length 5712
11:13:28.765086 IP6 S > C: Flags [.], seq 811944:817656, ack 3432, win 179, options [nop,nop,TS val 3498278787 ecr 864447700], length 5712
11:13:28.765905 IP6 S > C: Flags [.], seq 817656:823368, ack 3432, win 179, options [nop,nop,TS val 3498278787 ecr 864447700], length 5712
11:13:28.765956 IP6 S > C: Flags [.], seq 823368:829080, ack 3432, win 179, options [nop,nop,TS val 3498278787 ecr 864447700], length 5712
11:13:28.766005 IP6 S > C: Flags [.], seq 829080:831936, ack 3432, win 179, options [nop,nop,TS val 3498278787 ecr 864447700], length 2856
11:13:28.766869 IP6 S > C: Flags [.], seq 831936:834792, ack 3460, win 179, options [nop,nop,TS val 3498278789 ecr 864447700], length 2856
11:13:28.766898 IP6 S > C: Flags [.], seq 834792:840504, ack 3460, win 179, options [nop,nop,TS val 3498278789 ecr 864447700], length 5712
11:13:28.766947 IP6 S > C: Flags [.], seq 840504:841932, ack 3460, win 179, options [nop,nop,TS val 3498278789 ecr 864447700], length 1428
11:13:28.766997 IP6 S > C: Flags [.], seq 841932:843360, ack 3460, win 179, options [nop,nop,TS val 3498278789 ecr 864447700], length 1428
11:13:28.767532 IP6 C > S: Flags [.], ack 783384, win 4106, options [nop,nop,TS val 864447710 ecr 3498278784], length 0
11:13:28.767541 IP6 C > S: Flags [.], ack 789096, win 4151, options [nop,nop,TS val 864447710 ecr 3498278784], length 0
11:13:28.767543 IP6 C > S: Flags [.], ack 791952, win 4173, options [nop,nop,TS val 864447710 ecr 3498278784], length 0
11:13:28.767544 IP6 C > S: Flags [.], ack 794808, win 4195, options [nop,nop,TS val 864447710 ecr 3498278785], length 0
11:13:28.767546 IP6 C > S: Flags [.], ack 800520, win 4240, options [nop,nop,TS val 864447710 ecr 3498278785], length 0
11:13:28.767547 IP6 C > S: Flags [.], ack 804804, win 4273, options [nop,nop,TS val 864447710 ecr 3498278785], length 0
11:13:28.767549 IP6 C > S: Flags [.], ack 811944, win 4329, options [nop,nop,TS val 864447710 ecr 3498278787], length 0
11:13:28.767550 IP6 C > S: Flags [.], ack 817656, win 4374, options [nop,nop,TS val 864447710 ecr 3498278787], length 0
11:13:28.767552 IP6 C > S: Flags [.], ack 823368, win 4418, options [nop,nop,TS val 864447711 ecr 3498278787], length 0
11:13:28.767553 IP6 C > S: Flags [.], ack 829080, win 4463, options [nop,nop,TS val 864447711 ecr 3498278787], length 0
11:13:28.767554 IP6 C > S: Flags [.], ack 831936, win 4485, options [nop,nop,TS val 864447711 ecr 3498278787], length 0
11:13:28.767556 IP6 C > S: Flags [.], ack 834792, win 4508, options [nop,nop,TS val 864447711 ecr 3498278789], length 0
11:13:28.767557 IP6 C > S: Flags [.], ack 840504, win 4552, options [nop,nop,TS val 864447711 ecr 3498278789], length 0
11:13:28.767559 IP6 C > S: Flags [.], ack 843360, win 4575, options [nop,nop,TS val 864447711 ecr 3498278789], length 0

Just to be clear,what is the kernel version at the sender ?

Thanks !

v2.23.0 | Report a Bug | By Email