IETF Logo Mail Archive

Re: [aqm] [Bloat] TCP BBR paper is now generally available

Neal Cardwell <ncardwell@google.com> Sun, 04 December 2016 03:18 UTC

Return-Path: <ncardwell@google.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 30A4B12949B for <aqm@ietfa.amsl.com>; Sat, 3 Dec 2016 19:18:42 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.597
X-Spam-Level:
X-Spam-Status: No, score=-5.597 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-2.896, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Eiw6LmXz1T-8 for <aqm@ietfa.amsl.com>; Sat, 3 Dec 2016 19:18:40 -0800 (PST)
Received: from mail-oi0-x22d.google.com (mail-oi0-x22d.google.com [IPv6:2607:f8b0:4003:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 07847129443 for <aqm@ietf.org>; Sat, 3 Dec 2016 19:18:39 -0800 (PST)
Received: by mail-oi0-x22d.google.com with SMTP id b126so306811819oia.2 for <aqm@ietf.org>; Sat, 03 Dec 2016 19:18:39 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=pEShnqHpqjIX10ejQWGpLBgFpYVbLWCNmrmt1eGTSIo=; b=dBDuPUOJhPPwKfeRE4u8ovTT6w/RGtbHOkaxG6Rm9NSL6mARwUihxtS84ddfZhncFU CzLGz5vP6SQBz9D5/3EhydwHV8DF1pwtNmHwabLubqZQhxj+65tt6TPuT6sPNW5Somsf mGhGUL28BKGBLUoJxBfeZBjoFu2XgWneY/tQJ2nZxfR7NVYCAW+73aP+zNbZh6pokWO0 ahit5K1X609403QaO4o7auJo/O35qk+HDTAVhJXnlyYLLllE9s8ckpUtB++5PLaeThkn aROy3AaEYLLwhUBL2OlmUxr/B6FeukqGK990n3rXxux7RS+fNj++ZxX7s4CVTye5UyHm nuMw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=pEShnqHpqjIX10ejQWGpLBgFpYVbLWCNmrmt1eGTSIo=; b=AOTsWf2px36QDmmQlJ4E+VjHUoWVUB9diIJUE1Y7DoSEAtNTQQBsruwjjFoFwqDt/s Jlkw4apHtLjHZjSqEXxwGRMZdTrGfLQxpIEZ2DRsocCDRMVtdPMM/JGAOBXnrB99/py2 OyV4RCcZP+d2IrzNWsxdyi0MDvQxUzQP3m5SVLNchbT0A5ZXL2KfqDQSdKwjBQv1ycXo vTPON/Xe5S7WKVu56poe4G9/ll/0U82WrKJC7FC1VNuWbk0M4PkzdV/rE8J/VRwzVa+Y hDv2SLUPGUmJSxifMZ1eVXD4cEdc7HEHBgdlSDgzN1NIAWuLJWvZloAvtaNtjnrCr+Oi qGkQ==
X-Gm-Message-State: AKaTC01hV8o0zcfw/Hwm9NurSnDbVD87gKMXfiFUlPFlw2y/GSkR0jG5RlDLUm/goVAg8yw/nX2n3UxIKu++2Ycf
X-Received: by 10.157.53.50 with SMTP id o47mr26764529otc.19.1480821518972; Sat, 03 Dec 2016 19:18:38 -0800 (PST)
MIME-Version: 1.0
Received: by 10.202.73.195 with HTTP; Sat, 3 Dec 2016 19:18:08 -0800 (PST)
In-Reply-To: <1480807468.18162.436.camel@edumazet-glaptop3.roam.corp.google.com>
References: <1480721486.18162.392.camel@edumazet-glaptop3.roam.corp.google.com> <CADVnQym9iPJ+GR7BN9fPRe3on_j=OxUD0D83DS6Dzf1xLKvtnA@mail.gmail.com> <20161203191353.GA972@sesse.net> <1480796415.18162.408.camel@edumazet-glaptop3.roam.corp.google.com> <9A821B94-143B-446F-8852-2EB0158DD57C@gmail.com> <1480799260.18162.418.camel@edumazet-glaptop3.roam.corp.google.com> <20161203213428.GB38041@sesse.net> <1480801837.18162.419.camel@edumazet-glaptop3.roam.corp.google.com> <20161203221338.GC38041@sesse.net> <1480805737.18162.425.camel@edumazet-glaptop3.roam.corp.google.com> <20161203230335.GA40411@sesse.net> <1480806940.18162.434.camel@edumazet-glaptop3.roam.corp.google.com> <1480807468.18162.436.camel@edumazet-glaptop3.roam.corp.google.com>
From: Neal Cardwell <ncardwell@google.com>
Date: Sat, 03 Dec 2016 22:18:08 -0500
Message-ID: <CADVnQynhHoD2-3TEgTFrQPZwejRvGkKO-uND0tEegzVNpPjZOg@mail.gmail.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Content-Type: multipart/mixed; boundary="001a113d7e189da97d0542cca04f"
Archived-At: <https://mailarchive.ietf.org/arch/msg/aqm/NjsoRYqEzLGhOFNMDFHp-bKyKP0>
Cc: "Steinar H. Gunderson" <sgunderson@bigfoot.com>, bloat <bloat@lists.bufferbloat.net>, "aqm@ietf.org" <aqm@ietf.org>, Jonathan Morton <chromatix99@gmail.com>
Subject: Re: [aqm] [Bloat] TCP BBR paper is now generally available
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 04 Dec 2016 03:18:42 -0000

>    http://storage.sesse.net/bbr.pcap -- ssh+tar+gnupg

I agree with Eric that for the ssh+tar+gnupg case the ACK stream seems
like the culprit here. After about 1 second, the ACKs are suddenly
very stretched and very delayed (often more than 100ms). See the
attached screen shots.

I like Eric's theory that the ACKs might be going through fq.
Particularly since the uplink data starts having issues around the
same time as the ACKs for the downlink data.

neal

On Sat, Dec 3, 2016 at 6:24 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Sat, 2016-12-03 at 15:15 -0800, Eric Dumazet wrote:
>> On Sun, 2016-12-04 at 00:03 +0100, Steinar H. Gunderson wrote:
>> > On Sat, Dec 03, 2016 at 02:55:37PM -0800, Eric Dumazet wrote:
>> > > Note that starting from linux-4.4, e1000e gets gro_flush_timeout that
>> > > would help this precise workload
>> > >
>> > > https://git.kernel.org/cgit/linux/kernel/git/davem/net.git/commit/?id=32b3e08fff60494cd1d281a39b51583edfd2b18f
>> > >
>> > > Maybe you can redo the experiment in ~5 years when distro catches up ;)
>> >
>> > I can always find a backport, assuming the IPMI is still working. But it
>> > really wasn't like this earlier :-) Perhaps something changed on the path,
>> > unrelated to BBR.
>>
>> If the tcpdump is taken on receiver, how would you explain these gaps ?
>> Like the NIC was frozen for 40 ms  !
>
> Wait a minute. If you use fq on the receiver, then maybe your old debian
> kernel did not backport :
>
> https://git.kernel.org/cgit/linux/kernel/git/davem/net.git/commit/?id=9878196578286c5ed494778ada01da094377a686
>
>
>

v2.23.0 | Report a Bug | By Email