Re: [tcpm] Linux doesn’t implement RFC3465

Vidhi Goel <vidhi_goel@apple.com> Tue, 12 November 2019 21:45 UTC

Return-Path: <vidhi_goel@apple.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E46A312006D for <tcpm@ietfa.amsl.com>; Tue, 12 Nov 2019 13:45:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level:
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=apple.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id myl_xHcZABRj for <tcpm@ietfa.amsl.com>; Tue, 12 Nov 2019 13:45:01 -0800 (PST)
Received: from ma1-aaemail-dr-lapp01.apple.com (ma1-aaemail-dr-lapp01.apple.com [17.171.2.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DE5D2120127 for <tcpm@ietf.org>; Tue, 12 Nov 2019 13:45:00 -0800 (PST)
Received: from pps.filterd (ma1-aaemail-dr-lapp01.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp01.apple.com (8.16.0.27/8.16.0.27) with SMTP id xACLbDJP008681; Tue, 12 Nov 2019 13:44:56 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=sender : from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=VHf+gTiXf1ew19J5OA3Xh/naRVY+SpiI3LtRItEVSDY=; b=j1F9h1cX2Z2lDHDwYjMvibq1XRDLv9Zd2zyMejazUo93grbuzJ8LS+fGHmkzz1cg1/pW 4PKVVHqvXJuP9MqVqPl51N956lBMc4YIFp5P7EutQVuLgGrblfboiIw3Xn+3Q72+tDBh 4JEC0Kmiz3Y8/VKIhlekoLUATuu6E6aBTOICbS6w6Bw7QQ6UfmSl83MiqCWXeOhGX5YP Atq/0qpRyTuy8/ZhS23f4oNI3wik9MQSzOBkbKTcjLRyGGGQ3ch4kthRz6+g2xliIL+j UVS6pRGpl2Yfb5f5J+8Duc/Jxhc77S8x2nHP+j6iKHBiY2Ab+8JFPQvTLxouiL0NH0uR Uw==
Received: from mr2-mtap-s01.rno.apple.com (mr2-mtap-s01.rno.apple.com [17.179.226.133]) by ma1-aaemail-dr-lapp01.apple.com with ESMTP id 2w6eexhmcv-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 12 Nov 2019 13:44:56 -0800
Received: from nwk-mmpp-sz12.apple.com (nwk-mmpp-sz12.apple.com [17.128.115.204]) by mr2-mtap-s01.rno.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPS id <0Q0V001XRLQVS7D0@mr2-mtap-s01.rno.apple.com>; Tue, 12 Nov 2019 13:44:55 -0800 (PST)
Received: from process_milters-daemon.nwk-mmpp-sz12.apple.com by nwk-mmpp-sz12.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) id <0Q0V00600LEEG100@nwk-mmpp-sz12.apple.com>; Tue, 12 Nov 2019 13:44:55 -0800 (PST)
X-V-A:
X-V-T-CD: 87af957b8464877f93b78faa11c65a6a
X-V-E-CD: 8ad83cf34a8c3732cb73dea81270d36d
X-V-R-CD: 2eec4d4b333905bf2498bde820845c7b
X-V-CD: 0
X-V-ID: ac0ca2b7-3a23-4faf-a07f-64408324150d
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-11-12_08:,, signatures=0
Received: from vimac.scv.apple.com (vimac.scv.apple.com [17.192.171.33]) by nwk-mmpp-sz12.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPSA id <0Q0V008JCLQP4D20@nwk-mmpp-sz12.apple.com>; Tue, 12 Nov 2019 13:44:49 -0800 (PST)
Sender: vidhi_goel@apple.com
From: Vidhi Goel <vidhi_goel@apple.com>
Message-id: <601D9D4F-A82C-475A-98CC-383C1F876C44@apple.com>
Content-type: multipart/alternative; boundary="Apple-Mail=_0B2AAECD-F241-409C-A752-DDFF9427AF62"
MIME-version: 1.0 (Mac OS X Mail 13.0 \(3578.1\))
Date: Tue, 12 Nov 2019 13:44:49 -0800
In-reply-to: <CADVnQynkBxTdapXN0rWOuWO3KXQ2qb6x=xhB35XrMU38JkX2DQ@mail.gmail.com>
Cc: "tcpm@ietf.org" <tcpm@ietf.org>, mallman@bbn.com
To: Neal Cardwell <ncardwell=40google.com@dmarc.ietf.org>
References: <78EF3761-7CAF-459E-A4C0-57CDEAFEA8EE@apple.com> <CADVnQynkBxTdapXN0rWOuWO3KXQ2qb6x=xhB35XrMU38JkX2DQ@mail.gmail.com>
X-Mailer: Apple Mail (2.3578.1)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-11-12_08:, , signatures=0
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/EeqjWJNcQtDufxX_BkBuyS_yNQE>
Subject: Re: [tcpm] =?utf-8?q?Linux_doesn=E2=80=99t_implement_RFC3465?=
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Nov 2019 21:45:03 -0000

+Mark Allman

Thank you Neal for the detailed explanation.

I believe that ABC was written to solve the problem with ACK counting by counting the number of bytes acknowledged for misbehaving receivers. Limiting the increase to 2*MSS was a good solution to avoid bursts at the time.

I agree that increasing the congestion window and controlling the burst rate are orthogonal issues. ABC intended to solve them by regulating the congestion window itself and this approach might have become outdated with implementations trying to solve these two issues separately.

Is it worth investing time in writing another draft to document what Linux (and possibly other implementations) do for more accurate congestion window growth and for solving the burst send issue via pacing, TSQ etc.?

Thanks,
Vidhi

> On Nov 5, 2019, at 3:25 PM, Neal Cardwell <ncardwell=40google.com@dmarc.ietf.org> wrote:
> 
> 
> 
> On Tue, Nov 5, 2019 at 5:08 PM Vidhi Goel <vidhi_goel=40apple..com@dmarc.ietf.org <mailto:40apple.com@dmarc.ietf.org>> wrote:
> Hi All,
> 
> RFC 3465 suggests to increase the congestion window by the number of previously unacknowledged bytes ACKed by each ACK limiting the increase to L (L = 2MSS or 1MSS in certain conditions). The limiting factor L helps to avoid the large bursts that could occur due to stretch ACKs, ACK loss or compressed ACKs (GRO/GSO).
> 
> But, when I looked at Linux source code, it implements one aspect of ABC (byte counting to defend from ACK attacks) but doesn’t  “cap” the increase to L. 
>  https://github.com/torvalds/linux/blob/master/net/ipv4/tcp_cong..c#L385 <https://github.com/torvalds/linux/blob/master/net/ipv4/tcp_cong.c#L385>
> 
> This is a bit confusing. How should we address this discrepancy?
> 
> The differences between the Linux code and the ABC RFC  ( https://tools.ietf.org/html/rfc3465 <https://tools.ietf.org/html/rfc3465> ) are intentional.. The Linux TCP code in this area is largely from:
> 
> 9f9843a751d0 tcp: properly handle stretch acks in slow start
> 95224ac1801c Merge branch 'tcp_stretch_acks'
> d6b1a8a92a14 tcp: fix timing issue in CUBIC slope calculation
> 9cd981dcf174 tcp: fix stretch ACK bugs in CUBIC
> c22bdca94782 tcp: fix stretch ACK bugs in Reno
> 814d488c6126 tcp: fix the timid additive increase on stretch ACKs
> e73ebb0881ea tcp: stretch ACK fixes prep
> 
> That code is based on experience at Google, both with internal traffic and google.com <http://google.com/> and YouTube traffic. Experience was showing that preventing the full use of ACKs in growing the cwnd in slow start and CUBIC's rapid growth phase causes anemic growth in real-world situations.
> 
> The basic principle in the Linux code is that the sender should use the ACKs to learn about the capacity of the path (both in volumetric and rate dimensions), and should not ignore that information. This allows the sender to quickly grow and achieve high throughput, even in the presence of stretch ACKs, which are pervasive, due to TSO/GSO, GRO/LRO, etc.
> 
> Considering bursts is important, but that can be tackled as an orthogonal issue. Bursts are avoided in the Linux TCP ecosystem by the combination of TSO autosizing, pacing, TSQ, and fair queueing.
> 
> Here is a summary of the Linux TCP design in this area:
>   https://netdevconf.info/1.2/papers/bbr-netdev-1.2.new.new.pdf <https://netdevconf.info/1.2/papers/bbr-netdev-1.2.new.new.pdf>
> 
> best,
> neal
>