Re: [tcpm] [EXTERNAL] Re: Linux doesn’t implement RFC3465

Vidhi Goel <vidhi_goel@apple.com> Fri, 06 August 2021 19:59 UTC

Return-Path: <vidhi_goel@apple.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8F363A1415 for <tcpm@ietfa.amsl.com>; Fri, 6 Aug 2021 12:59:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.552
X-Spam-Level:
X-Spam-Status: No, score=-2.552 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.452, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=apple.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JzfYElxNUqOY for <tcpm@ietfa.amsl.com>; Fri, 6 Aug 2021 12:59:43 -0700 (PDT)
Received: from ma1-aaemail-dr-lapp03.apple.com (ma1-aaemail-dr-lapp03.apple.com [17.171.2.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 667313A1413 for <tcpm@ietf.org>; Fri, 6 Aug 2021 12:59:43 -0700 (PDT)
Received: from pps.filterd (ma1-aaemail-dr-lapp03.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp03.apple.com (8.16.0.42/8.16.0.42) with SMTP id 176JbZca016919; Fri, 6 Aug 2021 12:59:40 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=ytFHMdr0T1kDnIa5tPWctaadBsRkOr8dpn4uf6wnTAI=; b=cNDrDCtmuWJVLaiC8F2kRptp50vYNg849sMsLAweWZcTV+KdR0QFKKxCRC4jyWY1y/hK oohCLz8IKIoaj+GKqn9MTWQ3pJsFcDiLBAbtGz+HKUmlB80O80pbmC32tKQKJLEo5nmm lbrekSgkPPpFHkWAkhEhnEIaHvJndHBLCcqIq5O4jVS6HUKg8Ayv783TNikjKkX3x1AA UcxV+Cjt8yRuin0c8PXstNKsB51IM2cXy/k9UPLXMnpczIdw0QSjzGjRYcK3SJePbVUQ GbPMyFDl3dQ8zIHpec0zvGC5PECRxLeZ9d9zj8e20kLYrC5m5KfLAMwF5smxQG6DZ0zG AQ==
Received: from rn-mailsvcp-mta-lapp04.rno.apple.com (rn-mailsvcp-mta-lapp04.rno.apple.com [10.225.203.152]) by ma1-aaemail-dr-lapp03.apple.com with ESMTP id 3a55gwrhke-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 06 Aug 2021 12:59:40 -0700
Received: from rn-mailsvcp-mmp-lapp03.rno.apple.com (rn-mailsvcp-mmp-lapp03.rno.apple.com [17.179.253.16]) by rn-mailsvcp-mta-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.9.20210415 64bit (built Apr 15 2021)) with ESMTPS id <0QXF004J1OVF3R30@rn-mailsvcp-mta-lapp04.rno.apple.com>; Fri, 06 Aug 2021 12:59:39 -0700 (PDT)
Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp03.rno.apple.com by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.9.20210415 64bit (built Apr 15 2021)) id <0QXF00Q00OSAKR00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Fri, 06 Aug 2021 12:59:39 -0700 (PDT)
X-Va-A:
X-Va-T-CD: 637c60740750869fdac3fa3bd3c9b6e8
X-Va-E-CD: 0356cfd288405c7383fa735d89592edc
X-Va-R-CD: 776b594e15c7bc14f32f514fb67ec3c1
X-Va-CD: 0
X-Va-ID: a5c29d9b-84ef-4a68-8735-546473599155
X-V-A:
X-V-T-CD: 637c60740750869fdac3fa3bd3c9b6e8
X-V-E-CD: 0356cfd288405c7383fa735d89592edc
X-V-R-CD: 776b594e15c7bc14f32f514fb67ec3c1
X-V-CD: 0
X-V-ID: 2cf2987e-4b84-40d5-ad67-2926c7b0896f
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-08-06_06:2021-08-06, 2021-08-06 signatures=0
Received: from smtpclient.apple (unknown [17.11.36.131]) by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.9.20210415 64bit (built Apr 15 2021)) with ESMTPSA id <0QXF00S2LOVERA00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Fri, 06 Aug 2021 12:59:39 -0700 (PDT)
From: Vidhi Goel <vidhi_goel@apple.com>
Message-id: <13E800C6-8113-451E-9604-D67C6D45A5DF@apple.com>
Content-type: multipart/signed; boundary="Apple-Mail=_149BCDAC-FDEB-4935-9700-63929DFAB64D"; protocol="application/pkcs7-signature"; micalg=sha-256
MIME-version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.11\))
Date: Fri, 06 Aug 2021 12:59:38 -0700
In-reply-to: <CAK6E8=e1+BHd6vAfKgQq0LgnEd_qXbqWwS-exL2Y1VAK2umY7Q@mail.gmail.com>
Cc: Neal Cardwell <ncardwell=40google.com@dmarc.ietf.org>, Praveen Balasubramanian <pravb=40microsoft.com@dmarc.ietf.org>, "tcpm@ietf.org" <tcpm@ietf.org>, "mallman@icir.org" <mallman@icir.org>
To: Yuchung Cheng <ycheng=40google.com@dmarc.ietf.org>
References: <78EF3761-7CAF-459E-A4C0-57CDEAFEA8EE@apple.com> <CADVnQynkBxTdapXN0rWOuWO3KXQ2qb6x=xhB35XrMU38JkX2DQ@mail.gmail.com> <601D9D4F-A82C-475A-98CC-383C1F876C44@apple.com> <54699CC9-C8F5-4CA3-8815-F7A21AE10429@icsi.berkeley.edu> <DF5EF1C7-0940-478A-9518-62185A79A288@apple.com> <E150D881-4AB3-4AEA-BE0C-1D4B47B2C531@icir.org> <CADVnQynjE+D-OSvdOVROjT3y1cnHHWqdNQSmphLAJ+HsBTUAJQ@mail.gmail.com> <A1B50403-2405-4348-9626-025D255DEAE7@icir.org> <CADVnQykM8p-bVz_oPrje1yNh9_7_isAUL+wnQWDoY9Gs18sLPQ@mail.gmail.com> <11FE4818-87E7-4FD8-8F45-E19CD9A3366A@apple.com> <CAK6E8=fFWAE_NSr45i2mdh6NmYDusUFW3GYGtuo-FcL07sox9A@mail.gmail.com> <D6B865F7-9865-4B6F-986B-F44ABE5F12B0@apple.com> <756432D9-4331-454D-82EB-346CF54A355E@icir.org> <CAK6E8=c+KeQxWJq0e98hY9XsQ2vhdr3SiKkypC7kwdZbBRgdXA@mail.gmail.com> <A39F73BE-4BF1-479D-911F-0CAC6D91D924@icir.org> <CAK6E8=eEnVtMNBpu0noFAud4BTWdupCH+QY1beFjTtD9ADkK5g@mail.gmail.com> <CADVnQynWSCpEBeEtHL0JHCBYwyymX0vku_VbfeDQ_snUoCX=ZA@mail.gmail.com> <76891287-22E6-4071-87C4-8F3A1FD3C2D1@apple.com> <CADVnQy=6XE7mFZRdBar3YXjUMc5URJYcsJvNdUGy26Zz7gajKQ@mail.gmail.com> <PH0PR00MB10302B312DB96B8A6324C55FB6F09@PH0PR00MB1030.namprd00.prod.outlook.com> <CADVnQymFri1mNW9a7WgWWNxp6pedrMkgx8e6qzshYmyw8D1JfA@mail.gmail.com> <CAK6E8=fBV_0F7ybTRLS9Y7c96Qf709jXWo8ZcciR3-Lnw-B+gg@mail.gmail.com> <CAK6E8=fmi=kzxeMFBMOo8f4n+8yZdrj8JtUWivqFE=E7aNWO9Q@mail.gmail.com> <CAK6E8=e1+BHd6vAfKgQq0LgnEd_qXbqWwS-exL2Y1VAK2umY7Q@mail.gmail.com>
X-Mailer: Apple Mail (2.3654.100.0.2.11)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-08-06_06:2021-08-06, 2021-08-06 signatures=0
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/zfM1ah6MsMy3UnSxOZP22nS3Zmk>
Subject: Re: [tcpm] =?utf-8?q?=5BEXTERNAL=5D_Re=3A_Linux_doesn=E2=80=99t_impl?= =?utf-8?q?ement_RFC3465?=
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Aug 2021 19:59:48 -0000

I agree with Yuchung’s suggestion for all the reasons he provided. And its better to have it at one place.

Vidhi

> On Aug 6, 2021, at 12:53 PM, Yuchung Cheng <ycheng=40google.com@dmarc.ietf.org> wrote:
> 
> Hi WG
> 
> I have been wondering if we (= IETF) should just update RFC5681 directly, instead of another RFC3465-bis with experimental status.
> 
> Appropriate byte counting is essential but the RFC5681 of L=1 is detrimental. There are far more people who read RFC5681 to implement the new stack instead of RFC3465. So we should fold the experimental RFC3465 updates into RFC5681 directly, and obsolete RFC3465.
> 
> This is orthogonal to the final value of L :-)
> 
> 
> 
> 
> On Tue, Aug 3, 2021 at 9:42 AM Yuchung Cheng <ycheng@google.com <mailto:ycheng@google.com>> wrote:
> 
> 
> On Mon, Aug 2, 2021 at 6:12 PM Yuchung Cheng <ycheng@google.com <mailto:ycheng@google.com>> wrote:
> 
> 
> On Mon, Aug 2, 2021 at 5:53 PM Neal Cardwell <ncardwell=40google.com@dmarc.ietf.org <mailto:40google.com@dmarc.ietf.org>> wrote:
> 
> 
> On Mon, Aug 2, 2021 at 8:46 PM Praveen Balasubramanian <pravb=40microsoft.com@dmarc.ietf.org <mailto:40microsoft.com@dmarc.ietf.org>> wrote:
> In experiments a few years ago on DC networks, values over L=8 resulted in a noticeable increase in packet drops and retransmissions (without pacing). Windows TCP has been using L=8 for many years now. If we do want to specify a fallback L value for implementations that cannot pace, my suggestion would be to use the value 8.
> 
>  
> 
> Neal, are there cases where Linux is or can be deployed with infinite L and no pacing?
> 
> 
> Yes, "infinite L and no pacing" is the default behavior for Linux TCP, starting in 2013 for slow-start and then starting in 2015 for congestion avoidance.
> To be more clear: both fq_pacing and TCP pacing have been disabled by default in Linux upstream. We do not know how much Linux senders enable them today besides the Google servers.
> 
> Regarding L = 8, to avoid another round of why or why not. We could say inf-L causes line-rate burst up to the stretched ACK degree so put a comfortable L if you prefer, then mention implementation practice like yours. At the end of the day it's ad-hoc (or "art") and subject to change. It might be sensible to cap at cwnd to disincentivize receivers / middle-boxes bunching up 10 rounds of ACKs.
> Sorry please ignore my previous message about the cwnd cap. It is completely unnecessary -- since with ack-clocking and appropriate counting, a correct sender would never release more than a cwnd-worth of data. I was imagining the multiple application-limited burst could let the receiver keep holding up ACKs, but that can never exceed a cwnd worth of data.
>  
> 
>  
> Yuchung pasted the URLs for the exact Linux commits above, which are:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9f9843a751d0a2057f9f3d313886e7e5e6ebaac9 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9f9843a751d0a2057f9f3d313886e7e5e6ebaac9>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9cd981dcf174d26805a032aefa791436da709bee <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9cd981dcf174d26805a032aefa791436da709bee>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c22bdca94782f05b9337d8548bde51b2f38ef17f <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c22bdca94782f05b9337d8548bde51b2f38ef17f>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=814d488c61260521b1b3cc97063700a5a6667c8f <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=814d488c61260521b1b3cc97063700a5a6667c8f>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e73ebb0881ea5534ce606c1d71b4ac44db5c6930 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e73ebb0881ea5534ce606c1d71b4ac44db5c6930>
> 
> But I understand that not everyone is in a position to read GPL-licensed code. :-)
> 
> best regards,
> neal
> 
>  
>  
> 
> From: tcpm <tcpm-bounces@ietf.org <mailto:tcpm-bounces@ietf.org>> On Behalf Of Neal Cardwell
> Sent: Monday, August 2, 2021 4:18 PM
> To: Vidhi Goel <vidhi_goel@apple.com <mailto:vidhi_goel@apple.com>>
> Cc: Extensions <tcpm@ietf.org <mailto:tcpm@ietf.org>>; Mark Allman <mallman@icir.org <mailto:mallman@icir.org>>
> Subject: [EXTERNAL] Re: [tcpm] Linux doesn’t implement RFC3465
> 
>  
> 
>  
> 
>  
> 
> On Mon, Aug 2, 2021 at 7:02 PM Vidhi Goel <vidhi_goel@apple.com <mailto:vidhi_goel@apple.com>> wrote:
> 
>  
> 
> On Mon, Aug 2, 2021 at 3:37 PM Mark Allman <mallman@icir.org <mailto:mallman@icir.org>> wrote:
> 
> 
> > The fact is that Linux CC has long moved to infinite L since 2031,
> 
> So, if our experience is with L=\infinity and it is demonstrably OK
> why don't we say *THAT* instead of "make L=5 or L=10"?  I would
> submit that it makes more sense to leverage experience than it does 
> 
> to make things up.
> 
> +1
> 
>  
> 
> Yes, I agree that would be a great approach to take.
> 
>  
> 
> So, we are saying it is fine to ignore L completely and simply increase cwnd by bytes_acked during slow start? And if this causes large bursts to be sent out (when an implementation doesn’t do pacing), that is fine? 
> 
>  
> 
> Yes, I think that is the proposal on the table, and it sounds good to me.
> 
>  
> 
> A rationale would be:
> 
>  
> 
> (1) Implementations SHOULD pace (RFC 7661).
> 
>  
> 
> (2) Implementations that don't pace will generally be causing large bursts for many different reasons anyway (data and/or ACK aggregation in the network or end hosts), restart from idle,...) so having a constant L does not provide enough protection from bursts to justify the cost in reduced performance (in the form of slower slow-start). In support of this, experience with this as the default behavior in Linux TCP over the  2013-2021 period suggests this works well enough in practice.
> 
>  
> 
> neal
> 
>  
> 
>  
> 
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org <mailto:tcpm@ietf.org>
> https://www.ietf.org/mailman/listinfo/tcpm <https://www.ietf.org/mailman/listinfo/tcpm>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm