Re: [tcpm] [EXTERNAL] Re: Linux doesn’t implement RFC3465

Yuchung Cheng <ycheng@google.com> Mon, 09 August 2021 20:48 UTC

Return-Path: <ycheng@google.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C5F663A16AF for <tcpm@ietfa.amsl.com>; Mon, 9 Aug 2021 13:48:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -18.097
X-Spam-Level:
X-Spam-Status: No, score=-18.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.499, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FrRO7LO_wyne for <tcpm@ietfa.amsl.com>; Mon, 9 Aug 2021 13:48:14 -0700 (PDT)
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EE21F3A16C9 for <tcpm@ietf.org>; Mon, 9 Aug 2021 13:48:13 -0700 (PDT)
Received: by mail-wm1-x32d.google.com with SMTP id h24-20020a1ccc180000b029022e0571d1a0so912084wmb.5 for <tcpm@ietf.org>; Mon, 09 Aug 2021 13:48:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hFI3TtC4ZkCQrc0clo/4IAUbSLubwMLaHCMSZqUpK1w=; b=nFKV+9Cht9wUZGvtMyG0TjMjZT8ySuvsdu6qbA9Z1Kg9neZOHz5eAjr/qNSOU3hNPO HkQ25e3wKeg9CvNlLPwx8rtf/sEA3q5cW9yltiqgX+4iu7DZ0vdKkMoWuYQaITfMVpU/ RLwuJYuw7hltCCFoe4Z89UYiHyXOXqU+p5PKbVjxjI22e//dpJVHdFLD7Ohgq2mPhA3h LiN5faIFbJ2QG09Z0DxaMvNqTg93shNAg9IICbhyYuB8QMmm7yqa1BUeJyl33gnrkBsG 0O2Dvj34Ypm3hIWcze4ra11NQ2H7Rlsi6pra5D6zzMssoTBSoWR3dzRJqYZ1E7hMfWeh hC9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hFI3TtC4ZkCQrc0clo/4IAUbSLubwMLaHCMSZqUpK1w=; b=kfda4L8c8iLEr5Tb/v39o82E7Tdl7GhzEivtn0ZNOT7xOwxKQpkizFAaGAieNwpXki SOuv03Jzej9UA/aGR/0xJgKDnDJ06KynQeiQHbnjQoMEOLPPKOdkTZXLF3vtKXd6wgvj 61/9KZt3l/BkKzHbJXTY4Rz8HXBJyqKFHNH+eRrhPkZPlHGNWAG09uhMOlT8lj4rFydg OVBeGLen06dPtt2Qt28KGOJUdLNLT7sN7EkCEi2yXe/8zlWzBTr2oWfoFCEd05aUJoSt DulLdLkk8nQX8xI0rCkx4UrLvHQxXFup9G346BpJCuy1mCgC89jYbU5JuqbhhNWJhsoO et2Q==
X-Gm-Message-State: AOAM530j1TcJ4g5+vneUlP4LTkA/xJciRfBbs8/sRaNhKNUcE2ngylUe 662dNCFYE3vbfiooqO4ESabJKu7BR+ljSJ1x53W3hQ==
X-Google-Smtp-Source: ABdhPJxcrlwxdJHu5noeQS7xiToCUYpZYWqgvsuF/7FG3rzTs+tlDStXekxMkVctYgqwuzmF88p3IVxUL1zq+upPVTg=
X-Received: by 2002:a05:600c:4145:: with SMTP id h5mr347765wmm.7.1628542087196; Mon, 09 Aug 2021 13:48:07 -0700 (PDT)
MIME-Version: 1.0
References: <78EF3761-7CAF-459E-A4C0-57CDEAFEA8EE@apple.com> <CADVnQynkBxTdapXN0rWOuWO3KXQ2qb6x=xhB35XrMU38JkX2DQ@mail.gmail.com> <601D9D4F-A82C-475A-98CC-383C1F876C44@apple.com> <54699CC9-C8F5-4CA3-8815-F7A21AE10429@icsi.berkeley.edu> <DF5EF1C7-0940-478A-9518-62185A79A288@apple.com> <E150D881-4AB3-4AEA-BE0C-1D4B47B2C531@icir.org> <CADVnQynjE+D-OSvdOVROjT3y1cnHHWqdNQSmphLAJ+HsBTUAJQ@mail.gmail.com> <A1B50403-2405-4348-9626-025D255DEAE7@icir.org> <CADVnQykM8p-bVz_oPrje1yNh9_7_isAUL+wnQWDoY9Gs18sLPQ@mail.gmail.com> <11FE4818-87E7-4FD8-8F45-E19CD9A3366A@apple.com> <CAK6E8=fFWAE_NSr45i2mdh6NmYDusUFW3GYGtuo-FcL07sox9A@mail.gmail.com> <D6B865F7-9865-4B6F-986B-F44ABE5F12B0@apple.com> <756432D9-4331-454D-82EB-346CF54A355E@icir.org> <CAK6E8=c+KeQxWJq0e98hY9XsQ2vhdr3SiKkypC7kwdZbBRgdXA@mail.gmail.com> <A39F73BE-4BF1-479D-911F-0CAC6D91D924@icir.org> <CAK6E8=eEnVtMNBpu0noFAud4BTWdupCH+QY1beFjTtD9ADkK5g@mail.gmail.com> <CADVnQynWSCpEBeEtHL0JHCBYwyymX0vku_VbfeDQ_snUoCX=ZA@mail.gmail.com> <76891287-22E6-4071-87C4-8F3A1FD3C2D1@apple.com> <CADVnQy=6XE7mFZRdBar3YXjUMc5URJYcsJvNdUGy26Zz7gajKQ@mail.gmail.com> <PH0PR00MB10302B312DB96B8A6324C55FB6F09@PH0PR00MB1030.namprd00.prod.outlook.com> <CADVnQymFri1mNW9a7WgWWNxp6pedrMkgx8e6qzshYmyw8D1JfA@mail.gmail.com> <CAK6E8=fBV_0F7ybTRLS9Y7c96Qf709jXWo8ZcciR3-Lnw-B+gg@mail.gmail.com> <CAK6E8=fmi=kzxeMFBMOo8f4n+8yZdrj8JtUWivqFE=E7aNWO9Q@mail.gmail.com> <CAK6E8=e1+BHd6vAfKgQq0LgnEd_qXbqWwS-exL2Y1VAK2umY7Q@mail.gmail.com> <13E800C6-8113-451E-9604-D67C6D45A5DF@apple.com> <CADVnQykH-kxkpdOGgQZxxWeCggGR22ffpgKnE6+PK9gZkVjXtQ@mail.gmail.com> <CAAK044Q1o1rdtNgFBMctHuEuQETJPJbm=et2WtwXTaB-SPg9pg@mail.gmail.com> <CAK6E8=dcUzw3ycM3AJoFNrJMbKskim6UYGP2oYS3PAQMRG0z2w@mail.gmail.com> <CADVnQy=+1+d0Yfd+-jnM5Nef7vJS3_OZRyf17B96A3=Ack0iSQ@mail.gmail.com> <CAAK044Q5APha_+8Q-EsyGTKb2d0_1Zjhtn8h-O0iHFQ7jVSCMw@mail.gmail.com> <54F87CD1-061E-48FE-8512-BFC0E2E4F96B@apple.com>
In-Reply-To: <54F87CD1-061E-48FE-8512-BFC0E2E4F96B@apple.com>
From: Yuchung Cheng <ycheng@google.com>
Date: Mon, 9 Aug 2021 13:47:29 -0700
Message-ID: <CAK6E8=cK6ccEQW_bgde+fX5KJ5gDuq=Siqj=+4v9Nr12tJ8QoA@mail.gmail.com>
To: Vidhi Goel <vidhi_goel=40apple.com@dmarc.ietf.org>
Cc: Yoshifumi Nishida <nsd.ietf@gmail.com>, Neal Cardwell <ncardwell@google.com>, "tcpm@ietf.org" <tcpm@ietf.org>, "mallman@icir.org" <mallman@icir.org>
Content-Type: multipart/alternative; boundary="0000000000009c8f9105c9268009"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/VEjtaEdw5xRMrnR1N7a1eA1fSa0>
Subject: Re: [tcpm] =?utf-8?q?=5BEXTERNAL=5D_Re=3A_Linux_doesn=E2=80=99t_impl?= =?utf-8?q?ement_RFC3465?=
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Aug 2021 20:48:20 -0000

+1 to Yoshifumi's proposal as well.



On Mon, Aug 9, 2021 at 1:33 PM Vidhi Goel <vidhi_goel=
40apple.com@dmarc.ietf.org> wrote:

> I also like Yoshi’s suggestion to fold the initial window into RFC5681-bis.
>
> That sounds reasonable to me as well. Although the initial window is a
>> somewhat independent issue that might evolve independently from the
>> congestion algorithm itself, so I can imagine advantages to keeping it
>> separate.
>>
>
> I think Initial Window is not independent from congestion control module,
> because lets say in future with higher BDPs we decide to set IW=1000, then
> the congestion control algorithm should be able to immediately detect
> congestion, if any, even at the start of the connection and for such a high
> initial value, probably requires other ways to detect congestion besides
> packet loss.
> Yes, it is separate from how the algorithm works itself, but still
> deciding the initial value depends a lot on the congestion control
> algorithm.
>
> Thanks,
> Vidhi
>
> On Aug 9, 2021, at 10:07 AM, Yoshifumi Nishida <nsd.ietf@gmail.com> wrote:
>
> Yes, that was my intention. Sorry for being unclear.
> I thought updating RFC5681 could be impactful. So, if it would happen, I
> would like to make sure we won't have another update for a long time.
> --
> Yoshi
>
> On Mon, Aug 9, 2021 at 9:53 AM Neal Cardwell <ncardwell@google.com> wrote:
>
>> I think Yoshifumi is suggesting that if the WG re-spins  RFC5681 then in
>> addition to folding in discussion of ABC/RFC3465 the RFC5681bis could also
>> include the IW10 content in RFC6928. That could help save time in avoiding
>> promoting RFC6928 from experimental to proposed standard.
>>
>> That sounds reasonable to me as well. Although the initial window is a
>> somewhat independent issue that might evolve independently from the
>> congestion algorithm itself, so I can imagine advantages to keeping it
>> separate.
>>
>> neal
>>
>>
>> On Mon, Aug 9, 2021 at 12:39 PM Yuchung Cheng <ycheng=
>> 40google.com@dmarc.ietf.org> wrote:
>>
>>> Sorry I don't understand your suggestion. Is that related to ABC? could
>>> you explain more
>>>
>>> On Mon, Aug 9, 2021 at 1:41 AM Yoshifumi Nishida <nsd.ietf@gmail.com>
>>> wrote:
>>>
>>>> I don't have a strong opinion on this yet, But, if we *could* move in
>>>> this direction, it might be good to think about the IW explanation in
>>>> RFC5681 as well?
>>>> if we do this, we might not need to discuss promoting RFC6928.
>>>> --
>>>> Yoshi
>>>>
>>>> On Sat, Aug 7, 2021 at 7:57 AM Neal Cardwell <ncardwell=
>>>> 40google.com@dmarc.ietf.org> wrote:
>>>>
>>>>> I also agree with Yuchung’s suggestion, for all of the reasons he
>>>>> provided.
>>>>>
>>>>> best,
>>>>> neal
>>>>>
>>>>>
>>>>> On Fri, Aug 6, 2021 at 3:59 PM Vidhi Goel <vidhi_goel=
>>>>> 40apple.com@dmarc.ietf.org> wrote:
>>>>>
>>>>>> I agree with Yuchung’s suggestion for all the reasons he provided.
>>>>>> And its better to have it at one place.
>>>>>>
>>>>>> Vidhi
>>>>>>
>>>>>> On Aug 6, 2021, at 12:53 PM, Yuchung Cheng <
>>>>>> ycheng=40google.com@dmarc.ietf.org> wrote:
>>>>>>
>>>>>> Hi WG
>>>>>>
>>>>>> I have been wondering if we (= IETF) should just update RFC5681
>>>>>> directly, instead of another RFC3465-bis with experimental status.
>>>>>>
>>>>>> Appropriate byte counting is essential but the RFC5681 of L=1 is
>>>>>> detrimental. There are far more people who read RFC5681 to implement the
>>>>>> new stack instead of RFC3465. So we should fold the experimental RFC3465
>>>>>> updates into RFC5681 directly, and obsolete RFC3465.
>>>>>>
>>>>>> This is orthogonal to the final value of L :-)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 3, 2021 at 9:42 AM Yuchung Cheng <ycheng@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Aug 2, 2021 at 6:12 PM Yuchung Cheng <ycheng@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 2, 2021 at 5:53 PM Neal Cardwell <ncardwell=
>>>>>>>> 40google.com@dmarc.ietf.org> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Aug 2, 2021 at 8:46 PM Praveen Balasubramanian <pravb=
>>>>>>>>> 40microsoft.com@dmarc.ietf.org> wrote:
>>>>>>>>>
>>>>>>>>>> In experiments a few years ago on DC networks, values over L=8
>>>>>>>>>> resulted in a noticeable increase in packet drops and retransmissions
>>>>>>>>>> (without pacing). Windows TCP has been using L=8 for many years now. If we
>>>>>>>>>> do want to specify a fallback L value for implementations that cannot pace,
>>>>>>>>>> my suggestion would be to use the value 8.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Neal, are there cases where Linux is or can be deployed with
>>>>>>>>>> infinite L and no pacing?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes, "infinite L and no pacing" is the default behavior for Linux
>>>>>>>>> TCP, starting in 2013 for slow-start and then starting in 2015 for
>>>>>>>>> congestion avoidance.
>>>>>>>>>
>>>>>>>> To be more clear: both fq_pacing and TCP pacing have been disabled
>>>>>>>> by default in Linux upstream. We do not know how much Linux senders enable
>>>>>>>> them today besides the Google servers.
>>>>>>>>
>>>>>>>> Regarding L = 8, to avoid another round of why or why not. We could
>>>>>>>> say inf-L causes line-rate burst up to the stretched ACK degree so put a
>>>>>>>> comfortable L if you prefer, then mention implementation practice like
>>>>>>>> yours. At the end of the day it's ad-hoc (or "art") and subject to change.
>>>>>>>> It might be sensible to cap at cwnd to disincentivize receivers /
>>>>>>>> middle-boxes bunching up 10 rounds of ACKs.
>>>>>>>>
>>>>>>> Sorry please ignore my previous message about the cwnd cap. It is
>>>>>>> completely unnecessary -- since with ack-clocking and appropriate counting,
>>>>>>> a correct sender would never release more than a cwnd-worth of data. I was
>>>>>>> imagining the multiple application-limited burst could let the receiver
>>>>>>> keep holding up ACKs, but that can never exceed a cwnd worth of data.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> Yuchung pasted the URLs for the exact Linux commits above, which
>>>>>>>>> are:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9f9843a751d0a2057f9f3d313886e7e5e6ebaac9
>>>>>>>>>
>>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9cd981dcf174d26805a032aefa791436da709bee
>>>>>>>>>
>>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c22bdca94782f05b9337d8548bde51b2f38ef17f
>>>>>>>>>
>>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=814d488c61260521b1b3cc97063700a5a6667c8f
>>>>>>>>>
>>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e73ebb0881ea5534ce606c1d71b4ac44db5c6930
>>>>>>>>>
>>>>>>>>> But I understand that not everyone is in a position to read
>>>>>>>>> GPL-licensed code. :-)
>>>>>>>>>
>>>>>>>>> best regards,
>>>>>>>>> neal
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *From:* tcpm <tcpm-bounces@ietf.org> *On Behalf Of * Neal
>>>>>>>>>> Cardwell
>>>>>>>>>> *Sent:* Monday, August 2, 2021 4:18 PM
>>>>>>>>>> *To:* Vidhi Goel <vidhi_goel@apple.com>
>>>>>>>>>> *Cc:* Extensions <tcpm@ietf.org>rg>; Mark Allman <mallman@icir.org>
>>>>>>>>>> *Subject:* [EXTERNAL] Re: [tcpm] Linux doesn’t implement RFC3465
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Aug 2, 2021 at 7:02 PM Vidhi Goel <vidhi_goel@apple.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Aug 2, 2021 at 3:37 PM Mark Allman <mallman@icir.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> > The fact is that Linux CC has long moved to infinite L since
>>>>>>>>>> 2031,
>>>>>>>>>>
>>>>>>>>>> So, if our experience is with L=\infinity and it is demonstrably
>>>>>>>>>> OK
>>>>>>>>>> why don't we say *THAT* instead of "make L=5 or L=10"?  I would
>>>>>>>>>> submit that it makes more sense to leverage experience than it
>>>>>>>>>> does
>>>>>>>>>>
>>>>>>>>>> to make things up.
>>>>>>>>>>
>>>>>>>>>> +1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, I agree that would be a great approach to take.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So, we are saying it is fine to ignore L completely and simply
>>>>>>>>>> increase cwnd by bytes_acked during slow start? And if this causes large
>>>>>>>>>> bursts to be sent out (when an implementation doesn’t do pacing), that is
>>>>>>>>>> fine?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, I think that is the proposal on the table, and it sounds
>>>>>>>>>> good to me.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> A rationale would be:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (1) Implementations SHOULD pace (RFC 7661).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (2) Implementations that don't pace will generally be causing
>>>>>>>>>> large bursts for many different reasons anyway (data and/or ACK aggregation
>>>>>>>>>> in the network or end hosts), restart from idle,...) so having a constant L
>>>>>>>>>> does not provide enough protection from bursts to justify the cost in
>>>>>>>>>> reduced performance (in the form of slower slow-start). In support of this,
>>>>>>>>>> experience with this as the default behavior in Linux TCP over the
>>>>>>>>>> 2013-2021 period suggests this works well enough in practice.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> neal
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> tcpm mailing list
>>>>>>>>> tcpm@ietf.org
>>>>>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>> tcpm mailing list
>>>>>> tcpm@ietf.org
>>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>> tcpm mailing list
>>>>> tcpm@ietf.org
>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>>
>>>> _______________________________________________
>>>> tcpm mailing list
>>>> tcpm@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>>
>>>
>