Re: [tcpm] [EXTERNAL] Re: Linux doesn’t implement RFC3465

Yuchung Cheng <ycheng@google.com> Fri, 06 August 2021 19:54 UTC

Return-Path: <ycheng@google.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 782823A13D9 for <tcpm@ietfa.amsl.com>; Fri, 6 Aug 2021 12:54:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -18.097
X-Spam-Level:
X-Spam-Status: No, score=-18.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.499, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N59LkXbTnK30 for <tcpm@ietfa.amsl.com>; Fri, 6 Aug 2021 12:54:35 -0700 (PDT)
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8FBAC3A13CA for <tcpm@ietf.org>; Fri, 6 Aug 2021 12:54:31 -0700 (PDT)
Received: by mail-wr1-x42a.google.com with SMTP id m12so12415285wru.12 for <tcpm@ietf.org>; Fri, 06 Aug 2021 12:54:31 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=06Pon2QBCWSot8PslbNQItE0UN34qDLlNZyELGYlxpE=; b=dbZkDDBvX9BCtbs3ia2gUTGJjrToPC0yPLnklVS4GsWZlEmolO0bUkVz/lkAyPr/8d 4ZuDHUxNZZwHgWOV4mKOOjtuWwfOCsmVHNNWeAOzlYAbTtIjYeSHFKiJcoqQfhLgbGjR 8i8VaFX4jst90bJ/tcapGtX4eJzDcO4Dq3z48Zti22FEjnPH51QKis0/WnuqpTv0Yyav o912u2bmAO5yEDEPcxfVTpVkdQvEKw6Vwq7tOdvi6o1MBL0jU2twtJ3o7yc06Z5K81hw EP32u0QKHksKOdtooMSJPyosFFArW5HEfffPdrNrY9wZmZYTBUU5LcjyHGSkU6womZdu GpfA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=06Pon2QBCWSot8PslbNQItE0UN34qDLlNZyELGYlxpE=; b=M5u6mR6kaqipR3D3rHZvETxGZyhA7N5gf0KyzJd76WkstsXAyTf+4XbYC0TIzmymyW RtfTcULBFheQB+Z/4GwGoZVlbvniG2w2iUCP44f7VWcg6Kd9UItDmbpjNSZrC+DBhZTy DoGFFr/KJzYWQNFCJnIb8TQGJnFy7Lr6rJ5gCnnG0LV6w5rISq5UviRgD7Hnn1x+V0ip m+0dyjc2Cz6dC5IAVUnCYTSoIRSxKSkfs7adDWHsWCelFSwYqe5vwAAIraa9W2wPz1h3 7oOo1vKx+jnqpmP1BjTRSjHIEU6SAddH1gT2/g5faQo1DbB27gVtl7E0NH4cmlFEvm9E 6jgg==
X-Gm-Message-State: AOAM5323J+vcyYMw+3oKENVW0YvbrshYJ/9vUIa//Nsj4lJCvDV5equa jpOKeoCv3KLRPmDQJZZq1a6z/jIh+uCZ828EAsv1qUfstw467A==
X-Google-Smtp-Source: ABdhPJyibC4J8V95k4kSHivQT0QCHKyGrdkdQqC5JqgiT+t3wCe0mw869NPxrMkIeRfqOoFc0eDNkbMlg6ohMpmmsV0=
X-Received: by 2002:adf:e107:: with SMTP id t7mr12529031wrz.165.1628279668749; Fri, 06 Aug 2021 12:54:28 -0700 (PDT)
MIME-Version: 1.0
References: <78EF3761-7CAF-459E-A4C0-57CDEAFEA8EE@apple.com> <CADVnQynkBxTdapXN0rWOuWO3KXQ2qb6x=xhB35XrMU38JkX2DQ@mail.gmail.com> <601D9D4F-A82C-475A-98CC-383C1F876C44@apple.com> <54699CC9-C8F5-4CA3-8815-F7A21AE10429@icsi.berkeley.edu> <DF5EF1C7-0940-478A-9518-62185A79A288@apple.com> <E150D881-4AB3-4AEA-BE0C-1D4B47B2C531@icir.org> <CADVnQynjE+D-OSvdOVROjT3y1cnHHWqdNQSmphLAJ+HsBTUAJQ@mail.gmail.com> <A1B50403-2405-4348-9626-025D255DEAE7@icir.org> <CADVnQykM8p-bVz_oPrje1yNh9_7_isAUL+wnQWDoY9Gs18sLPQ@mail.gmail.com> <11FE4818-87E7-4FD8-8F45-E19CD9A3366A@apple.com> <CAK6E8=fFWAE_NSr45i2mdh6NmYDusUFW3GYGtuo-FcL07sox9A@mail.gmail.com> <D6B865F7-9865-4B6F-986B-F44ABE5F12B0@apple.com> <756432D9-4331-454D-82EB-346CF54A355E@icir.org> <CAK6E8=c+KeQxWJq0e98hY9XsQ2vhdr3SiKkypC7kwdZbBRgdXA@mail.gmail.com> <A39F73BE-4BF1-479D-911F-0CAC6D91D924@icir.org> <CAK6E8=eEnVtMNBpu0noFAud4BTWdupCH+QY1beFjTtD9ADkK5g@mail.gmail.com> <CADVnQynWSCpEBeEtHL0JHCBYwyymX0vku_VbfeDQ_snUoCX=ZA@mail.gmail.com> <76891287-22E6-4071-87C4-8F3A1FD3C2D1@apple.com> <CADVnQy=6XE7mFZRdBar3YXjUMc5URJYcsJvNdUGy26Zz7gajKQ@mail.gmail.com> <PH0PR00MB10302B312DB96B8A6324C55FB6F09@PH0PR00MB1030.namprd00.prod.outlook.com> <CADVnQymFri1mNW9a7WgWWNxp6pedrMkgx8e6qzshYmyw8D1JfA@mail.gmail.com> <CAK6E8=fBV_0F7ybTRLS9Y7c96Qf709jXWo8ZcciR3-Lnw-B+gg@mail.gmail.com> <CAK6E8=fmi=kzxeMFBMOo8f4n+8yZdrj8JtUWivqFE=E7aNWO9Q@mail.gmail.com>
In-Reply-To: <CAK6E8=fmi=kzxeMFBMOo8f4n+8yZdrj8JtUWivqFE=E7aNWO9Q@mail.gmail.com>
From: Yuchung Cheng <ycheng@google.com>
Date: Fri, 6 Aug 2021 12:53:51 -0700
Message-ID: <CAK6E8=e1+BHd6vAfKgQq0LgnEd_qXbqWwS-exL2Y1VAK2umY7Q@mail.gmail.com>
To: Neal Cardwell <ncardwell=40google.com@dmarc.ietf.org>
Cc: Praveen Balasubramanian <pravb=40microsoft.com@dmarc.ietf.org>, "tcpm@ietf.org" <tcpm@ietf.org>, "mallman@icir.org" <mallman@icir.org>
Content-Type: multipart/alternative; boundary="00000000000040d5a605c8e9671c"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/bUDL1ku3GeJ1927MyuOp7ho1x_A>
Subject: Re: [tcpm] =?utf-8?q?=5BEXTERNAL=5D_Re=3A_Linux_doesn=E2=80=99t_impl?= =?utf-8?q?ement_RFC3465?=
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Aug 2021 19:54:40 -0000

Hi WG

I have been wondering if we (= IETF) should just update RFC5681 directly,
instead of another RFC3465-bis with experimental status.

Appropriate byte counting is essential but the RFC5681 of L=1 is
detrimental. There are far more people who read RFC5681 to implement the
new stack instead of RFC3465. So we should fold the experimental RFC3465
updates into RFC5681 directly, and obsolete RFC3465.

This is orthogonal to the final value of L :-)




On Tue, Aug 3, 2021 at 9:42 AM Yuchung Cheng <ycheng@google.com> wrote:

>
>
> On Mon, Aug 2, 2021 at 6:12 PM Yuchung Cheng <ycheng@google.com> wrote:
>
>>
>>
>> On Mon, Aug 2, 2021 at 5:53 PM Neal Cardwell <ncardwell=
>> 40google.com@dmarc.ietf.org> wrote:
>>
>>>
>>>
>>> On Mon, Aug 2, 2021 at 8:46 PM Praveen Balasubramanian <pravb=
>>> 40microsoft.com@dmarc.ietf.org> wrote:
>>>
>>>> In experiments a few years ago on DC networks, values over L=8 resulted
>>>> in a noticeable increase in packet drops and retransmissions (without
>>>> pacing). Windows TCP has been using L=8 for many years now. If we do want
>>>> to specify a fallback L value for implementations that cannot pace, my
>>>> suggestion would be to use the value 8.
>>>>
>>>>
>>>>
>>>> Neal, are there cases where Linux is or can be deployed with infinite L
>>>> and no pacing?
>>>>
>>>
>>> Yes, "infinite L and no pacing" is the default behavior for Linux TCP,
>>> starting in 2013 for slow-start and then starting in 2015 for congestion
>>> avoidance.
>>>
>> To be more clear: both fq_pacing and TCP pacing have been disabled by
>> default in Linux upstream. We do not know how much Linux senders enable
>> them today besides the Google servers.
>>
>> Regarding L = 8, to avoid another round of why or why not. We could say
>> inf-L causes line-rate burst up to the stretched ACK degree so put a
>> comfortable L if you prefer, then mention implementation practice like
>> yours. At the end of the day it's ad-hoc (or "art") and subject to change.
>> It might be sensible to cap at cwnd to disincentivize receivers /
>> middle-boxes bunching up 10 rounds of ACKs.
>>
> Sorry please ignore my previous message about the cwnd cap. It is
> completely unnecessary -- since with ack-clocking and appropriate counting,
> a correct sender would never release more than a cwnd-worth of data. I was
> imagining the multiple application-limited burst could let the receiver
> keep holding up ACKs, but that can never exceed a cwnd worth of data.
>
>
>>
>>>
>>
>>> Yuchung pasted the URLs for the exact Linux commits above, which are:
>>>
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9f9843a751d0a2057f9f3d313886e7e5e6ebaac9
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9cd981dcf174d26805a032aefa791436da709bee
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c22bdca94782f05b9337d8548bde51b2f38ef17f
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=814d488c61260521b1b3cc97063700a5a6667c8f
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e73ebb0881ea5534ce606c1d71b4ac44db5c6930
>>>
>>> But I understand that not everyone is in a position to read GPL-licensed
>>> code. :-)
>>>
>>> best regards,
>>> neal
>>>
>>>
>>>
>>>>
>>>>
>>>> *From:* tcpm <tcpm-bounces@ietf.org> *On Behalf Of * Neal Cardwell
>>>> *Sent:* Monday, August 2, 2021 4:18 PM
>>>> *To:* Vidhi Goel <vidhi_goel@apple.com>
>>>> *Cc:* Extensions <tcpm@ietf.org>rg>; Mark Allman <mallman@icir.org>
>>>> *Subject:* [EXTERNAL] Re: [tcpm] Linux doesn’t implement RFC3465
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Aug 2, 2021 at 7:02 PM Vidhi Goel <vidhi_goel@apple.com> wrote:
>>>>
>>>>
>>>>
>>>> On Mon, Aug 2, 2021 at 3:37 PM Mark Allman <mallman@icir.org> wrote:
>>>>
>>>>
>>>> > The fact is that Linux CC has long moved to infinite L since 2031,
>>>>
>>>> So, if our experience is with L=\infinity and it is demonstrably OK
>>>> why don't we say *THAT* instead of "make L=5 or L=10"?  I would
>>>> submit that it makes more sense to leverage experience than it does
>>>>
>>>> to make things up.
>>>>
>>>> +1
>>>>
>>>>
>>>>
>>>> Yes, I agree that would be a great approach to take.
>>>>
>>>>
>>>>
>>>> So, we are saying it is fine to ignore L completely and simply increase
>>>> cwnd by bytes_acked during slow start? And if this causes large bursts to
>>>> be sent out (when an implementation doesn’t do pacing), that is fine?
>>>>
>>>>
>>>>
>>>> Yes, I think that is the proposal on the table, and it sounds good to
>>>> me.
>>>>
>>>>
>>>>
>>>> A rationale would be:
>>>>
>>>>
>>>>
>>>> (1) Implementations SHOULD pace (RFC 7661).
>>>>
>>>>
>>>>
>>>> (2) Implementations that don't pace will generally be causing large
>>>> bursts for many different reasons anyway (data and/or ACK aggregation in
>>>> the network or end hosts), restart from idle,...) so having a constant L
>>>> does not provide enough protection from bursts to justify the cost in
>>>> reduced performance (in the form of slower slow-start). In support of this,
>>>> experience with this as the default behavior in Linux TCP over the
>>>> 2013-2021 period suggests this works well enough in practice.
>>>>
>>>>
>>>>
>>>> neal
>>>>
>>>>
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> tcpm mailing list
>>> tcpm@ietf.org
>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>
>>