Re: [tcpm] Linux doesn’t implement RFC3465

Yuchung Cheng <> Mon, 02 August 2021 22:02 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id E16903A1E6B for <>; Mon, 2 Aug 2021 15:02:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -18.097
X-Spam-Status: No, score=-18.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.499, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id GTh9brbaOx09 for <>; Mon, 2 Aug 2021 15:02:02 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 3CCEC3A1E69 for <>; Mon, 2 Aug 2021 15:02:02 -0700 (PDT)
Received: by with SMTP id p5so23124238wro.7 for <>; Mon, 02 Aug 2021 15:02:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BivLKVOKBddBdWQJdWrS2LbXbK/DK/KQSfV5JiGptCE=; b=q/pAzXkLDxoie3BeKc1aytn19oZgyhHLF6bid0JrahOyOxX+xZ7j5s60o62elF/uzS FZ3+EWQwh1KRWN8KDGiCkiC6qsCVkkRUFbyAi+GiGWaywcJU4TyTs80w/6to9pkkX1kj zl8MXhRk+B5GoY9N6jk435J7L/q49iWT0/HHNYWsAfGS4TCJJAmKHoHmQCPqt0w0mqL9 X6kkfqol3et0TWdXy1LVttknZJCfDBO9+KzqM/uKelNqTfwtgiZuuFRILmXXerH4kOW8 QKnx3qNkpaRr+4AjlUNeeU5MLFAfTUS3lDw8QoKvbR6U574MlSfMPm+B9f4b7hDOq6QO upHw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BivLKVOKBddBdWQJdWrS2LbXbK/DK/KQSfV5JiGptCE=; b=XrGEiNeo2Nu80hfL3PB5PWoxTZOGCWHdrSTiEGHhgp77Yal5/9RnlL8S+5FBrVGgXr znk+A9hBQs3Drt0ALQl35rpDfiO+aaJF1/L6MIpHYDu0NKOgx+DioZ1nB0KG+qvjWlQE hvjmn+qGG1lCo3Ic4kZObwDqOVGJCkTb71YeF7Zsg16Id/9JYPBCafb+7Zy77wXPaFTm SDVmQv2EeyRLdTpe0X0m/iQZY4DDV2FGEhu6ht1/loIsXdfdQVxJqan1yxKGSSuE8LdI gttF67x20s/VUWGY5g7EGDRy7DozznNkdmJ62sY28YjlMd1XOWgbLWqxHrOpcB24HEJd +OKw==
X-Gm-Message-State: AOAM533Gyx6u71/ZLZkgp14o+NGL7KUM4lKjiRWn6V0BPK1jS87t+sED H9GlIYSCaS91odQecrAq7VNmJeVtst2tPvqXkH0CvA==
X-Google-Smtp-Source: ABdhPJy9NWmD+j/Qs+48u1BKwGa9cPwXgtofGTUPb37uOPXHTuGy7jyi/k6L3ywTXZVUNpgswtsykmIQwAgLi8y+ptk=
X-Received: by 2002:adf:cd86:: with SMTP id q6mr19393958wrj.422.1627941715361; Mon, 02 Aug 2021 15:01:55 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <> <> <> <> <> <> <> <> <> <> <>
In-Reply-To: <>
From: Yuchung Cheng <>
Date: Mon, 2 Aug 2021 15:01:18 -0700
Message-ID: <>
To: Mark Allman <>
Cc: Vidhi Goel <>, Neal Cardwell <>, Extensions <>
Content-Type: multipart/alternative; boundary="000000000000a91d9505c89ab7b7"
Archived-At: <>
Subject: Re: [tcpm] =?utf-8?q?Linux_doesn=E2=80=99t_implement_RFC3465?=
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 02 Aug 2021 22:02:08 -0000

The choice of L=10 is fairly arbitrary -- just like IW10. But what's the
show-stopper we should be concerned about, in the big picture of having an
interoperable and performant Internet.

The fact is that Linux CC has long moved to infinite L since 2031, with or
without pacing, for slow-start and congestion avoidance. Windows is using 8
for hystart++. We can go through 100 rounds of what's a more scientific L
with more <big-CDN> experiments. In the end a bigger L causes higher burst
and risks more losses and queues. That number and stats would continue to

Many people have 100Mbps broadband, and stretched ACKs reaching a degree of
hundreds MTUs due to various bunching to improve performance. ABC L=2 is
really an inappropriate outdated limit.

On Mon, Aug 2, 2021 at 11:03 AM Mark Allman <> wrote:

> >  “This document RECOMMENDS using mechanisms like Pacing to control
> >  how many bytes are sent to the network at a point of time. But if
> >  it is not possible to implement pacing, an implementation MAY
> >  implicitly pace their traffic by applying a limit L to the
> >  increase in congestion window per ACK during slow start. In
> >  modern stacks, acknowledgments are aggregated for various reason,
> >  CPU optimization, reducing network load etc. Hence it is common
> >  for a sender to receive an aggregated ACK that acknowledges more
> >  than 2 segments. For example, a stack that implements GRO could
> >  aggregate packets up to 64Kbytes or ~44 segments before passing
> >  on to the TCP layer and this would result in a single ACK to be
> >  generated by the TCP stack. Given that an initial window of 10
> >  packets in current deployments has been working fine, the draft
> >  makes a recommendation to set L=10 during slow start. This would
> >  mean that with every ACK, we are probing for a new capacity by
> >  sending 10 packets in addition to the previously discovered
> >  capacity. Implementations MAY choose to set a lower limit if they
> >  believe an increase of 10 is too aggressive."
> >
> > Does this sound like what we would like to say?
> Not really, IMO.  I think a few things here ...
>   - I agree that if pacing is in play that we don't need to worry
>     about an L.
>   - I think the above L=10 reasoning is at best pretty weak.  Just
>     because IW=10 works OK once / connection does not mean
>     continually sending "10 more" will work out OK.  It may.  It may
>     not.  But, the above sort of coupling between IW-10 and L=10
>     seems highly tenuous without any sort of data.
>   - The real issue with picking a number is that it is so hard to
>     reason about because the behavior All Depends.  E.g., consider
>     something like IW=10.  We know that will allow 10 or fewer
>     segments to be pumped into the network when the connection
>     starts.  That's pretty easy to reason about / understand.  But,
>     with L=10 we might have bursts anywhere from 2 packets to
>     cwnd+10 packets on every ACK---depending on how the ACKs are
>     stretched.  And, cwnd isn't a constant.  So, if the idea is to
>     somehow limit bursting then sometimes we're limiting to X and
>     others to Y and still others to Z.  It's an inconsistent mess.
>     Making L something arbitrary without evidence seems like a bad
>     path to me.
>   - Of course, by making L=10 a MAY we're effectively saying "no L,
>     anywhere" anyways.  If we're going to define an L it should be a
>     SHOULD unless there is pacing.
>   - Somewhat related to the above, it isn't clear what "10 more" is
>     more than in qualitative terms, as well.  Say an ACK rolls in
>     that covers 10 packets.  How were those packets sent?  Were they
>     sent a back-to-back burst so "10 more" is in fact a back-to-back
>     burst that is 10 more than previously (or, 2x)?  Or, did we send
>     those 10 packets in 5 little bursts of 2 packets each so that
>     "10 more" is actually 18 more than the previous burst size---a
>     10x increase?  Of course, it could be BOTH for the same ACK!
>     I.e., the segments were sent 2 at a time and aggregated
>     somewhere in the middle.  This lack of clarity again makes the
>     choice of L feel pretty arbitrary.
> I agree we can elide L if pacing is in place.  But, twiddling with L
> by feel is crude and will produce an inconsistent approach to bursts
> that doesn't seem to me to be particularly helpful because we don't
> really grok the implications.
> allman