Re: [tcpm] Cubic after RTO

Neal Cardwell <ncardwell@google.com> Thu, 22 February 2024 15:18 UTC

Return-Path: <ncardwell@google.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 22952C14F6E8 for <tcpm@ietfa.amsl.com>; Thu, 22 Feb 2024 07:18:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.606
X-Spam-Level:
X-Spam-Status: No, score=-17.606 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BSgc9jUABfjg for <tcpm@ietfa.amsl.com>; Thu, 22 Feb 2024 07:18:35 -0800 (PST)
Received: from mail-oi1-x22c.google.com (mail-oi1-x22c.google.com [IPv6:2607:f8b0:4864:20::22c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 38620C14EB17 for <tcpm@ietf.org>; Thu, 22 Feb 2024 07:18:35 -0800 (PST)
Received: by mail-oi1-x22c.google.com with SMTP id 5614622812f47-3c049ccb623so5332766b6e.1 for <tcpm@ietf.org>; Thu, 22 Feb 2024 07:18:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708615114; x=1709219914; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=N59TyOHPhMKBMOFXqbYEbiSodB+PXV86gAeFFvXES94=; b=rwUmBbR0/NA1SbZvh4diV8esW9yOTMZvBPouDY/bdQX4Qt0gLfYSfBBg02LV6YLAku fPZ34r40D4mR5NhrXMfxaHnNpFycBnyaX/msQrjU00HoiGs2HMeq+c6yLBt6mTpZrnnG NvAaUC8rsnhsv84xBvZ8WXkWq6UK/XL7KoQhyA5yzmBVUbCvLkQAjJ6DtJeTpoFX4Nkx Rg95eExxLj0jPs7ykp55TkIwSt9gKLygtX9Ruy1Zoz72wjBhBdgVS0yY4mjZ84uj4zl9 /XD92Ho6EHSWTKrR2eGh1Povxg6BNLUx9hec3P8IrDbUbRVlFHvd/VhHJGPssZx/CvZT GB2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708615114; x=1709219914; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=N59TyOHPhMKBMOFXqbYEbiSodB+PXV86gAeFFvXES94=; b=P6qGrl0IwFeLYJR7fttjj2MxJBk2tA34RZnfap3wIBGCdDB1PtW9RDbswfkZPhUAbJ RMtNybpQTM1YGgvirOavufoh28FpKdTC+wYWnVzwHodUKH8pWnpdsRngCHhOs9VAaaiC 15fMBzw+ndpe0C7dD8x8ep4RLYGJ3LcBVLq96QXR3ImBQF4TYagPgW+dncJ9xfuktY9z n096HMYfelLqAQBnH1dql68boica6yH7/AOauE4S/haYhVxuPUNTK8XrcktlEVNc3Bfg F8wuVZXLA0caHibVK7i89W+Iq1YO6s/gsI4hj49FBekkzS8HS8VyDwzyP3qRSV95+ta/ moJw==
X-Gm-Message-State: AOJu0YzeQSgktlb4XTW8h05+JBPoYBYv5fbTSyOTSbm6JxEUe8byhWVz Rubu6x+stPYYRTJiFPVEyeT/7lY0Dt7yjSk1XLwgixleKk7FLk96+djGypyp/2f0UkaBkZp3D+D CR+FmGjaXV1YIn4qa5n9vHJSQl9R/tGNJifkh
X-Google-Smtp-Source: AGHT+IFmhr4Rhn0tOroTMjxNaapGSKz0w+T/N3FiXQoiXHY9jBtQjjJssLlOVjGKK3tIm+CqejmO2uogBIUV3IcoDoU=
X-Received: by 2002:a05:6808:140e:b0:3c0:3a17:b9d4 with SMTP id w14-20020a056808140e00b003c03a17b9d4mr24071244oiv.36.1708615114178; Thu, 22 Feb 2024 07:18:34 -0800 (PST)
MIME-Version: 1.0
References: <efae63cc-1de2-4a20-9379-80a2067a67a5@gmx.at>
In-Reply-To: <efae63cc-1de2-4a20-9379-80a2067a67a5@gmx.at>
From: Neal Cardwell <ncardwell@google.com>
Date: Thu, 22 Feb 2024 10:18:13 -0500
Message-ID: <CADVnQynADEb5LWpCFxDa5Etxqtn9UDu4CtpMZn+_ozs=SY1ytQ@mail.gmail.com>
To: rs.ietf@gmx.at
Cc: "tcpm@ietf.org" <tcpm@ietf.org>, Vidhi Goel <vidhi_goel@apple.com>, Yuchung Cheng <ycheng@google.com>
Content-Type: multipart/alternative; boundary="000000000000f0c2940611f9f4b3"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/NaGjcCEEPy5k4dRM5Ly2u0__anc>
Subject: Re: [tcpm] Cubic after RTO
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Feb 2024 15:18:39 -0000

On Thu, Feb 22, 2024 at 9:10 AM Scheffenegger, Richard <rs.ietf@gmx.at>
wrote:

> Group,
>
> I need some help to understand where I seem to be missing something
> around RFC9438 (Cubic) and the Behavior after RTO...
>
>    > During the first congestion avoidance stage after a timeout, CUBIC
>    > increases its congestion window size using Figure 1, where t is the
>    > elapsed time since the beginning of the current congestion avoidance
>    > stage, K is set to 0, and Wmax is set to the congestion window size
>    > at the beginning of the current congestion avoidance stage. In
>    > addition, for the Reno-friendly region, West SHOULD be set to the
>    > congestion window size at the beginning of the current congestion
>    > avoidance stage.
>
> Let's assume, the RTT is around 10ms, and there is one CC event every 10
> RTTs - or 100ms.
>
> Now, with the above definition, if now a chain of RTOs happen, t_epoch
> is not reset at all, but may be many orders of magnitute in time in the
> past, compared with the "steady state" - especially when there are a
> number of subsequent RTOs with exponential backoff.
>

In the RFC and in the Linux TCP CUBIC code, t_epoch *is* reset. :-) Note
that the RFC says:

"t is the elapsed time since the beginning of the current congestion
avoidance stage"

The CUBIC curve is only used during the congestion avoidance stage, meaning
cwnd > ssthresh. And this "t is the elapsed time since the beginning of the
current congestion avoidance stage" means that basically t_epoch (where per
the RFC t_epoch is "The time in seconds at which the current congestion
avoidance stage started") is set to "now" every time that the connection
enters congestion avoidance (meaning cwnd cross above ssthresh).

That means that there is no concern with multiple RTO recoveries causing
t_epoch to be far in the past. Because t_epoch is set to "now" every time
the connection crosses into congestion avoidance and starts using the CUBIC
curve computation to set the cwnd.

That's how the Linux code works as well. Every time there's a fast recovery
or RTO event, Linux TCP CUBIC sets ca->epoch_start to 0 (code link
<https://elixir.bootlin.com/linux/v6.7/source/net/ipv4/tcp_cubic.c#L346>).
Then, when in congestion avoidance, when CUBIC seesca->epoch_start is 0 (code
link
<https://elixir.bootlin.com/linux/v6.7/source/net/ipv4/tcp_cubic.c#L235>),
it sets ca->epoch_start = tcp_jiffies32, meaning it sets t_epoch to now.

best regards,
neal




> As K is also reset, the next ACK after RTO received will result in cwnd
> to be calculated (by fig 1)
>
> Wcubic(t) = C * (t - K)^3 + Wmax
>
> With C = 0.4, K = 0, and t may be even higher than 60 sec - and not the
> "normal" 0.1 sec. This results in cwnd to be set to 0.4 * 60^3 + Wmax
> [segments] 86400 segments + the prior maximum cwnd - which doesn't sound
> correct...
>
>
> What is the linux implementation (as the baseline) actually doing after
> RTO here?
>
> Thanks,
>    Richard
>
>
>