Re: [tcpm] TARR - the two options for R

Juhamatti Kuusisaari <> Thu, 31 March 2022 07:29 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 86A273A1845 for <>; Thu, 31 Mar 2022 00:29:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.108
X-Spam-Status: No, score=-2.108 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id i8An-CoMOffa for <>; Thu, 31 Mar 2022 00:29:06 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::e2b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 653D33A0ED3 for <>; Thu, 31 Mar 2022 00:29:06 -0700 (PDT)
Received: by with SMTP id z134so20930983vsz.8 for <>; Thu, 31 Mar 2022 00:29:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=hSMwiv8T3TDWbno7QnAEbN8eBrYeyrcXw9dsp67yzRk=; b=TpytoavFDcOVHKTpXfqf7Dc4oPVG3aW54R6M7EPNrB0a7cAW471ezFvE+stiekTJTW bLbc8ooSKW7iW96tSUTMh+4z8XNym/24Oo6IUnPR3ppjRHoRzKryoFFqaeQbxTO3AmKS CQboDLi02zOkjV5cwn+YFH0UBRghyJtVOmUwhvxgj8GlGlBC/JOMUe5WlGFUBFGrbMXK lFgdEvDyelZj4/Ti8eECyV3sm50Tzyr6epAvLnC+yPvnDgF93iKrkYwPepadzMogPdp1 wIryM9atZglb9K+333f70vXobYr+yS60moVCce4v1mwHXKJtZC3iEu56S/R8rpoiT3HC kHzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=hSMwiv8T3TDWbno7QnAEbN8eBrYeyrcXw9dsp67yzRk=; b=LDONdAiBFjADFeMmKXsKNNQjSOVPJTUeTciV4xjeXQdHBpWeCDUWFYc+hj0g7wM5Wh 4cYk4qFZLDq6TcCJdUSXyfM4pDBd/ZrdzmwpBI7zm0Te+Fnz0ww+pjNJkhNoWDAdYpm6 E9jQr8TieBgWG8r7Ev3R+Gjm0yQhBV7P4dtYNN63ra2qqd3J1gNBdsSwHAiDEesbWRG+ jgXhyfSh/ZtQ5+C734kcjjney4Ckiusc4fHcO/zqAtkIbY4NFaAuD+1Lf6VjvLj52exE VKBEsEkqw3KDenQhiIH6TUscabTKN06oy7WOllaQkmdpyUEuS+FeZQGRYrlN60VfJtQn fXVQ==
X-Gm-Message-State: AOAM532vE3crRCcsYI23IBVOMvnFj8n7oZ71oJ9Rko99onj1KQ5EnPwd PZY0BjMuWVmKd1BIa//c2o4i/2RmPY4jU3MorY8=
X-Google-Smtp-Source: ABdhPJxI9Ghl7P8Yq3HSkjjh9KdIqjEuxxYXBCQyv7yR6ETCYyKanoQT43SG9gvD1uGtXRlhG4E0LkyyOLWVs1axCfQ=
X-Received: by 2002:a05:6102:5cc:b0:320:9bd2:3823 with SMTP id v12-20020a05610205cc00b003209bd23823mr21013819vsf.81.1648711744923; Thu, 31 Mar 2022 00:29:04 -0700 (PDT)
MIME-Version: 1.0
References: <>
In-Reply-To: <>
From: Juhamatti Kuusisaari <>
Date: Thu, 31 Mar 2022 10:28:52 +0300
Message-ID: <>
To: Jonathan Morton <>
Cc: tcpm IETF list <>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <>
Subject: Re: [tcpm] TARR - the two options for R
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 31 Mar 2022 07:29:09 -0000


+1 for Option 2 with a larger max value. We already have 800 Gbps
links available off-the-shelf and speeds won't go down in the future.


On Wed, 30 Mar 2022 at 22:18, Jonathan Morton <> wrote:
> During IETF-113, it was asked which of the two encoding options for the R value should be chosen, and I suggested that there were arguments in favour of both, with discussion to follow here.  The text in -04 is as follows:
> > R: The size of this field is 7 bits. The field carries the ACK rate requested by the sender. The minimum value of R is 1.
> >
> > Note: there are currently two options being considered regarding the semantics of the R field:
> >
> > OPTION 1: the R field corresponds to the binary encoding of the requested ACK rate. The maximum value of R is 127. A receiver MUST ignore an R field with all bits set to zero. (TO-DO: if OPTION 1 is selected, see how to handle all bits of R being equal to zero.)
> >
> > OPTION 2: the R field is composed of two subfields: the 5 leftmost bits represent a mantissa (m) and the 2 rightmost bits represent an exponent (e). The value of the requested ACK rate is obtained as R = (m+1)*2^(2*e). The maximum value of R is 2048.
> It was further suggested that in Option 1, the value could be encoded as offset-1, so that (for the 7-bit field described) a range of 1-128 inclusive was available, and an encoding of all-zeroes would be interpreted as R=1.  I'll assume this suggestion is adopted for this discussion.
> The main argument in favour of Option 1 is that it is very easy for the sender to encode and the receiver to decode, reducing the likelihood of implementation errors.  The maximum R value of 128 already represents a 64-fold improvement in the status quo, with respect to the number of acknowledgement packets generated by a conforming receiver.  It also allows for the opposite case of disabling Delayed Acks, for those situations where it's needed or desirable, in the simplest reasonable manner.
> Option 2 is slightly more complex to encode and decode, but allows for a wider range of R values to be represented, still as low as 1, but now as high as 2048, with granularity decreasing as R increases beyond 32.  A question was raised as to whether an R value this high was reasonable, or whether the lower cap of Option 1 would be sufficient for current and future needs.
> In today's Internet, link speeds of 1Gbps are commonplace (especially on LAN but also increasingly on WAN), and under suitable conditions even a single TCP flow could use it all - with current research going into ensuring that these "suitable conditions" are also normally encountered.  I will therefore use this speed as my benchmark for illustrative purposes, with the understanding that many links are slower today, and conversely will probably be faster still in the future.  For the latter case, I will also consider the theoretical limits of a TCP connection.
> The standard Ethernet frame size limits an IP packet to 1500 bytes on most networks.  At 1Gbps (and neglecting various overheads), this allows for 83333 data packets per second at the maximum frame size.  Under standard Delayed Ack rules (ie. R=2), this results in 41667 ack packets per second travelling the other way, or almost one every 25 microseconds.  This imposes a significant load on the network's ability to handle individual packets, and also imposes a significant processing burden on both of the TCP endpoints.
> On a shared-medium link, many of these ack packets will be enqueued awaiting access to the medium, which could take several milliseconds.  This could result in as many as a thousand ack packets being released in a line-rate burst.  Assuming ack decimation does not occur, this burst will tend to persist all the way back to the sender.  Typically, all but the final ack in the burst yields only redundant information, which will be superseded before the next data segment can be prepared and sent.  Ultimately, it is a waste of energy to even emit those redundant acks in the first place.
> With R=128, the density of ack packets under the same conditions reduces to 651 per second, or about one every millisecond and a half.  This is clearly a major and worthwhile improvement in efficiency.  If we limited the discussion to Ethernet LANs, an R value even this high would not be required at 1Gbps, since the RTT is typically below 1ms in such conditions.  However, Internet paths are typically much longer, and R=128 is clearly justified as soon as a 1Gbps TCP flow goes beyond the LAN, with the RTT exceeding 6ms (using the rule of thumb that at least 4 acks per RTT are needed to assure smooth delivery).
> But the question is whether R values *exceeding* 128 are justifiable.  At a "typical" Internet RTT of 100ms, we would like to have an ack every 25ms due to the rule of thumb, and this is achieved by R in the vicinity of 1000.  Longer RTTs (eg. Eastern Europe to US Pacific Coast) might sometimes justify even higher R values.  Since TARR's option 2 has a maximum representable R of 2048, it is a good match for this scenario.
> We can also sanity-check R=2048 by comparing it to an extreme case in the IP and TCP specifications.  IP packets (and thus TCP segments) can be as large as 64KB (2^16), and the TCP receive window can be as large as 1GB (2^30).  In this case, the window is approximately 2^14 (16K) segments.  Assuming the congestion window actually grows to fill the receive window (which would imply an LFN par excellence), R=2048 would produce 8 acks per RTT, which is a reasonable correspondence to the rule of thumb.  In these extreme conditions, that would be one ack per 128MB of data.
> The LFN in the room is that stretch-acks are known to promote burstiness in TCP senders, and *either* option - R=128 or R=2048 - would make stretch-acks the norm rather than the exception.  Hence TARR should RECOMMEND that conforming senders use a burst mitigation mechanism such as TCP Pacing, commensurate with the R values emitted.
>  - Jonathan Morton
> _______________________________________________
> tcpm mailing list