[tcpm] TARR - the two options for R

Jonathan Morton <chromatix99@gmail.com> Wed, 30 March 2022 19:17 UTC

Return-Path: <chromatix99@gmail.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 1A2AE3A02BB for <tcpm@ietfa.amsl.com>; Wed, 30 Mar 2022 12:17:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.859
X-Spam-Status: No, score=-1.859 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id cpJJUFmVTOgN for <tcpm@ietfa.amsl.com>; Wed, 30 Mar 2022 12:17:46 -0700 (PDT)
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [IPv6:2a00:1450:4864:20::135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BEB543A0112 for <tcpm@ietf.org>; Wed, 30 Mar 2022 12:17:45 -0700 (PDT)
Received: by mail-lf1-x135.google.com with SMTP id m3so37526291lfj.11 for <tcpm@ietf.org>; Wed, 30 Mar 2022 12:17:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:content-transfer-encoding:mime-version:subject:message-id:date :to; bh=s9OacQPAhdwqBMHt8uz2eyDqZV2bC7v7iZrfIP09TLk=; b=p3eyJ4CQ2WAHpDiSJuTcp7FQCZceYuqKSUBqMe9nydCq5hE9qXMohNPUGsIGmp8tVf o8/7277p25GY9RW0flqkUpX9YdyMHTccTdooTk8N+3iVPrCzT1L42L4Tn5LoMq260skT /qgTNczssZQxRr5ooN24fmzTbpEfjrCWQ0BQlYT1I4tfGJXEMk7Q+lv8Xo+lUeb7ozVc o7bxYwHQn3VpZR8bVHofpuVCsHq4V1BPP/yNF+VHv3I7rs9u3pD9HtBOUkFFsnCUhoU6 tvZxBf4/QPHkTpOGxkuZ00L7oQkf7PDpyIU/jWLJ62s5dYPPvtuD83o3wWHDy6Y1SX1O Nw3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:to; bh=s9OacQPAhdwqBMHt8uz2eyDqZV2bC7v7iZrfIP09TLk=; b=AlSJFaRHKzZpOJ+HvRAh9Kz2KrxXbsGcnUe3T1/ujTVcabOBYLMnEJQnFyMC2zJjv+ QwKZGo0jA5u1RL7tMF4kBUTK6ZrSSG/cnOrlCle2hGe05I/BhbI/Z1BxAjMeRd57HOLw kO9VQVw9kTs1kv8PS4v7SDFuc6G5vC/RbsE0/YqtdUWARkFyqlQSim4bfqC4on3tahTo PKjVpKgSF26/m8bjkV6byQL6UqMp/tZepd2M8RRE4qa7dsHHYFcaHDBFmyOuL/EKkSd9 BykwyVWK9RTCMWwg42IRV/0XnWECqAqqp62P3Q9wSIm3fwxX3PQv+7bNN90ImFiepGbn qDwA==
X-Gm-Message-State: AOAM531kyYDypuZP0h73ouFuHy5imRd2qEZGJiPMiGEQo+SnFt8J+2P1 j+b49U7HpILpiX4i8sKRiV8TkCgRTYQ=
X-Google-Smtp-Source: ABdhPJxnCsJ0MtFQsME/4sEDji1ge+YmqTfYlKwJXLh+iW18D3Ou2FaHIVexTLmFfZTPzvXmU1WoNw==
X-Received: by 2002:a05:6512:12d4:b0:44a:2794:b3dd with SMTP id p20-20020a05651212d400b0044a2794b3ddmr8274481lfg.206.1648667863612; Wed, 30 Mar 2022 12:17:43 -0700 (PDT)
Received: from smtpclient.apple (178-55-94-146.bb.dnainternet.fi. []) by smtp.gmail.com with ESMTPSA id r9-20020ac25f89000000b0044a1008c5d7sm2423974lfe.234.2022. for <tcpm@ietf.org> (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Mar 2022 12:17:43 -0700 (PDT)
From: Jonathan Morton <chromatix99@gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.\))
Message-Id: <8D68AD5E-F8F1-47D9-900C-B15C539D6C25@gmail.com>
Date: Wed, 30 Mar 2022 22:17:41 +0300
To: tcpm IETF list <tcpm@ietf.org>
X-Mailer: Apple Mail (2.3654.
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/o8sfqnFsZjRZXakJzNIsm897JPI>
Subject: [tcpm] TARR - the two options for R
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Mar 2022 19:18:00 -0000

During IETF-113, it was asked which of the two encoding options for the R value should be chosen, and I suggested that there were arguments in favour of both, with discussion to follow here.  The text in -04 is as follows:

> R: The size of this field is 7 bits. The field carries the ACK rate requested by the sender. The minimum value of R is 1.
> Note: there are currently two options being considered regarding the semantics of the R field:
> OPTION 1: the R field corresponds to the binary encoding of the requested ACK rate. The maximum value of R is 127. A receiver MUST ignore an R field with all bits set to zero. (TO-DO: if OPTION 1 is selected, see how to handle all bits of R being equal to zero.)
> OPTION 2: the R field is composed of two subfields: the 5 leftmost bits represent a mantissa (m) and the 2 rightmost bits represent an exponent (e). The value of the requested ACK rate is obtained as R = (m+1)*2^(2*e). The maximum value of R is 2048.

It was further suggested that in Option 1, the value could be encoded as offset-1, so that (for the 7-bit field described) a range of 1-128 inclusive was available, and an encoding of all-zeroes would be interpreted as R=1.  I'll assume this suggestion is adopted for this discussion.

The main argument in favour of Option 1 is that it is very easy for the sender to encode and the receiver to decode, reducing the likelihood of implementation errors.  The maximum R value of 128 already represents a 64-fold improvement in the status quo, with respect to the number of acknowledgement packets generated by a conforming receiver.  It also allows for the opposite case of disabling Delayed Acks, for those situations where it's needed or desirable, in the simplest reasonable manner.

Option 2 is slightly more complex to encode and decode, but allows for a wider range of R values to be represented, still as low as 1, but now as high as 2048, with granularity decreasing as R increases beyond 32.  A question was raised as to whether an R value this high was reasonable, or whether the lower cap of Option 1 would be sufficient for current and future needs.

In today's Internet, link speeds of 1Gbps are commonplace (especially on LAN but also increasingly on WAN), and under suitable conditions even a single TCP flow could use it all - with current research going into ensuring that these "suitable conditions" are also normally encountered.  I will therefore use this speed as my benchmark for illustrative purposes, with the understanding that many links are slower today, and conversely will probably be faster still in the future.  For the latter case, I will also consider the theoretical limits of a TCP connection.

The standard Ethernet frame size limits an IP packet to 1500 bytes on most networks.  At 1Gbps (and neglecting various overheads), this allows for 83333 data packets per second at the maximum frame size.  Under standard Delayed Ack rules (ie. R=2), this results in 41667 ack packets per second travelling the other way, or almost one every 25 microseconds.  This imposes a significant load on the network's ability to handle individual packets, and also imposes a significant processing burden on both of the TCP endpoints.

On a shared-medium link, many of these ack packets will be enqueued awaiting access to the medium, which could take several milliseconds.  This could result in as many as a thousand ack packets being released in a line-rate burst.  Assuming ack decimation does not occur, this burst will tend to persist all the way back to the sender.  Typically, all but the final ack in the burst yields only redundant information, which will be superseded before the next data segment can be prepared and sent.  Ultimately, it is a waste of energy to even emit those redundant acks in the first place.

With R=128, the density of ack packets under the same conditions reduces to 651 per second, or about one every millisecond and a half.  This is clearly a major and worthwhile improvement in efficiency.  If we limited the discussion to Ethernet LANs, an R value even this high would not be required at 1Gbps, since the RTT is typically below 1ms in such conditions.  However, Internet paths are typically much longer, and R=128 is clearly justified as soon as a 1Gbps TCP flow goes beyond the LAN, with the RTT exceeding 6ms (using the rule of thumb that at least 4 acks per RTT are needed to assure smooth delivery).

But the question is whether R values *exceeding* 128 are justifiable.  At a "typical" Internet RTT of 100ms, we would like to have an ack every 25ms due to the rule of thumb, and this is achieved by R in the vicinity of 1000.  Longer RTTs (eg. Eastern Europe to US Pacific Coast) might sometimes justify even higher R values.  Since TARR's option 2 has a maximum representable R of 2048, it is a good match for this scenario.

We can also sanity-check R=2048 by comparing it to an extreme case in the IP and TCP specifications.  IP packets (and thus TCP segments) can be as large as 64KB (2^16), and the TCP receive window can be as large as 1GB (2^30).  In this case, the window is approximately 2^14 (16K) segments.  Assuming the congestion window actually grows to fill the receive window (which would imply an LFN par excellence), R=2048 would produce 8 acks per RTT, which is a reasonable correspondence to the rule of thumb.  In these extreme conditions, that would be one ack per 128MB of data.

The LFN in the room is that stretch-acks are known to promote burstiness in TCP senders, and *either* option - R=128 or R=2048 - would make stretch-acks the norm rather than the exception.  Hence TARR should RECOMMEND that conforming senders use a burst mitigation mechanism such as TCP Pacing, commensurate with the R values emitted.

 - Jonathan Morton