Re: [tcpm] Seeking WG opinions on ACKing ACKs with good cause (was: Possible error in accurate-ecn)

Jonathan Morton <> Fri, 19 March 2021 04:28 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 4221D3A102E for <>; Thu, 18 Mar 2021 21:28:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.848
X-Spam-Status: No, score=-1.848 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id mNIo-XBSdeWC for <>; Thu, 18 Mar 2021 21:28:02 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 548213A102C for <>; Thu, 18 Mar 2021 21:28:02 -0700 (PDT)
Received: by with SMTP id 75so7996067lfa.2 for <>; Thu, 18 Mar 2021 21:28:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=atQZMyotYbFjoBpj/a9X0vfU8eBYCzuQfpDdfHg8HCE=; b=kY5hWTJeqmPX2CtJ9RSxSZGqtiR8Ti+ZKBoPVL/dQZ8sClVoCdUMk0G3ApaoXgy+Xs XVgCHcTCZhDGq/pyOXIYw0OWPa3RHCl9bGWNelE4+VPb9fax6YVL4wYg4xcpBXUC0y7F 8iTmADbeFlhXrkozZn7PB+c6gxWkfjXaoSHiqYX8AEZ/20pXoUMfI+U6wrltWnZXZDAX LR9SxiSTq7E1xnDzCnyIuO37Kq1ASDpno5qG4k83EPcE5CFFZe1q3lL7XayytE3+LMIF vMULL3j5SE4y2FgbN2RYCK69MvJS95UxuHp6yOfwXVgQLGowjX1mpwBDN9O9c1Xso3R2 VJWA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=atQZMyotYbFjoBpj/a9X0vfU8eBYCzuQfpDdfHg8HCE=; b=nbcuAODibxAcrBncmGFMSk3usxgf/lDqfoFdkikSZNWgQbjkA9lSRK9vtMOi/YrZ3n t4GRPIXx+u1+aqTmv5WzcixthTSVHPwx0RrwjS0WOAIg5JosNZbJyjw/Z1DC7SmgiZAy dPJzW7nidBQ2So86OlcIBXp8925N43rFRqK47+Uiv0nNJCSAGqToeef4nUlMFGEn1AgE ky9bWQslDHrqHUtOHvL0Ggbvtrd1M95qgS6uWcUSFTiKxEk3ZfO40blAA/ikJvYelt9z FxQZa+OSqYJSsjHe/vYAOjTpPh69OjIk7tZf9d6VVf5Rbj27b5BLyhJvFXj+4ntuhL7y TxpQ==
X-Gm-Message-State: AOAM530mE9SLZ2zk7HALBfesxXIuT3BYbB/xHOHCk3fGOvkXzgvNvbLB f102MkbbGzQbeZvup6a+2m4=
X-Google-Smtp-Source: ABdhPJztw9r+iPOd/Gp9bKDQm8HnsdhpfsF9mTHBst9XOR06qNMhy4faNJtHjFdmXOwsig8JgLQCag==
X-Received: by 2002:ac2:5446:: with SMTP id d6mr7072820lfn.527.1616128079259; Thu, 18 Mar 2021 21:27:59 -0700 (PDT)
Received: from jonathartonsmbp.lan ( []) by with ESMTPSA id i3sm466439lfl.281.2021. (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 18 Mar 2021 21:27:58 -0700 (PDT)
Content-Type: text/plain; charset="us-ascii"
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.7\))
From: Jonathan Morton <>
In-Reply-To: <>
Date: Fri, 19 Mar 2021 06:27:57 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
To: "Scheffenegger, Richard" <>
X-Mailer: Apple Mail (2.3445.9.7)
Archived-At: <>
Subject: Re: [tcpm] Seeking WG opinions on ACKing ACKs with good cause (was: Possible error in accurate-ecn)
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 19 Mar 2021 04:28:04 -0000

> On 18 Mar, 2021, at 10:50 pm, Scheffenegger, Richard <> wrote:
> At least I can not think of a realistic scenario, which is expected to
> be encountered in a future L4S/SCE (shallow, instanteous marking)
> environment, that would run into such a pathological setup on a regular
> basis. But perhaps there are some...

Shallow marking thresholds increase the probability of packets being marked, especially under congested conditions.  I consider it quite likely that AQMs will sometimes end up marking every packet that passes through them, even if they are only small packets such as pure acks, and it is even possible for this to occur in both directions simultaneously (on slow links, for example - or faster links so congested that the fair share of each flow is slow).

During the design of Cake, I became aware of cases where an ack stream on the reverse path could become the limiting factor for goodput on the forward path, even when many (Not-ECT) acks were being dropped by the AQM because the stream had persistently exceeded the threshold at 5ms.  That is why Cake has an ack coalescing filter.  These conditions arise even with conventional AQM and relatively high RTT; they are mainly associated with the highly asymmetric links that are commonly provisioned at the last mile.

So these conditions are not so rare as to be beneath examination, at the very least.  The conditions giving rise to the provision of Cake's ack filter would also result in high marking rates of ECT acks.  The question is whether the resulting effects are mild enough to tolerate.

Summarising the situation so far:

- It must be assumed that AQMs are capable of CE-marking 100% of ECN-Capable packets, if the AQM considers that conditions warrant it.

- ECN-Capable pure acks give rise to the need to signal the receipt of CE marks on pure acks back to their sender.

- When the receiver of a CE-marked pure ack has data to send, it can piggyback that notification on a future data segment.  This is not a problem.

- When the receiver of a CE-marked pure ack *does not* have data to send, it must either generate a pure ack to carry the notification, or store the notification for some future opportunity.

- The former choice can be described as "acking an ack", and can lead to an infinite series of ack-acking exchanges if all such acks are CE marked in the network, and no mechanism for ending the exchange is specified.

- Additionally, if the sender of an ECN-Capable pure ack subsequently has data available to send, and does so in more than one segment, a subsequent notification of a CE mark on the ack could be misinterpreted as a DupAck indicating possible loss of the data segment, combined with a CE mark on the second segment or later.

The latter two points are what we need to evaluate, and guard against to the extent deemed necessary.

One very simple method of breaking the infinite chain of notifications would be to specify that acks of acks should themselves be Not-ECT.  These will never be CE marked (modulo broken middleboxes).  Another is to require at least two outstanding "pure ack" CE marks before a pure ack can be generated as a notification; this does not preclude immediate notification of CE marks either upon or via data segments.  The latter method would guarantee an exponential decay of the volume of acks exchanged per RTT, and eventually halt them entirely.

I would recommend language specifying that one of the above methods, or some equivalently effective protection, SHOULD be implemented.

There are a number of commonly-used mechanisms to disambiguate true DupAcks from spurious ones:

- If TCP Timestamps are used, the TSecr field could be compared with the TSval attached to the previously generated pure acks and/or each of the outstanding data segments.  This would indicate that the candidate DupAck was generated prior to receipt of any of the data segments, and must therefore be an "ack of an ack".

- If SACK has been negotiated, it is reasonable to assume that a lost segment followed by a received one would be notified by SACK, and not by a SACK-less DupAck.  The latter is therefore probably an "ack of an ack".

I would recommend language stating that ECN-Capable pure acks SHOULD NOT be generated unless either TCP Timestamps or TCP SACK has been successfully negotiated on the connection, or some equivalent disambiguating information is available, AND that disambiguating information is actually used to prevent spurious retransmissions.  Since both Timestamps and SACK are in common use, that shouldn't be onerous.

However, it is also true that a spurious retransmission on occasion is probably not very harmful.

 - Jonathan Morton