Re: [tcpm] [EXTERNAL] Re: Seeking WG opinions on ACKing ACKs with good cause

Neal Cardwell <> Sun, 11 July 2021 18:28 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id E5BA53A189A for <>; Sun, 11 Jul 2021 11:28:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -16.197
X-Spam-Status: No, score=-16.197 tagged_above=-999 required=5 tests=[DKIMWL_WL_MED=-0.499, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id fp9arDgzSRFg for <>; Sun, 11 Jul 2021 11:28:47 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::92f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 33C6E3A1899 for <>; Sun, 11 Jul 2021 11:28:47 -0700 (PDT)
Received: by with SMTP id r9so6150771ual.7 for <>; Sun, 11 Jul 2021 11:28:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=XDESUxuxmcXsY8ZalRAEDtUCWgY8jSGl5t5PEWUp1zw=; b=nXSncw0ZUr+xzEXy5Oprzf1pCYu/endIeKUAJsXR+JKIFnP+/qoA0toKQv1OofZYXM qkm6AJ1KmhUMOXOLq7l6SwUitLGrMUyrickbtCUl3yDegDKUBBSCJZshpVSm+cJ2Pux6 qivDWf9T4rXqmkClJhKqnX6MPosXDTz9JzMdlO1Nw76KIirdu4QDAlNJDDHBzMCWfU5A N++gDVGQAssBuEUSEOCPZXHSMsZP6aOV5qoVcTXjvYz0v22n4ivomzvHH4keQ7vNxs9r a9mfoPinR4VPUwJIxGyr6QxfakmT94xnPiBf2MysOBAiw4sTMmT0NAjutRWzSbesqYq2 LfOQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=XDESUxuxmcXsY8ZalRAEDtUCWgY8jSGl5t5PEWUp1zw=; b=Vf4+aYUNyflpZxdO9SrjMBjdXtcLD8Evt5mjm9hNnKez075jSCglg5FpEaQg+udj1O H5OKFGKLZJraV5P1D+DDu3ad4M4g7mzg7oV1FCjjy68n3BUSkaH2Xko4gzhNWESIFPaI 9d8Sac7DEqQlkTlxxsKIiTy0+k6YUpWourjvjGhJZH7ZzTWCf3IxnsizxYJ4AIK2hp+j Ze1lMy4a69OlmAzHdwFqfLylew/KQ9Cn48oHtb2n4GPNvpXut6BCYtoLrwYaPl8PPEHS 1zqdruP0iXLTpsJAxgc2oCeGX6zrIkHi0g6ovNJLmrsE9xpOCiHuir8CAW0gj1jO7a7n 8woA==
X-Gm-Message-State: AOAM531vi2qwcFdJGn8fF1qjiANTF+ziav3XOJhMcTsNG4Lue0E9k9wi ivmSf/98negOFSHujBxcdRqO5gCT7bAgpdKZnNVna0zIumHbzw==
X-Google-Smtp-Source: ABdhPJwbQMFgSgE/wEzG7qbRrzICPLp9fWen4RG5aTShEIKqmW0E3LZoQeHCE1DrlABxzH5pxej95+mwuT+clw4zsXc=
X-Received: by 2002:ab0:25c5:: with SMTP id y5mr28103653uan.142.1626028125317; Sun, 11 Jul 2021 11:28:45 -0700 (PDT)
MIME-Version: 1.0
References: <> <>
In-Reply-To: <>
From: Neal Cardwell <>
Date: Sun, 11 Jul 2021 14:28:28 -0400
Message-ID: <>
To: Christian Huitema <>
Cc: Bob Briscoe <>,, Mirja Kuehlewind <>,, Yuchung Cheng <>
Content-Type: multipart/alternative; boundary="000000000000ce8ea905c6dd2c31"
Archived-At: <>
Subject: Re: [tcpm] [EXTERNAL] Re: Seeking WG opinions on ACKing ACKs with good cause
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 11 Jul 2021 18:28:53 -0000

On Sun, Jul 11, 2021 at 11:55 AM Christian Huitema <>

> > On Jul 11, 2021, at 4:42 AM, Bob Briscoe <> wrote:

> > The implementation will have its own ACK ratio - that's out of scope of
> AccECN, except to set this max of 6 CEs, which is to mitigate wrap of the
> 3-bit counter of CE-marks (which in the worst case of 100% marking in the
> data direction could then induce 1 ACK per 6 packets). This shouldn't limit
> forward performance, because it only increases the reverse ACK rate if
> there is heavy congestion in the forward path, when the Data Sender should
> be reducing the forward rate anyway.
> 1 ACK per 6 packets would cause performance issues on high speed links.
> Common setup is "4 to 8 ACK per RTT", which means intervals much larger
> than 6 packets.

I share this concern about the AccECN requirement of one ACK per 6
CE-marked data segments potentially causing performance issues with
high-speed links. Today, with most high-speed last-hop link technologies
(wifi, cellular, Ethernet) TCP receivers often receive (from lower layers
of the hardware and software networking stack) large aggregates  (often up
to 64KBytes or around 44 packets), and generate a single ACK for that
aggregate. AFAICT changing this scenario to require 1 packet per 6
CE-marked data segments means that during CE-marked scenarios receiving
such an aggregate would require ceil(44/6) = 8 ACKs, increasing the number
of ACKs by up to 8x.

I have two main concerns in that scenario:

(1) CPU load: This seems like it would impose much higher CPU load on the
data receiver, at just the moment when the receiver may already be under
stress due to receiving data near the maximum rate of the link.

(2) Congestion and ACK loss: This requirement for generating up to 8 ACKs
per aggregate seems like it is likely to produce a tight burst of 8 ACKs,
which is going to increase congestion and increase the odds of losing at
least one ACK, which is going to cause accuracy problems for "Accurate"
ECN. :-)

I understand that this proposed "ACK per 6 CE-marked data segments" rule is
necessary to avoid issues with the 3-bit ACE field wrapping. So IMHO this
is one of the good arguments against including the ACE field in the AccECN

The other main concerns I have with the ACE field are:

o Complexity

o Redundancy with respect to the AccECN counter options

o Potential problems with middleboxes

o Known problems with NICs and drivers (based on Ilpo's nice talk, "Accurate
ECN Linux Implementation: Experiences and Challenges", at the April 2020
TCPM interim meeting)

I realize that if AccECN does not have the ACE field feature, then AccECN
and TCP L4S will not be usable on paths with middleboxes that strip the AccECN
counter options. But IMHO living without the ACE field is preferable. IMHO
it's acceptable to say that L4S can only be used with (a) QUIC, or (b) TCP
connections where no middleboxes are stripping AccECN options.