Re: [tcpm] [EXTERNAL] Re: Seeking WG opinions on ACKing ACKs with good cause

Christian Huitema <> Sun, 11 July 2021 05:38 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C09623A1EAE for <>; Sat, 10 Jul 2021 22:38:35 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.69
X-Spam-Status: No, score=-0.69 tagged_above=-999 required=5 tests=[NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id bBIlGXZLvhqf for <>; Sat, 10 Jul 2021 22:38:30 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id E00373A1EAC for <>; Sat, 10 Jul 2021 22:38:29 -0700 (PDT)
Received: from ([] by with esmtp (Exim 4.92) (envelope-from <>) id 1m2SAb-0016wb-DG for; Sun, 11 Jul 2021 07:38:26 +0200
Received: from (unknown []) by (Postfix) with ESMTPS id 4GMwfl2lqCzDPZ for <>; Sat, 10 Jul 2021 22:38:23 -0700 (PDT)
Received: from [] ( by with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from <>) id 1m2SAZ-0001PF-8t for; Sat, 10 Jul 2021 22:38:23 -0700
Received: (qmail 11763 invoked from network); 11 Jul 2021 05:38:22 -0000
Received: from unknown (HELO []) ([]) (envelope-sender <>) by (qmail-ldap-1.03) with ESMTPA for <>; 11 Jul 2021 05:38:22 -0000
To: Bob Briscoe <>, Yoshifumi Nishida <>, "" <>, Mirja Kuehlewind <>, "" <>
Cc: "" <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
From: Christian Huitema <>
Message-ID: <>
Date: Sat, 10 Jul 2021 22:38:21 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: quoted-printable
Content-Language: en-US
Authentication-Results:; auth=pass smtp.auth=
X-Spampanel-Outgoing-Class: unsure
X-Spampanel-Outgoing-Evidence: Combined (0.15)
X-Recommended-Action: accept
X-Filter-ID: Pt3MvcO5N4iKaDQ5O6lkdGlMVN6RH8bjRMzItlySaT9WLQux0N3HQm8ltz8rnu+BPUtbdvnXkggZ 3YnVId/Y5jcf0yeVQAvfjHznO7+bT5yjTC8Fov82/EJuxz8FihBPKj/EwzSHE5FGYwwjsNRPCCma DytoMWzyPlKVtFE64rTmD6wdmZPcItWbGe10hXJtXL4FsauCVkDjmcYJdU3yWp7KuHNaaKdg7iBE ZefdsNUFWKwa/wzJUjmazeC7Imcapebr0kNyYC289u5HlaNj1BQ6V51u76v35b1wNe/MvdL/hXir I7jpLA3NtNK1rbkD2+J9PgaoF8SQHto3le4zsHTaeQtlKubP6iUTjj6yPARK6buALVaA782LKxg6 vRmng8N1aLhXqdc+jC1RcnVud53D5caUhbVtvqItBqoizkEt9O20UjkwI0v+LOlw05G4BS+iyyNq bT8dUMXMJ4tUCMj6G37ZfAMLceP5aNHPt26RBupu5v1nytoNnc138GfEJRQ2qC7jjynPIHPNqSn4 QTXUjLjYWQt1/5xnQymMoPsgr/U0flMcy2Vi/IcBgY4arPaiJ1W6hAyiRC61jekdwIcXNugoOEbH RyFULpSjm7jZ1h/HfDRQ5Ig8VhPsPE8NlkBmbR1LS6Kx8w5MHqDEE4BBJaBtD32RALQCSg1oDtMl PyREjmB5EAdXVBvzPnf7DRojSVizNl0ce/s7u0P9b9Tml6eOMCV9kYYwkPx6ZsXvIUzTXkDAiiJi mGhLUFuSOTgtjQWHblEKb/bSn512w9qoLbHrX5KHDBoYpFP6SoHZcpPgEJKLbDyaC/LdLvvY/Vzw IcESbxiIxnB7tuHetfpVB9v9zY0h8asEYmbGGsJkWjQ4xyeNtxxq2TXT/AfNgWH7GTOjKdjx/QnC QSBbKRX0XF3qP/dj48psOHFCwviQxKSBCGH0S84CnKX/NUAV3jR5NeVaJQBh0uawl0Cg8nrgSS6D 7FozWS6JHKREtqdEPzY64lXv2dr2sny4a4Sj0cqzvWDlDrFILmNCVZ/264kd2x35zAiBFPp64JaI ysAWfpirH8g1GOR1IFGt5BWm
Archived-At: <>
Subject: Re: [tcpm] [EXTERNAL] Re: Seeking WG opinions on ACKing ACKs with good cause
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 11 Jul 2021 05:38:36 -0000

On 7/10/2021 2:13 PM, Bob Briscoe wrote:
> Yoshi, Richard, Mirja, tcpm list,
> We came to a good consensus in late March on the conditions to place 
> around ACKing ACKs in order to keep congestion feedback flowing, but 
> to ensure ping-pong ACKs are rapidly damped as well as avoiding false 
> DupACK detection. I have written that up in a rev of the draft, which 
> I will post separately so as not to distract from the question I have 
> here...
> ...While writing up, I realized we had glossed over a possibly more 
> important point, when we proposed to change the number in the basic 
> rule from 2 to 3, as follows:
>       An AccECN Data Receiver MUST immediately send an ACK once 'n'
>       CE marks have arrived since the previous ACK, where 'n' SHOULD
>       be 3 and MUST be in the range 3 to 6 inclusive.
> I'd like to suggest an alternative:
>       An AccECN Data Receiver MUST immediately send an ACK once 'n'
>       CE marks have arrived since the previous ACK. If there is new
>       data to acknowledge, 'n' SHOULD be 2.  If there is no new data
>       to acknowledge, 'n' SHOULD be 3 and MUST be no less than 3.
>       In either case, 'n' MUST be no greater than 6.
> Rationale: The data ACKs case shouldn't be compromised by the ACKs of 
> ACKs case.
> When we were originally only thinking of the data ACKs case (before we 
> realized this rule might trigger ACKs of ACKs), we recommended n=2  
> because it ensures congestion information is kept fresh, and during 
> runs of congestion it generates more ACKs, which might make it more 
> likely that at least one ACK survives some types of coalescing before 
> the ACE field wraps. Also 2 is the default for the equivalent 
> repetition of DCTCP feedback.
> Then, we noticed the ACKs of ACKs case, and we wanted to damp any ACK 
> ping-pong, so we recommended 3. Without really discussing the pros and 
> cons, we extended 3 to all cases (both data ACKs and ACKs of ACKs). I 
> suspect the main reason was that it is just simpler to have a single 
> rule. But these two cases are likely to be controlled from different 
> parts of the code anyway.
> I've also realized that, in the data ACKs case, it was wrong to set a 
> lower bound on 'n', let alone a /mandatory/ lower bound. There might 
> be scenarios where it makes sense to trigger an ACK every CE mark; we 
> have no reason to stop implementers doing that if they want. A lower 
> bound could even be perversely read to mean that you're not allowed to 
> immediately ACK data until you've received 'n' CE marks.
> Thoughts? 

I am looking at this from the QUIC point of view. In general, frequent 
ACKs provide better control, but there are two points of tension: 
managing low bandwidth return paths; and, reducing the impact of ACK on 

There are a number of scenarios in which the return path used by ACKs 
has a much lower bandwidth than the data path. If ACKs are two frequent, 
the return path can become congested. ACKing every two or 3 packets is 
only sustainable if the return path bandwidth is not too small compared 
to the data path -- 1/10th is generally sustainable, but lower that 
1/20th and the ACKs have to be spaced. The alternative to spacing would 
be increased RTT due to congestion on the return path, and random losses 
of ACKs. Of course, even with ACK spacing, there are limits to 
asymmetry: control becomes loose if the transport does not receive 
multiple ACKs per RTT.

ACK frequency has been found a key factor in the performance of QUIC 
implementations. For example, implementations may achieve 5Gbps with 
spaced ACK, but the performance drops to a few 100 Mbps if we insist on 
an ACK for 2 packets. High performance requires sending packets in 
batches, using API like UDP GSO, and performance seems best when the 
frequency of ACKs more or less matches the frequency of batches.

-- Christian Huitema