Re: [aqm] [tcpm] TCP ACK Suppression

Jonathan Morton <> Sat, 10 October 2015 23:08 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 267BC1A1F73; Sat, 10 Oct 2015 16:08:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.75
X-Spam-Status: No, score=-1.75 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id aj_wLdg6j4lf; Sat, 10 Oct 2015 16:08:22 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4010:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 4050A1A1B0C; Sat, 10 Oct 2015 16:08:22 -0700 (PDT)
Received: by lbwr8 with SMTP id r8so111713294lbw.2; Sat, 10 Oct 2015 16:08:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=fbjf+u8KEnMkTII6HjJGLaLNyjuKbRAIZeaXT5VeIfM=; b=bcVNChJSjPr4ekGOs0M2gcCenaTXCMACKp9BYpUTgxfB0HkPFBRS4c4js751d7bcQZ hcYn++PRtLcQOS38IMvhyMCI6wdguwQhyvYU/w6XbeDKluWNc5DlXLMZrrXt37l2AZpQ jO59o26tlFn2kyfvwxd/odK8NjvqVcxF5bejme9Toe4RfiJY+bWCq++k+JDZNolR0XTv tbf3CMSbfMWJIiTl3Jfoa5nzkOqei7jgS3ODwlJyvbLsRdMJbM9sNMycPETLVaMEFtGD 7v/3pmzWqPj3Ub1EoCWRtduq1luv0k0XK+HbZnYe5Ik0KqOb+OyORbEssib1XbJpOpsF cNMA==
X-Received: by with SMTP id ad2mr9185833lbc.2.1444518500241; Sat, 10 Oct 2015 16:08:20 -0700 (PDT)
Received: from ( []) by with ESMTPSA id as6sm1498257lbc.16.2015. (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 10 Oct 2015 16:08:19 -0700 (PDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
From: Jonathan Morton <>
In-Reply-To: <>
Date: Sun, 11 Oct 2015 02:08:07 +0300
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
To: David Lang <>
X-Mailer: Apple Mail (2.2104)
Archived-At: <>
Cc: "" <>, Joe Touch <>, "" <>, "LAUTENSCHLAEGER, Wolfram \(Wolfram\)" <>, Greg White <>, "" <>
Subject: Re: [aqm] [tcpm] TCP ACK Suppression
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 10 Oct 2015 23:08:24 -0000

> On 11 Oct, 2015, at 01:28, David Lang <> wrote:
>> So even if I could wave a magic wand and deploy this on every router, it would only have an effect in a few places. Endpoint links where the available bandwidth drastically changes are not the exclusive list of palces that this would effect, but I think it's pretty close.
> Also, since it costs CPU to go through the queue looking for ACK packets to try and combine, it is a feature unlikly to be enbled on high speed devices. It would probably only be enabled when the savings in the reduced number of ACKs that would be transmitted is significant enough to be worth the effort. Endpoint devices for home users where there is probably also a highly asymmetric link are a very clear win. Core routers are almost never going to be a win.

In the case of wifi, aggregation and huge per-TXOP overheads mean that the cost of sending multiple acks instead of just one is negligible.  It’s simply not worth performing any sort of compression or consolidation except for (losslessly reversible) aggregation, because the gains pale into insignificance next to that unavoidable overhead.

The question is more arguable in the case of DOCSIS, where the cost of setting up a TXOP is in latency and temporal quantisation rather than channel overheads.  However, acks are individually really small compared to data packets, so the benefit of dropping each one is correspondingly small.  I do think it’s important to avoid dropping dupacks that might be associated with congestion or loss signalling, because that slows down the sender’s response to these relatively infrequent but significant events.

Both of the above situations are independent of reverse-channel congestion.

One of our many bufferbloat tests involves a 10:1 asymmetric link carrying a single downstream TCP flow, fifty upstream TCP flows, and a small number of latency-measuring (UDP and ICMP) flows.  Different queue types cope with that workload in very different ways:

- Dumb FIFOs and “pure” flow isolators (like SFQ) don’t drop many acks; usually they drop data packets instead, since they’re bigger and thus more likely to overflow the upstream queue.  This results in the downstream flow being starved via ack-clocking (flow isolators being more predictable in this respect).

- “Pure” AQMs without flow isolation tend not to care whether the packets they’re shooting at are acks or data packets.  Dropped acks  save much less queue space than dropped data packets, so the AQM tends to tune its drop rate so that the upstream data flows back off sufficiently to allow almost all of the acks through.  This permits full downstream bandwidth, but the acks consume a significant proportion of the uplink, which is then unavailable for the upstream flows.

- Flow-isolating AQMs (like fq_codel and cake) give the ack flow an equal share of the bandwidth with each of the uplink data flows, and are able to give each flow an independent drop rate.  In practice, the (responsive) data flows get a modest drop rate, while the ack flow (being antiresponsive, since thinning the acks increases the downstream rate and thus the incoming ack rate) ends up with a very high drop rate.  This demonstrably works better with more perfect flow isolation (fq_codel suffers from significant hash collisions in this scenario, cake does not); it also exercises an edge case in codel when the drop rate rises to a particularly high value (without care, it can wrap around, resulting in a wildly fluctuating downstream rate).

Cake takes care to handle this extreme scenario sensibly.  It effectively performs ack thinning when the uplink is congested, without actually being aware of what an ack *is* - it’s just another packet.  DupACKs are not a problem, since it doesn’t spread the surviving packets between distinct TXOPs as DOCSIS modems apparently can; indeed, due to acks’ small size compared to the DRR quantum, the requisite three DupACKs will quite likely still go out back-to-back, even if they weren’t consecutively originated.  None of this required special handling - it just falls out of a well-engineered, protocol-agnostic AQM scheme.

The important point, however, is that Cake *only* performs ack thinning when the link is congested.  If there’s sufficient bandwidth to send all the acks, it sends them all.

 - Jonathan Morton