Re: [aqm] TCP ACK Suppression

David Lang <> Sat, 10 October 2015 21:15 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 291C91AC40B for <>; Sat, 10 Oct 2015 14:15:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.91
X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id K2LqQGM3Scy0 for <>; Sat, 10 Oct 2015 14:15:30 -0700 (PDT)
Received: from ( []) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 988771AC3FE for <>; Sat, 10 Oct 2015 14:15:30 -0700 (PDT)
Received: from ( []) by (8.13.4/8.13.4/Debian-3) with ESMTP id t9ALFSKa032017; Sat, 10 Oct 2015 14:15:28 -0700
Date: Sat, 10 Oct 2015 14:15:28 -0700 (PDT)
From: David Lang <>
To: Simon Barber <>
In-Reply-To: <>
Message-ID: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <>
User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Archived-At: <>
Subject: Re: [aqm] TCP ACK Suppression
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 10 Oct 2015 21:15:34 -0000

Ok, I think we hve a couple different situations here. RFC3449 section 5.2.1 
covers several special signals that multile acks cover that should not be 

Then what I was understanding from the comment below was a way to mitigate the 
impact of the consolodated ack getting dropped by the network (since one lost 
ack out of a train of them has very little impact, but if they are combined into 
one stretch ack and that ack gets dropped, it's the equivalent of the entire 
trin of acks getting lost)

So here's some thoughts for people.

with the existing signalling (double or triple acks), if the string of acks 
happens to land exactly on the edge of a transmit window, it may get split 
across multiple windows, at which point the signal can be lost. This cannot be 
any worse than if one of the ack packets ws dropped by the network anyway, but 
how bad is it?

Also, if the stretched ack packet gets duplicated into the next transmit window 
to cover the possibility of it getting lost by the network and it both doesn't 
get lost and there is no furhter ack traffic before the next transmit window, 
this then will generate a exact duplicate ack. What harm cn this do? I believe 
that there are already conditions where various failure modes can cause packets 
to be duplicated already, so its not an impossible situation today. I also think 
that it would be fairly rare to trigger this as it would requie that there be 
multiple acks in the queue one transmit window and none for the next transmit 
window, but overall this would probably increase the odds of duplicate ack 
packets arriving.

David Lang

On Sat, 10 Oct 2015, Simon Barber wrote:

> I'm talking specifically about duplicate ACKs - same ACK, same contents - 
> signalling that the receiver has seen a hole. These should be communicated as 
> quickly as possible to reduce head of line blocking and application latency - 
> hence the TCP spec that you should stop delaying your ACKs in this state.
> Simon
> On 10/8/2015 9:08 PM, David Lang wrote:
>> Well, the only situation we are talking about is when an AQM function finds 
>> that there are multiple acks for one TCP session in it's queue. The dispute 
>> is if these acks are can be considered 'duplicates' (since the later ones 
>> will ack data for the earlier ones) and so can be 'deduped' or if there is 
>> a significant amount of value in sending every ack through anyway.
>> David Lang
>> On Thu, 8 Oct 2015, Simon Barber wrote:
>>> I like this suggestion - but it's only needed in the case where there are 
>>> dup acks.
>>> Simon
>>> On 10/8/2015 7:09 PM, David Collier-Brown wrote:
>>>> One short interjection: send at least two acks, separated by a time based 
>>>> on available packets and/or average noise-burst durations on links with 
>>>> noise problems...
>>>> On 08/10/15 08:04 PM, David Lang wrote:
>>>>> On Thu, 8 Oct 2015, Joe Touch wrote:
>>>>>> On 10/8/2015 3:29 PM, David Lang wrote:
>>>>>>> On Thu, 8 Oct 2015, Joe Touch wrote:
>>>>>>>> On 10/8/2015 2:31 PM, David Lang wrote:
>>>>>>>>> On Thu, 8 Oct 2015, Joe Touch wrote:
>>>>>>>>>> On 10/7/2015 12:42 AM, LAUTENSCHLAEGER, Wolfram (Wolfram) wrote:
>>>>>>>>>> ...
>>>>>>>>>>> Is this topic addressed in some RFC already?
>>>>>>>>>> It's a direct violation of RFC793, which expects one ACK for every 
>>>>>>>>>> two
>>>>>>>>>> segments:
>>>>>>>>>> 4.2 Generating Acknowledgments
>>>>>>>>>>   The delayed ACK algorithm specified in [Bra89] SHOULD be used by 
>>>>>>>>>> a
>>>>>>>>>>   TCP receiver.  When used, a TCP receiver MUST NOT excessively 
>>>>>>>>>> delay
>>>>>>>>>>   acknowledgments.  Specifically, an ACK SHOULD be generated for at
>>>>>>>>>>   least every second full-sized segment, and MUST be generated 
>>>>>>>>>> within
>>>>>>>>>>   500 ms of the arrival of the first unacknowledged packet.
>>>>>>>>> actually, this is only a violation of the SHOULD section, not the 
>>>>>>>>> MUST
>>>>>>>>> section.
>>>>>>>> When you violate a SHOULD, you need to have a good reason that 
>>>>>>>> applies
>>>>>>>> in a limited subset of cases.
>>>>>>>> "it benefits me" isn't one of them, otherwise the SHOULD would 
>>>>>>>> *always*
>>>>>>>> apply.
>>>>>>>>> And if the Ack packets are going to arrive at wire-speed anyway (due 
>>>>>>>>> to
>>>>>>>>> other causes), is there really an advantage to having 32 ack packets
>>>>>>>>> arriving one after the other instead of making it so that the first 
>>>>>>>>> ack
>>>>>>>>> packet (which arrives at the same time) can ack everything?
>>>>>>>> If the first ACK confirms everything, you're giving the endpoint a 
>>>>>>>> false
>>>>>>>> sense of how fast the data was received. This is valid only if the
>>>>>>>> *last* ACK is the only one you retain, but then you'll increase 
>>>>>>>> delay.
>>>>>>> why does it give the server a false sense of how fast the data was
>>>>>>> received? the packets don't have timestamps that the server can trust,
>>>>>>> they are just packets arriving.
>>>>>> Well, the only reason we can no longer trust them is that an
>>>>>> intermediate device has tampered with them.
>>>>> no, you could not trust any timestamps in the packets even if nothing 
>>>>> changes the packets between endpoints.
>>>>>> See, this is the problem - the DOCSIS modem wants to do what *it* 
>>>>>> wants,
>>>>>> assuming everyone else plays by the rules, but it doesn't care whether
>>>>>> it violates the assumptions other parties are making.
>>>>>> That's an example of "tragedy of the commons".
>>>>>>> And if the server concludes something
>>>>>>> different from 32 packets arriving, each acking 2 packet, but all
>>>>>>> arriving one after the other at it's wire speed (let's say it's a slow
>>>>>>> network, only Gig-E) compared to a single packet arriving that acks 64
>>>>>>> packets of data at once, it's doing something very strange and making
>>>>>>> assumptions about how the network works that are invalid.
>>>>>> Says who? The RFCs say that this assumption SHOULD be reasonable.
>>>>>>>> Unless you know that the endpoint supports ABC and pacing, yes, 
>>>>>>>> there's
>>>>>>>> a very distinct advantage to getting 32 ACKs rather than 1. It also
>>>>>>>> helps with better accuracy on the RTT calculation, which is based on
>>>>>>>> sampling (and you've killed 97% of the samples).
>>>>>>> the 97% of the samples that I've killed would be producing invalid 
>>>>>>> data
>>>>>>> for your calculation because they were delayed in returning.
>>>>>> Why do you think that is invalid data? That's an accurate measure of 
>>>>>> the
>>>>>> return path of the ACK stream.
>>>>> so how do you sanely conclude anything from 32 ack packets arriving at 
>>>>> wire speed back-to-back?
>>>>>> ...
>>>>>>>>> And if there is such an advantage, does it outweight the 
>>>>>>>>> disadvantages
>>>>>>>>> that the extra ack packets cause by causing highly asymmetric links 
>>>>>>>>> to
>>>>>>>>> be overloaded and drop packets?
>>>>>>>> Why is it so bad to drop packets?
>>>>>>> because forcing packets for other services to be dropped to make room
>>>>>>> for acks degrades those other services.
>>>>>> Sure, but remember that we're not here to support the cable company's
>>>>>> business model. They deployed networks that had severely
>>>>>> underprovisioned backchannels so they could use shared channels rather
>>>>>> than routers one step lower in the hierarchy. Now they pull this stunt
>>>>>> so they can fix what's broken with their provisioning model.
>>>>>> The trouble is that it has effects for others in the network, not just
>>>>>> the cable company.
>>>>> It's not just cable companies. the same sort of thing will happen with 
>>>>> half-duplex wifi links where acks will accumulate while data packets are 
>>>>> flowing in the other direction.
>>>>> stop trying to say that this is the fault of one subset of industry and 
>>>>> recognize that there are lots of legitimate reasons for this.
>>>>> Highly asymmetric links are not just 'cable companies underprovisioning 
>>>>> their networks'. DSL lines are highly asymmetric due to the difference 
>>>>> in the cost of the transmitters on each end of the link. As are 
>>>>> Satellite IP systems, etc.
>>>>>>>> TCP isn't supposed to be the most efficient in EVERY corner case. 
>>>>>>>> It's
>>>>>>>> supposed to *always work* in EVERY corner case.
>>>>>>> I don't see how it fails to work in this case. As people have pointed
>>>>>>> out, some cable routers have been doing this for 15 years and the
>>>>>>> Internet has not imploded from it yet, so the drawbacks of dropping
>>>>>>> these already-delayed and redundant ack packets cannot be the
>>>>>>> end-of-the-internet that you are painting it to be
>>>>>> Oh, right. That argument. We haven't seen it break anything, so it
>>>>>> *must* be safe.
>>>>>> What would you see if it were broken? Maybe hosts that burst into the
>>>>>> net and caused router buffers to overload? Hmmm.
>>>>> that happens without this, so you can't blame it on the missing acks.
>>>>>>> We are talking about only doing this in one specific case, the case
>>>>>>> where other things have already caused some of the acks to be delayed 
>>>>>>> to
>>>>>>> the point where later acks have 'caught up' with them on the network 
>>>>>>> and
>>>>>>> both early and late acks are sitting in the same queue on the same
>>>>>>> device waiting to be sent at the same time.
>>>>>> They're in a queue. That means the early ones go out before the late
>>>>>> ones. You have two choices if you coalesce their information:
>>>>>> a) delete the early ACKs
>>>>>>     Oh, but you wouldn't do *that* because it would hit *your*
>>>>>>     customers with a higher delay.
>>>>> but by not having to transmit the early acks, the later ack goes out 
>>>>> faster, so the customers get less of a delay in getting the data they 
>>>>> have received acked.
>>>>> If the only thing in the queue is acks, then the last ack in the queue 
>>>>> goes out as fast as the first ack would. By doing this you transmit 
>>>>> less, which can speed up the network overall as the next station can 
>>>>> transmit it's data faster (thinking of wifi as an example)
>>>>> If there are other packets in the queue, you still can delete all the 
>>>>> acks except the last one that will fit into the burst with no 
>>>>> degredation in how fast data is acknoleged (and you increase the amount 
>>>>> of usable data that is sent in that timeslot instead of wasting it on 
>>>>> redundant ack packets)
>>>>>> b) delete the late ACKs and alter the early ones
>>>>>>     Giving your customers a false sense of how fast their
>>>>>>     data was getting there. Roadrunner pulled stunts like this
>>>>>>     in the early 90's too. It's not exactly news.
>>>>> unless there are other packets in the flow that the ack is jumping, I 
>>>>> see no problem with this. You aren't sending out an ack before the data 
>>>>> is arrived, you just aren't delaying the last ack unneccessarily.
>>>>>>> At this point there are three possiblilities
>>>>>>> 1. all the acks get sent back-to-back, wasting bandwith with their
>>>>>>> redundancy
>>>>>> That's not a waste; that's information.
>>>>> very low value information at best.
>>>>>>> 2. send only the newest ack, trashing all the ones that would be 
>>>>>>> redundant
>>>>>> If you wait to send it last, maybe... but then you're still encouraging
>>>>>> the receiver to burst its next transmissions. We already know that sort
>>>>>> of bursting causes problems (even if *we* don't see them, someone 
>>>>>> does).
>>>>> how would the burst be any different if the server gets 32 acks back to 
>>>>> back vs 1 ack that covers everything. The additional acks aren't going 
>>>>> to get there any faster than the single one would.
>>>>>>> 3. the total of the acks that are queued exceeds the next transmit
>>>>>>> window, so only some of the acks get sent, the newest one doesn't and
>>>>>>> gets delayed further.
>>>>>>> we know that #2 doesn't break the Internet,
>>>>>> No, you really don't. What you know is that #2 is cheap and benefits
>>>>>> you. Everyone continually doing that *will* break the Internet.
>>>>> you keep stating that, but you are short on details about why a string 
>>>>> of acks at wire speed is better than a single ack covering the same data 
>>>>> 'because I say so' doesn't cut it.
>>>>>>> it's within the range of
>>>>>>> responses permitted by the RFC SHOULD.
>>>>>> SHOULD means that breaking it needs to be done for a reason. I've long
>>>>>> argued that the SHOULD should never be there in the first place without
>>>>>> explaining why it isn't a MUST or a MAY and the conditions under which
>>>>>> it might be appropriate to violate it. RFC793 doesn't have that 
>>>>>> context,
>>>>>> unfortunately, but it doesn't mean that any - and every - SHOULD is
>>>>>> intended to be willfully ignored at all times.
>>>>>>> It decreases load on congested links.
>>>>>>                       ^^^^^^^^^
>>>>>> severely underprovisioned
>>>>> no, merely congested for some reason. Any shared media will have the 
>>>>> same situation, when another station is transmitting, a queue builds.
>>>>>>> But you keep insisting that it's a horrible thing to consider doing.
>>>>>> The tragedy of the commons is a horrible thing. Just because something
>>>>>> doesn't hurt you or you can't see how it hurts others doesn't mean 
>>>>>> there
>>>>>> isn't a problem.
>>>>>> I've outlined the reasons why this is bad - basically it works only
>>>>>> under the assumption that DOCSIS modems get to play by their own rules
>>>>>> and every one else plays fair. If that's not a bad idea, I don't know
>>>>>> what is.
>>>>> you say that it breaks timing assumptions and calculations, but then 
>>>>> don't explain how the train of acks arriving at wire speed would let 
>>>>> your calculations be any more accurate.
>>>>> David Lang
>>>>> _______________________________________________
>>>>> aqm mailing list
>>> _______________________________________________
>>> aqm mailing list
>> _______________________________________________
>> aqm mailing list
> _______________________________________________
> aqm mailing list