Re: [tcpm] On Sender Control of Delayed Acks in TCP

[Neal Cardwell <[e-mail]>]

On Sat, May 2, 2020 at 5:40 AM Carles Gomez Montenegro
<[e-mail]> wrote:
>
> Hi Bob,
>
> (Chiming in...)
>
> > Neal,
> >
> > On 01/05/2020 13:19, Neal Cardwell wrote:
> >> On Fri, May 1, 2020 at 6:23 AM Carles Gomez Montenegro
> >> <[e-mail]> wrote:
> >>> Hi Neil,
> >>>
> >>> Thanks a lot for your comments!
> >>>
> >>> Please find below my inline responses:
> >>>
> >>>> On Wed, Apr 29, 2020 at 1:03 PM Scheffenegger, Richard
> >>>> <[e-mail]>
> >>>> wrote:
> >>>>> Eliciting an ACK under certain circumstances, for a timely
> >>>>> continuation
> >>>>> of the data transmission / growth of the congestion window is a known
> >>>>> method to reduce network delay.
> >>>>>
> >>>>> E.g. Linux has been using the CWR flag for the purpose of sending out
> >>>>> an
> >>>>> immediate ACK by the receiver, since there is an increased chance of
> >>>>> a
> >>>>> very small cwnd when the sender had just reduced it's congestion
> >>>>> window.
> >>>>>
> >>>>> https://lore.kernel.org/patchwork/patch/970486/
> >>>>>
> >>>>> and we also found latency improvements doing this when running
> >>>>> ECN-enabled sessions
> >>>>>
> >>>>> https://reviews.freebsd.org/D22670
> >>>> Yes, agreed.
> >>>>
> >>>> IMHO an explicit ACK-pull mechanism would be very nice for the cwnd<=1
> >>>> case.
> >>> (While this question is about the solution space, I'm anyway
> >>> curious...)
> >>> In the context of datacenter networks, would you have any suggestion or
> >>> preference regarding the solutions that have been mentioned so far?
> >>>
> >>> Or perhaps any particular requirement for a potential solution?
> >> Among the solutions outlined in
> >>    https://tools.ietf.org/html/draft-gomez-tcpm-delack-suppr-reqs-00
> >> my first preference would be the AKP option, since that approach
> >> avoids burning a precious flag bit.
> >
> > [BB] Eh? AKP does burn a flag bit (and you say it does below). Quoting
> > the draft:
> >
> >     TCP ACK Pull defines the AKP flag as bit number 6 of the 13th byte of
> >     the TCP header.
>
> [CG] I think that Neal referred to section 5.4 of the requirements draft
>  ("A new 'ACK Pull' TCP option").

Yes, as Carles says, when I mentioned "AKP option" I was referring to
the alternative discussed in section 5.4, "A new 'ACK Pull' TCP
option", which does not burn a flag bit:

  https://tools.ietf.org/html/draft-gomez-tcpm-delack-suppr-reqs-00#section-5.4

When I mentioned "flag bit" I was referring to section 5.3, "TCP ACK
Pull (AKP) flag", which does burn a flag bit:

  https://tools.ietf.org/html/draft-gomez-tcpm-delack-suppr-reqs-00#section-5.3

> > (BTW, Carles, in this quoted sentence, you ought to call it "the TCP ACK
> > Pull /proposal/" otherwise sounds like it's an official part of TCP
> > already.)
>
> [CG] Well noted, thanks!  (Nevertheless, if that other document went
> ahead, I guess that "proposal" would need to be eventually removed.)
>
> >> The AKP option may be stripped by some middleboxes, but (a) as a
> >> performance optimization, that should be acceptable, and (b) for the
> >> datacenter case (where cwnd=1 is a motivating use case) this should
> >> not be a concern.
> >>
> >> IMHO a flag bit makes sense for a small signal that a sender might
> >> want to send frequently or at a high rate, but senders should not be
> >> trying to force immediate ACKs frequently.
> >
> > [BB] For me, the problem with the AKP approach is that it's /only/
> > really good in environments where you've got a small window. When you
> > have a large window and you can cope with a certain ACK ratio (say
> > 1:16), you would have to set AKP on every 16th packet, which doesn't
> > really express what the sender wants (and if the receiver had just sent
> > an ACK of its own accord just before it got one of these regular AKP
> > requests, should it send another?). The sender in this case really wants
> > to say "I don't particularly mind which packets you ACK, but there's no
> > need to do it more often than 1:16".
> >
> > I think there's some mileage in a TCP Option that aggregates a few
> > behaviours together under one new TCP Option. I reckon 3 bits would be
> > enough for a good delayed ACK option, but a whole TCP Option for that
> > could be overkill. There was going to be a talk about a low latency TCP
> > Option in netdev0x14 before it was postponed (sorry I don't remember who
> > by). That might be a good excuse to collect together a few related
> > functions. It could also improve the deployment incentives.
>
> [CG] Thanks for the comments!
>
> [CG] It seems that there are two different areas where avoiding the
> typical Delayed ACKs approach may be of particular interest: i) scenarios
> where very small window sizes are used (cwnd=1), and ii) scenarios where
> large window sizes, and high ACK ratios, can be used. We will reflect
> these considerations in the next draft update.

Yes, I wholeheartedly agree with Bob's point that it's also extremely
useful for TCP senders to be able to say things like: "I don't
particularly mind which packets you ACK, but there's no need to do it
more often than 1:16". QUIC has a mechanism like this, and it seems to
allow significant CPU savings, even allowing QUIC to run faster than
TCP in some cases, AFAICT due to the reduced overhead for handling
ACKs:
  https://www.fastly.com/blog/measuring-quic-vs-tcp-computational-efficiency
  https://tools.ietf.org/html/draft-iyengar-quic-delayed-ack-00

So I agree with the usefulness of the two-use-case taxonomy Carles has outlined:

1: "please ACK now"; because sender's cwnd is low

2: "ACK at frequency K"; because sender's cwnd is high

It seems those could both be addressed cleanly with one or two TCP
option kinds. A single TCP option kind might suffice if the ACK
frequency of 0 or 1 is taken to mean "please ACK now".

best,
neal