Re: [tcpm] More motivating scenarios for tcpm-ack-pull

[Jon Crowcroft <[e-mail]>]

Mark Handley said a lot of the middlebox corner case avoidance stuff is in
the tcpmp RFC...have got time to look just yet...

On Mon, 18 Nov 2019, 04:46 Carles Gomez Montenegro, <[e-mail]>
wrote:

> Hi Bob,
>
> Once again, thanks a lot for all the very useful feedback!
>
> Please find below some inline responses.
>
> > Carles, Jon, (re-sending, this time without accidentally omitting tcpm,
> > also see couple of addenda tagged [Bob adds:])
> >
> > I'm generally supportive of a mechanism to suppress delayed ACKs from
> > the receiver (in any transport protocol including TCP). So thank you.
> >
> > You might want to borrow at least some of the text for additional
> > motivating scenarios for such a facility from the first two posts on a
> > thread for a similar facility in QuiC:
> > https://github.com/quicwg/base-drafts/issues/1978
> >
> > Also pasted at the end.
> >
> > Some is not relevant to your specific ack-pull scheme, 'cos it's
> > proposing a more general facility in QUIC for the sender to the receiver
> > alters its ACK ratio. (theoretically there are more bits available in
> > QUIC, but this particular request has been put on hold while someone
> > goes and finds them ;)
>
> Thanks a lot. This is indeed very useful!
>
> As you explain, the more general facility for QUIC allows to suppress
> Delayed ACKs and also allows to modify the ACK ratio in a more general
> way.
>
> Regarding the Delayed ACK suppression discussion, the additional
> motivation is definitely important.
>
> >       Another motivating example (rather niche)
> >
> > Coincidentally, Just this morning I published a tech report that had to
> > use the hack of overlapping a byte from the previous segment to force a
> > quick ack:
> > TCP Prague Fall-back on Detection of a Classic ECN AQM
> > <https://arxiv.org/pdf/1911.00710.pdf#page9> (the link takes you to Fig
> > 2, which illustrates the hack)
> > I'd rather not have to do these sorts of hacks - it really messes with
> > the segmentation code.
>
> This is very useful feedback as well, and a nice coincidence!
>
> Regarding the hack, sending an old byte may be even worse when there is
> not a next data segment readily available for sending, which would lead to
> sending a whole packet just to carry an old byte.
>
> >       Middlebox traversal of bit 6
> >
> > I recall that someone did a study of traversal of each of the reserved
> > flags (Marcelo Bagnulo maybe?). I don't think it was that good [Bob
> > adds: I mean traversal - not the study!].
> > A good search engine might find it ;)
>
> I wasn't able to find the study, but this is an important consideration.
>
> One might think that if new functionality is defined and standardized,
> related middlebox traversal of it should improve over time. Perhaps that
> might be optimistic...
>
> >       More alternative approaches:
> >
> > 1/ At one stage, we tried to include a bit for Delayed Ack control in
> > another protocol called Accurate ECN. We were persuaded to take it out,
> > because it was mission creep (outside the scope of the protocol we were
> > working on):
> >
> https://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-03#appendix-B.4
>
> I was unaware of this, thanks!
>
> This is actually very aligned with what we are proposing...
>
> > 2/ Have you considered using the Urgent Pointer in a novel way? I.e. a
> > non-zero Urgent Pointer field, even tho the URG flag is zero.
> > [Bob adds:] From memory, traversal was pretty good - much better than
> > bits 4-6. But I think Windows treated it as a potential attack. I
> > believe Fernando Gont did a study on how 'invalid' TCP header fields are
> > handled.
> >
> > You could assign, say, 3 bits of the urgent pointer for a variable
> > ack_exp that is log base 2 of the ack ratio the sender would like. Then
> > the sender can request the receiver uses ack_ratio` = 2^ack_exp, as in
> > the QUIC proposal below,
> > With ack_exp = 0, you get ack_ratio = 2^0 = 1, which has the same effect
> > as ack-pull. But you also have a more general facility for high speed
> > machines to widen out the ack ratio.
> >
> > If this works, you could open up a registry for the other 13 bits. So
> > instead of using up a precious bit, you generate 13 more ;)
>
> Nice! :)
>
> Actually, we had not considered such an idea.
>
> In your proposal above, if a segment has the URG flag set to 1, would then
> the 3 ack_exp bits still be used as ack_exp bits, thus reducing the size
> of the urgent pointer in practice to 13 bits?
>
> > See:
> > https://tools.ietf.org/agenda/90/slides/slides-90-tcpm-10.pdf#page=5
> >
> > I think there were problems with some implementations and middleboxes
> > (bound to be). I've checked the minutes of the IETF meeting where that
> > slide was presented
> > <https://www.ietf.org/proceedings/90/minutes/minutes-90-tcpm>, but
> > there's no push-back in there. I suggest you delve back into the
> > archives of the tcpm mailing list around the time of that meeting to
> > find out if there were any show-stoppers.
>
> Thanks for the pointers!
>
> Sure, we'll check the tcpm mailing list archive.
>
> Cheers,
>
> Carles
>
>
> > Cheers
> >
> >
> >
> > Bob
> >
> >
> > Paste from QUIC thread:
> >
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
> >
> >
> >
> >       Problem/goals:
> >
> >  1. Getting up to speed fast without inducing much queue is one of the
> >     main areas where latency reductions are needed (for short and long
> >     flows). The sender needs frequent ACKing during this phase in many
> >     approaches (hystart, etc). A Linux TCP receiver starts with
> >     ack_ratio 1 and uses heuristics to identify the end of the sender's
> >     slow-start (or a re-start after idle). This has ossified a certain
> >     slow-start behaviour into all TCP senders (whether Linux or not). We
> >     are trying to new sender behaviours (paced chirping being an
> >     example), but we need a way for the sender to suppress delayed ACKs.
> >  2. At the other end of the scale, for long-running flows, many current
> >     CC approaches (e.g. BBR) use pacing not ACK-clocking so they can use
> >     very few ACKs per RTT. And fewer means less cache ejections for GRO.
> >     But the receiver doesn't know what CC the sender is using, so it
> >     doesn't know how many or few is too many or too few.
> >  3. Some middleboxes and the majority of link technologies (e.g. DOCSIS,
> >     LTE, Satellite) thin TCP ACKs when they detect the upstream is
> >     filled with TCP ACK stream(s), which are unresponsive. Otherwise the
> >     ACKs constrain downstream throughput (and any upstream data either
> >     in the flow itself, or in others). We don't want these links to
> >     attempt to guess which are the QUIC ACKs and try to thin them. QUIC
> >     can and should do this itself. QUIC has all the machinery for the
> >     sender CC to detect ACK congestion, but not the protocol to tell the
> >     receiver to do the thinning. This protocol addition would provide a
> >     sufficient hook for hosts to unilaterally add this to their CC
> >     behaviour.
> >
> >
> >       Some scenarios where the sender's preferred ack_ratio changes
> >       through the connection:
> >
> >  1.
> >
> >     A sender CC that wants the receiver to turn off DelAcks during
> >     flow-start (e.g. it's using hybrid slow-start as in Cubic and wants
> >     to get delay measurements more frequently) sets ack_exp=0 during
> >     flow-start (ack_ratio=1), then increases ack_exp during congestion
> >     avoidance. If it goes idle, then re-starts, it would set ack_exp=0
> >     again.
> >
> >     Note on heuristics: A Linux TCP receiver currently uses a heuristic
> >     to determine when the sender has exited slow-start. However,
> >     heuristics -> ossification. A Linux receiver's heuristic only works
> >     with the current pattern of slow-start. In TCP, when we tried to
> >     improve the pattern on the sender (paced chirping), the heuristic on
> >     the receiver killed us.
> >
> >  2.
> >
> >     Imagine a paced sender has hardware generic receive offload (GRO),
> >     so for a long-running flow it doesn't want a high rate of QUIC ACKs
> >     that are opaque to GRO. Let's say it would prefer at least 8 ACKs
> >     per RTT. Again, it starts with ack_exp=0, but in congestion
> >     avoidance it would use:
> >
> >     ack_ratio <= cwnd_in_packets/8
> >
> > Upshot: By remote controlling the receiver, the server offloads nearly
> > all the ACKs from large downloads, but still focuses its ACK-receiving
> > resources on getting each client up to speed.
> >
> > Note that calculation of ack_ratio lends itself to fast integer
> > arithmetic.
> >
> >
> >
> >
> >
> >
> >
> > Bob
> >
> >
> > --
> > ________________________________________________________________
> > Bob Briscoehttp://bobbriscoe.net/
> >
> >
>
>
>