Re: [tcpm] Linux doesn’t implement RFC3465

[Vidhi Goel <[e-mail]>]

Resurrecting the 3465 thread.

In the TCPM meeting at IETF 111, we discussed about this issue of L=2 which is a MUST in RFC 3465. This is a very strict requirement and stacks like Linux already doesn’t follow it.
> The basic principle in the Linux code is that the sender should use the ACKs to learn about the capacity of the path (both in volumetric and rate dimensions), and should not ignore that information. This allows the sender to quickly grow and achieve high throughput, even in the presence of stretch ACKs, which are pervasive, due to TSO/GSO, GRO/LRO, etc.
> 
> Considering bursts is important, but that can be tackled as an orthogonal issue. Bursts are avoided in the Linux TCP ecosystem by the combination of TSO autosizing, pacing, TSQ, and fair queueing.

As Neal described, the congestion controller should use the information in stretch ACKs to increase its congestion window so that it correctly adjusts the cwnd based on available link capacity.
Burstiness is an orthogonal issue which can be solved by pacing.

QUIC loss recovery (RFC 9002) also follows this approach.

Mark,
As a lot of transport and congestion control drafts reference RFC 3465, do you think we should update this RFC to reflect the current deployment? This would also be useful for someone who is just starting with a new implementation.

Thanks,
Vidhi

> On Nov 27, 2019, at 5:14 AM, Mark Allman <[e-mail]> wrote:
> 
> 
>> +Mark Allman
> 
> Just to clear it up, I *was* at BBN long ago when the ABC document
> was written.  It's a cool place to work.  I recommend it.  But, I
> now hang out at ICSI.
> 
>> I believe that ABC was written to solve the problem with ACK
>> counting by counting the number of bytes acknowledged for
>> misbehaving receivers. Limiting the increase to 2*MSS was a good
>> solution to avoid bursts at the time.
> 
> The main motivation behind ABC was to counteract delayed ACKs.  The
> common approach at the time was to just bump cwnd by one MSS every
> time an ACK rolled in.  So, if an ACK covered two segments because
> the receiver was delaying the ACKs then the growth rate during slow
> start was 1.5x per RTT instead of the 2x that was really
> envisioned.  During congestion avoidance the growth was 1 MSS every
> 2 RTTs instead of the envisioned every RTT.
> 
> A secondary motivation was to counteract these ACK division attacks
> that Savage taught us about.  I.e., we could ACK an MSS-sized packet
> one byte at a time and the sender would then increase the cwnd by
> MSS*MSS bytes in the prevalent ACK counting scheme (i.e., cwnd would
> get bumped by MSS bytes for every ACK).
> 
> The limit has two roots ...
> 
> (1) The limit is important in slow starts that follow an RTO.  As
>    the RFC discusses, in this case we might retransmit a single
>    packet and this will cause the receiver's window to slide a
>    great deal.  Therefore, an ACK may indicate that a ton of data
>    has left the network, but that isn't really the case.  So, we
>    don't want to increase the cwnd based on all the new bytes
>    ACKed.
> 
>    I have since mostly decided that this use of L is crude.
>    Probably there is a more elegant way to do it by using the
>    scoreboard and the SACK information to get a better
>    understanding of what left the network and when.  That said, in
>    this case L is simple and probably about right most of the
>    time.
> 
> (2) I think there was some general conservativeness to bursts and
>    using L everywhere quelled some of the worry.  Here L=2 was used
>    to exactly offset delayed ACKs.
> 
>> I agree that increasing the congestion window and controlling the
>> burst rate are orthogonal issues.
> 
> Yes.  In fact, we did subsequent research on mitigating bursts
> because we never really thought of ABC as somehow the way to control
> bursts (research papers available, but never went into RFCs).
> 
> And, in a world that leverages stretch ACKs as a routine I think
> Linux's approach of not using an L may well be correct.  Documenting
> that and the reasoning behind it in modern networks seems useful to
> me.
> 
> allman
>