Re: [tcpm] draft-ietf-tcpm-prr-rfc6937bis-03: set cwnd to ssthresh exiting fast recovery?

[Yoshifumi Nishida <[e-mail]>]

Hi Neal,

Yes, I think I understand your point.
I prefer the current logic in some ways as it's more conservative as I
think we cannot always presume that queue has been drained at the end of
recovery.
But, I also think it may look too conservative.
I am expecting that the authors provide some insights on this point.
--
Yoshi


On Tue, May 2, 2023 at 11:31 AM Neal Cardwell <[e-mail]> wrote:

> Hi Yoshi,
>
> You are right that because PRR always sets cwnd to ssthresh at the end of
> recovery, there will be some cases where with PRR cwnd jumps up drastically
> at the end of the recovery.
>
> However, AFAIK cwnd jumping up drastically, per se, is not a problem. Big
> bursts of packets going into the network is a problem. And given the
> dynamics of the alternative loss recovery algorithms (RFC6675 and PRR),
> both can allow bursts of packets; just in different circumstances:
>
> (1) RFC6675: Because RFC6675 sets cwnd once at the start of fast recovery,
> using (4.2) from RFC6675:
>
> ssthresh = cwnd = (FlightSize / 2)
>
> ...that means RFC6675 allows big bursts at the moment any loss is
> detected: any time L packets are lost, the sender can burst L more packets.
>
> (2) PRR: PRR is specifically designed to avoid big bursts in response to
> packet losses; no matter the structure or timing of the losses, PRR only
> allows a big burst at the end of Fast Recovery after all holes have been
> plugged, and the algorithm sets cwnd to ssthresh.
>
> So in your example ("For example, many packets were lost before entering
> recovery"), AFAICT RFC6675 can allow a big burst at the beginning of
> recovery, when the lost packets are detected. AFAICT in this case PRR can
> allow a burst of packets at the end of recovery when it sets cwnd to
> ssthresh, but at least at this point the bottleneck queue has potentially
> drained somewhat.
>
> Please let me know if that analysis misses something important. :-)
>
> Thanks!
> neal
>
>
> On Mon, May 1, 2023 at 5:22 PM Yoshifumi Nishida <[e-mail]>
> wrote:
>
>> Hi Randall,
>>
>> I might miss something, but here's what I've thought..
>> If we lost many packets in a RTT such as the Figure 5 in the 6937bis
>> draft, I think the window growth during the recovery period will be bound
>> by PRR-CRB or PRR-SSRB.
>> Hence, I think the cwnd at the end of recovery can be smaller than we
>> expect as shown in figure 5.
>> --
>> Yoshi
>>
>> On Mon, May 1, 2023 at 4:17 AM Randall Stewart <[e-mail]> wrote:
>>
>>> Hi Neal and Yoshi:
>>>
>>> Neal: So the FreeBSD implementation in rack, like linux, does the same
>>> exact thing set cwnd to ssthresh at
>>> exit from recovery.
>>>
>>> Yoshi: I don’t see how this would cause cwnd to be larger, since at the
>>> entry to recovery you set ssthresh = cwnd *  Beta. But
>>>           maybe I am missing something, can you give an example like
>>> Neal did below?
>>>
>>>
>>> Thanks
>>>
>>> R
>>>
>>> On May 1, 2023, at 5:32 AM, Yoshifumi Nishida <[e-mail]>
>>> wrote:
>>>
>>> Hi Neal,
>>>
>>> If we always set cwnd to ssthresh at the end of recovery, I am guessing
>>> there will be some cases where cwnd jumps up drastically at the end of the
>>> recovery. For example, many packets were lost before entering recovery.
>>> Or, am I missing something?
>>> --
>>> Yoshi
>>>
>>> On Wed, Apr 19, 2023 at 7:37 PM Neal Cardwell <ncardwell=
>>> [e-mail]> wrote:
>>>
>>>> Working through examples for the "draft-ietf-tcpm-prr-rfc6937bis-03 and
>>>> RecoverFS initialization" thread this evening, I ran into another potential
>>>> issue.
>>>>
>>>> The Linux TCP implementation of PRR explicitly/directly sets cwnd to
>>>> ssthresh at the end of fast recovery (in tcp_end_cwnd_reduction()). But
>>>> this behavior is not in the algorithm in the PRR RFC or draft, at least in
>>>> the figures in section 6, Algorithms. Maybe it is in the prose somewhere
>>>> and I missed it; but in that case I'd argue strongly to put this in the
>>>> figures in section 6, Algorithms.
>>>>
>>>> AFAICT in some cases this is strictly necessary to get cwnd to grow to
>>>> reach ssthresh. Without such a direct step, cwnd could end up far below
>>>> ssthresh at the end of recovery. Here's an example to illustrate:
>>>>
>>>> CC = CUBIC
>>>>
>>>> cwnd = 10
>>>>
>>>> The reordering degree was estimated to be large, so the connection will
>>>> wait for more than 3 packets to be SACKed before entering fast recovery.
>>>>
>>>> --- Application writes 10*MSS.
>>>>
>>>> TCP sends packets P1 .. P10.
>>>> pipe = 10 packets in flight (P1 .. P10)
>>>>
>>>> --- P2..P9 SACKed  -> do nothing
>>>>
>>>> (Because the reordering degree was previously estimated to be large.)
>>>>
>>>> --- P10 SACKed -> mark P1 as lost and enter fast recovery
>>>>
>>>> PRR:
>>>> ssthresh = CongCtrlAlg() = 7 packets // CUBIC
>>>> prr_delivered = 0
>>>> prr_out = 0
>>>> RecoverFS = snd.nxt - snd.una = 10 packets (P1..P10)
>>>>
>>>> DeliveredData = 1  (P10 was SACKed)
>>>>
>>>> prr_delivered += DeliveredData   ==> prr_delivered = 1
>>>>
>>>> pipe =  0  (all packets are SACKed or lost; P1 is lost, rest are SACKed)
>>>>
>>>> safeACK = false (snd.una did not advance)
>>>>
>>>> if (pipe > ssthresh) => if (0 > 7) => false
>>>> else
>>>>   // PRR-CRB by default
>>>>   sndcnt = MAX(prr_delivered - prr_out, DeliveredData)
>>>>          = MAX(1 - 0, 1)
>>>>          = 1
>>>>
>>>>   sndcnt = MIN(ssthresh - pipe, sndcnt)
>>>>          = MIN(7 - 0, 1)
>>>>          = 1
>>>>
>>>> cwnd = pipe + sndcnt
>>>>      = 0    + 1
>>>>      = 1
>>>>
>>>> retransmit P1
>>>>
>>>> prr_out += 1   ==> prr_out = 1
>>>>
>>>> --- P1 retransmit plugs hole; receive cumulative ACK for P1..P10
>>>>
>>>> DeliveredData = 1  (P1 was newly ACKed)
>>>>
>>>> prr_delivered += DeliveredData   ==> prr_delivered = 2
>>>>
>>>> pipe =  0  (all packets are cumuatively ACKed)
>>>>
>>>> safeACK = (snd.una advances and no further loss indicated)
>>>> safeACK = true
>>>>
>>>> if (pipe > ssthresh) => if (0 > 7) => false
>>>> else
>>>>   // PRR-CRB by default
>>>>   sndcnt = MAX(prr_delivered - prr_out, DeliveredData)
>>>>          = MAX(2 - 1, 1)
>>>>          = 1
>>>>   if (safeACK) => true
>>>>     // PRR-SSRB when recovery is in good progress
>>>>     sndcnt += 1   ==> sndcnt = 2
>>>>
>>>>   sndcnt = MIN(ssthresh - pipe, sndcnt)
>>>>          = MIN(7 - 0, 2)
>>>>          = 2
>>>>
>>>> cwnd = pipe + sndcnt
>>>>      = 0    + 2
>>>>      = 2
>>>>
>>>> So we exit fast recovery with cwnd=2 even though ssthresh is 7.
>>>>
>>>> As noted above, the Linux TCP implementation does not suffer this
>>>> problem because it explicitly/directly sets cwnd to ssthresh at the end of
>>>> fast recovery.
>>>>
>>>> I would recommend including this cwnd=ssthresh step at the end of
>>>> recovery in the draft, to ensure that cwnd reaches ssthresh at the end of
>>>> fast recovery, even in cases like this where there will be insufficient
>>>> delivered data in fast recovery to allow pipe to incrementally grow to
>>>> reach ssthresh using PRR-SSRB.
>>>>
>>>> Best regards,
>>>> neal
>>>>
>>>> _______________________________________________
>>>> tcpm mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>> <https://www.google.com/url?q=https://www.ietf.org/mailman/listinfo/tcpm&source=gmail-imap&ust=1683538345000000&usg=AOvVaw2cOITQpYcuP_M95396rEmw>
>>>>
>>> _______________________________________________
>>> tcpm mailing list
>>> [e-mail]
>>>
>>> https://www.google.com/url?q=https://www.ietf.org/mailman/listinfo/tcpm&source=gmail-imap&ust=1683538345000000&usg=AOvVaw2cOITQpYcuP_M95396rEmw
>>>
>>>
>>> ------
>>> Randall Stewart
>>> [e-mail]
>>>
>>>
>>>
>>>