Re: [tcpm] flow control and fast recovery

[Yoshifumi Nishida <[e-mail]>]

Hi Yuchung,

I tried to mention how much buffer size will be required to guarantee
fast retransmit and fast recovery works propoerly.
But, I'm not very sure if this plot is a proper example for detailed
discussions on standards. Because the behavior of the sender is a bit
strange to me.
It seems that it tries to do rate halving (or PRR) at the beginning of
recovery, but it speeds up after a while even though it stills in
recovery phase.
This aggressive increase will require more buffer space at the
receiver, but I think it's not a standard behavior.
Thanks,
--
Yoshifumi


On Tue, Aug 13, 2013 at 8:56 AM, Yuchung Cheng <[e-mail]> wrote:
> On Mon, Aug 12, 2013 at 10:39 PM, Yoshifumi Nishida
> <[e-mail]> wrote:
>> Hi Alejandro,
>>
>> Thanks for the dump file.
>> Please correct me if I miss something.
>>
>> During the first retransmit and fast recovery, the sender can transmit
>> 2/sender's cwnd - MSS bytes of new data. (because 1MSS is consumed by
>> retransmission)
>> This means in the worst case, you will need to have 3/2 sender's cwnd
>> receiver buffer to keep all data transmitted during this period.
>> But, the worst case can only happen when the retransmit segment by
>> fast retransmit arrives after all new data has arrived.
>> (Another example will be a case where the receiver's application
>> becomes slow to read data all of sudden during loss recovery.)
>>
>> You're right that if we want to prepare the worst case, the receiver
>> will need 1.5 times larger size of buffer than the sender's buffer.
>> But, I'm not sure this is a problem. Because I'm not very sure TCP
>> needs to guarantee full performance when sender's buffer size equals
>> receiver's buffer size.
> It's not about sender's buffer size.
>
> It'll be easier to explain with time sequence graph, but I doubt this
> list allows binary attachment so
> try this
> tcptrace -CSzxy <pcap> && xplot.org b2a_tsg.xpl
>
> During the recovery, the rwin remains stale and the yellow line is horizontal.
>
> The fundamental problem is that the receiver (likely a Linux box) does
> not account for OOO packets received to adjust RWIN.
> But congestion control, or cwin, does account for SACKed packets and
> retransmits.
>
> When the receiver have too many OOO packets it unintentionally thwart
> the sender, creating a bubble in the data pipeline at 5s.
>
> Due to bufferbloat, by the time the loss happens, the network has
> buffered a ton. By the time the fast retransmit finally arrives to
> repair the losses, the rwin opens up the flood gate, hence the big
> jump of the yellow line. Interesting the sender is probably
> application-limited so it didn't send out a burst.
>
> So the receiver is unintentionally throttling the sender to make
> forward progress during recovery. Not a good idea unless the receiver
> really can't afford the extra 100KB.
>
>
>>
>> In your tcpdump, the first retransmit segment by fast retransmit seems
>> to be lost, hence you got extra dup acks.
>> But, I think this is not a situation that fast retransmit logic expects.
>>
>> Thanks,
>> --
>> Yoshifumi
>>
>>
>> On Sun, Aug 11, 2013 at 11:07 PM, Alejandro Popovsky <[e-mail]> wrote:
>>> Hi Christoph,
>>>
>>> I left another example where the receiver is tuning its receive window, but
>>> not
>>> as dynamically as to prevent a sender stall during fast recovery:
>>>
>>> http://www.palermo.edu/ingenieria/comm/flowCtrlFastRecovery2.pdf
>>>
>>> http://www.palermo.edu/ingenieria/comm/exampleDumpFlowCtrlFastRecovery2.pcap
>>>
>>> Best regards, Alejandro.
>>>
>>>
>>>
>>> On 11/08/13 04:05 PM, Christoph Paasch wrote:
>>>>
>>>> Hello,
>>>>
>>>> On 11/08/13 - 14:19:50, [e-mail] wrote:
>>>>>
>>>>> I have just left an example connection in:
>>>>>
>>>>> http://www.palermo.edu/ingenieria/comm/exampleDumpFlowCtrlFastRecovery.pcap
>>>>
>>>> the trace looks rather like the receiver has its window capped at 64K.
>>>> E.g.,
>>>> through a socket-option. Because from the beginning on, the announced
>>>> window
>>>> is at 64K and it never changes.
>>>>
>>>> If the client would not cap the window, the autotuning should do its job
>>>> to
>>>> adjust the window at 2*BDP, and thus allow full speed - even during
>>>> recovery.
>>>>
>>>>
>>>> Cheers,
>>>> Christoph
>>>>
>>>>> I am leaving also an analysis of the connection showing the flow
>>>>> control limitation reached during fast recovery, here:
>>>>> http://www.palermo.edu/ingenieria/comm/flowCtrlFastRecovery.pdf
>>>>>
>>>>> Let me know if you want some other examples.
>>>>>
>>>>> Best regards, Alejandro.
>>>>>
>>>>>
>>>>>
>>>>> On 11/08/13 05:44 AM, Yoshifumi Nishida wrote:
>>>>>>
>>>>>> Hi Alejandro,
>>>>>> Is it possible to see tcpdump files for this? It might be better if we
>>>>>> can discuss with real data.
>>>>>> --
>>>>>> Yoshifumi
>>>>>>
>>>>>> On Fri, Aug 9, 2013 at 1:43 PM, Alejandro Popovsky wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have been observing that many connections become limited
>>>>>>> by the receiver window during fast recovery, even when this window
>>>>>>> is above RTT*maximumPathRate (and using windows scaling).
>>>>>>>
>>>>>>> This is because during fast recovery the congestion window is
>>>>>>> artificially inflated on each duplicate ack (after the third). And the
>>>>>>> number of unacked bytes may come up to double RTT*maximumPathRate.
>>>>>>>
>>>>>>> For this to be prevented, the receiver may grow its reception window
>>>>>>> up to double its size when generating duplicate acks.
>>>>>>>
>>>>>>>
>>>>>>> I observed this at the traffic of service providers that were having an
>>>>>>> important percentage of their traffic limited by flow control (most of
>>>>>>> the
>>>>>>> traffic is generally limited by the network, or by the data generation
>>>>>>> rate
>>>>>>> at the source).
>>>>>>>
>>>>>>>
>>>>>>> Best regards, Alejandro Popovsky.
>>>>>>>
>>>>> _______________________________________________
>>>>> tcpm mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/tcpm
>>>
>>>
>>> _______________________________________________
>>> tcpm mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/tcpm
>> _______________________________________________
>> tcpm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/tcpm