Re: [tcpm] draft-ietf-tcpm-rack-05 review

[Emmanuel Lochin <[e-mail]>]

Hi Yuchung,

Thanks for your answer, I've asked this question as we observed a 
decrease of the performance of TCP transfer in some reordering scenario 
when RACK is enabled. In brief, the problem has been solved by disabling 
RACK.
I am involved in a CNES project (French space agency) where SATCOM 
systems are used (GEO or LEO) and in particular I work with Nicolas Kuhn 
from CNES (in copy of this email) .

To assess whether or not the problem effectively came from RACK, we have 
started a bunch of experiments using Mininet or simply using the 
loopback interface. The results presented below are done with Mininet 
except the last one done in localhost

The x-axis reports the size of the flows, basically we send 5 TCP flows 
in the same time of a given size and we repeat the experiments hundred 
of times over a Netem queue. We then compute the following statistics 
over these 500 flows for each flow length.

There is no artificial losses introduced with Netem, we only use it only 
to reorder packets. The nominal RTT is 500ms (equiv. to a GEO system) 
where some packets are reordered every 5, 10, 15 or 20 packets sent. 
These reordered packets have then an RTT=250ms. As you can see, in all 
cases when RACK is enabled, the completion time (time difference between 
the first packet received at destination and the last one) is always 
worse with RACK. The kernel is 4.15.0-60-generic.



We thought, may be, it was due to the reordering pattern which is too 
deterministic. We thus applied several other random patterns.
Below we present the Netem config documented in the TC manpage for 
simplicity (i.e. reorder 25% 50% gap 5 and 10)


But results obtained where the same.

In all these previous experiments the reorder delay is 50% of the RTT, 
and indeed, the RTT is important.
If we decrease it to 100ms, when the reorder delay corresponds to 90% of 
the RTT (meaning that RTT=100ms and reordered packets get 90ms), in that 
case, RACK performs much more better as shown below.




Using the loopback interface, even with low RTT (i.e. 100ms), and when 
the reordering delay is half the RTT (50ms), RACK still does not perform 
always better than 3DUPACK.


We really believe that RACK is a good idea, but it seems that the kernel 
implementation does not lead to what we expected. Do you have 
experienced such behavior or do you have measurements that thwart mine ?

Thanks

Emmanuel



Le 07/09/2019 à 00:32, Yuchung Cheng a écrit :
> Hi Emmanuel,
>
> Yes setting the sysctl to 0 will completely disable RACK. Although by 
> default it supports DUPTHRESH-based detection (as a simple extension) 
> beside time-based detection. You can disable that extension w/ 0x4. 
> But if you want absolutely old-style pure DUPTHRESH-based then you can 
> set that to 0.
>
>
> On Fri, Sep 6, 2019 at 3:24 AM Emmanuel Lochin 
> <[e-mail] 
> <mailto:[e-mail]>> wrote:
>
>     Hi Yuchung, all,
>
>     I have a question concerning the GNU/Linux kernel implementation
>     of RACK.
>     There are 3 sysctl options to enable RACK (0x1 0x2 and 0x4) but
>     none to disable it (or may be 0x0 is not documented). As
>     tcp_input.c implements this boolean function:
>
>     static bool tcp_is_rack(const struct sock *sk)
>     {
>             return sock_net(sk)->ipv4.sysctl_tcp_recovery &
>     TCP_RACK_LOSS_DETECTION;
>     }
>
>     can I consider that I can disable RACK by typing sysctl -w
>     net.ipv4.tcp_recovery=0 ?
>     I ask this question because I've seen several update done inside
>     the kernel to implement it and I want to be sure that I can really
>     go back to 3DUPACK scheme this way.
>
>     Cheers,
>
>     Emmanuel
>
>     Le 05/09/2019 à 04:44, Martin Duke a écrit :
>>     I think the proposed text is fine.
>>
>>     On Wed, Sep 4, 2019 at 5:55 PM Yuchung Cheng <[e-mail]
>>     <mailto:[e-mail]>> wrote:
>>
>>         On Tue, Sep 3, 2019 at 3:31 PM Martin Duke
>>         <[e-mail] <mailto:[e-mail]>> wrote:
>>
>>     <snip>
>>
>>         > (3) sec 7.3
>>         >
>>         > We have evaluated using the smoothed RTT (SRTT from
>>         >    [RFC6298] RTT estimation) or the most recently measured RTT
>>         >    (RACK.rtt) using an experiment similar to that in the
>>         Performance
>>         >    Evaluation section.  They do not make any significant
>>         difference in
>>         >    terms of total recovery latency.
>>         >
>>         > If there is truly no difference, then why not use SRTT as
>>         the standard?
>>         > Every TCP implementation has to store this, while min_rtt
>>         is unneeded for many (most?) congestion controls.
>>         >
>>         > Alternatively, you could strengthen this paragraph to not
>>         sound like it makes no difference..
>>         That's a good point -- our experiment at Google servers
>>         indeed didn't
>>         show much difference. But I think it's still better to use
>>         min_RTT
>>         than SRTT. On buffer-bloat friendly C.C. the SRTT could be
>>         orders of
>>         magnitude longer than the actual paths' RTT -- in my opinion
>>         factoring
>>         this network queuing delay in reordering window is not a good
>>         idea. So
>>         how about adding
>>
>>         'While the experiment does not show a difference between min
>>         RTT and
>>         SRTT, SRTT is less desirable to size the reordering window as it
>>         includes network congestion or delayed ACKs effects."
>>
>>
>>
>>     _______________________________________________
>>     tcpm mailing list
>>     [e-mail]  <mailto:[e-mail]>
>>     https://www.ietf.org/mailman/listinfo/tcpm
>
>     -- 
>     Emmanuel LOCHIN
>     Professeur ISAE
>     ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
>     10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE -
>     http://www.isae-supaero.fr
>     Tel [phone] - Fax [phone]
>     Web :http://personnel.isae.fr/emmanuel-lochin/
>

-- 
Emmanuel LOCHIN
Professeur ISAE
ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE -
http://www.isae-supaero.fr
Tel [phone] - Fax [phone]
Web : http://personnel.isae.fr/emmanuel-lochin/