[tcpm] Re: using SACK info for RTTM?

[Yuchung Cheng <[e-mail]>]

On Wed, Jun 5, 2024 at 11:57 AM Neal Cardwell <[e-mail]> wrote:

> On Wed, Jun 5, 2024 at 2:21 PM Yoshifumi Nishida <[e-mail]>
> wrote:
>
>> Hi Yuchung, Neal, thank you so much. It’s interesting.
>>
>> So, it might be a good time to revive
>> https://datatracker.ietf.org/doc/draft-yang-tcpm-ets/ ?
>>
>
> Yes, reviving the work for some kind of standardization of more precise
> TCP timestamps is something we would like to do when time permits.
>
>
>> OTOH, I’m thinking why DSACK is not sufficient here.
>>
>
> DSACK can work sometimes, however it is not as good as timestamp undo, for
> at least a few reasons:
>
> (1) DSACK undo is slower: DSACK undo takes about two round trips longer
> than timestamp undo. With DSACK undo the data sender has one extra round
> trip to make a full set of spurious retransmissions for apparent holes in
> the sequence space, and then a second round trip to receive all the DSACKs
> for the spurious retransmissions. Only then can the data sender undo the
> congestion control response. With timestamp-based undo, usually a data
> sender will receive an ACK very shortly (say, O(1ms)) after its spurious
> retransmit that covers the spuriously retransmitted sequence and has a TS
> ECR from before the retransmit, allowing the undo to happen immediately.
>
> (2) DSACK undo is unreliable: if even a single ACK packet containing a
> DSACK is lost, then the data sender cannot undo the congestion control
> response. For example, if a flow is fully utilizing a 10Gbps * 100ms path,
> and thus (1500B MTU) its cwnd is at least 82,562, then the loss rate in the
> direction of returning ACKs needs to be zero in that round trip, or on
> average less than 1/82,562, or < 0.0012%. Not impossible, but a stringent
> requirement. With timestamp-based undo every incoming ACK has a TS ECR
> value that allows undo, so ACK loss is not a problem.
>

(3) DSACK undo is also more complicated to implement (esp corner cases)


> neal
>
>
>
>> Thanks,
>> —
>> Yoshi
>>
>> On Wed, Jun 5, 2024 at 10:06 AM Yuchung Cheng <[e-mail]> wrote:
>>
>>> Also TCP timestamp needs to really move to usec level for today's
>>> data-center networks, which Eric Dumazet finally upstreamed that feature
>>> (to opt-in). anything beyond 10us can't be used in Eifel
>>>
>>> On Wed, Jun 5, 2024 at 6:41 AM Neal Cardwell <[e-mail]>
>>> wrote:
>>>
>>>> IMHO by far the biggest benefit of TCP timestamps is not in RTT
>>>> measurement or PAWS, but in using them for "Eifel" undo (a la RFC 3522, RFC
>>>> 4015): quickly detecting spurious loss detection events due to reordering,
>>>> and quickly undoing the spurious congestion control slow-down response.
>>>> This is important since reordering is increasingly common due to many
>>>> increasingly common network mechanisms: link-layer retransmissions for
>>>> wifi/cellular links, traffic engineering, multipathing and ECMP/WCMP
>>>> load-balancing, protective load balancing (SIGCOMM 2022), protective
>>>> reroute (SIGCOMM 2023), multi-queue NICs, etc. Those factors make the 12
>>>> bytes of TCP option space overwhelmingly worth it.
>>>>
>>>> best regards,
>>>> neal
>>>>
>>>>
>>>> On Wed, Jun 5, 2024 at 3:03 AM Yoshifumi Nishida <[e-mail]>
>>>> wrote:
>>>>
>>>>> Hi Yuchung,
>>>>>
>>>>> Thanks for the explanation.
>>>>> I thought a bit about the trade-off between using 12 bytes options
>>>>> space and giving up measuring RTTs for retransmitted packets.
>>>>> But, I am included to prefer measuring RTTs for now.
>>>>>
>>>>> --
>>>>> Yoshi
>>>>>
>>>>> On Mon, Jun 3, 2024 at 1:57 PM Yuchung Cheng <[e-mail]>
>>>>> wrote:
>>>>>
>>>>>> hi Yoshifumi,
>>>>>>
>>>>>> Linux only uses TS-opts if needed to disambiguate on RTT samples
>>>>>> covering sequences that have been retransmitted. This applies to SACK or
>>>>>> non-SACK. In order words, if an S/ACK covers a sequence range that has
>>>>>> never been retransmitted, Linux does not use timestamp options.
>>>>>>
>>>>>> On Mon, Jun 3, 2024 at 1:29 PM Yoshifumi Nishida <[e-mail]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Neal, thank you so much for the comments.
>>>>>>>
>>>>>>> The linux algorithm you've described makes sense to me and it seems
>>>>>>> the scheme doesn't require timestamp options.
>>>>>>> However, as far as I've read linux code, it seems that linux still
>>>>>>> uses timestamp options for RTT measurement to some extent.
>>>>>>> I'm curious why linux is mixing two schemes for RTTM.
>>>>>>> --
>>>>>>> Yoshi
>>>>>>>
>>>>>>> On Mon, Jun 3, 2024 at 8:57 AM Neal Cardwell <[e-mail]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 3, 2024 at 11:02 AM Yoshifumi Nishida <
>>>>>>>> [e-mail]> wrote:
>>>>>>>>
>>>>>>>>> Hi folks,
>>>>>>>>>
>>>>>>>>> While I was checking RFC7323, I found the following sentence.
>>>>>>>>>
>>>>>>>>> RTTM update processing explicitly excludes segments not updating
>>>>>>>>> SND.UNA.  The original text could be interpreted to allow taking
>>>>>>>>> RTT samples when SACK acknowledges some new, non-continuous
>>>>>>>>> data.
>>>>>>>>>
>>>>>>>>> I am a bit curious about the rationale of this sentence.
>>>>>>>>> It seems to me that we cannot measure RTT when we have a gap in packet sequence with this rule.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Yes, that rule forbids using RFC7323 timestamps for calculating RTT
>>>>>>>> samples for SACKed sequence ranges.
>>>>>>>>
>>>>>>>> The rationale: AFAIK this rule is a necessary consequence of the
>>>>>>>> conditions under which TS.Recent is updated.
>>>>>>>>
>>>>>>>> The rules for updating TS.Recent are in sec 4.3, "Which Timestamp
>>>>>>>> to Echo":
>>>>>>>>   https://datatracker.ietf.org/doc/html/rfc7323#section-4.3
>>>>>>>> Rule (2) in sec 4.3 says:
>>>>>>>>   If:
>>>>>>>>     SEG.TSval >= TS.Recent and SEG.SEQ <= Last.ACK.sent
>>>>>>>>   then SEG.TSval is copied to TS.Recent; otherwise, it is ignored.
>>>>>>>>
>>>>>>>> Since out-of-order sequence ranges that are SACKed will fail the
>>>>>>>> SEG.SEQ <= Last.ACK.sent check, SACKed sequence ranges will not update
>>>>>>>> TS.Recent. So using TS.Recent to calculate an RTT sample for a SACKed
>>>>>>>> sequence range could, in general, give a vastly overestimated RTT sample.
>>>>>>>> So that's why it's forbidden by the RFC.
>>>>>>>>
>>>>>>>> However, in practice usually this does not need to be a big deal.
>>>>>>>> For example, Linux TCP still obtains an RTT sample for every
>>>>>>>> non-retransmitted SACKed sequence range, by:
>>>>>>>>
>>>>>>>> (a) recording the transmit time of every sequence range
>>>>>>>> (b) recording whether that sequence range was retransmitted, and
>>>>>>>> then
>>>>>>>> (c) using those two pieces of information when that sequence range
>>>>>>>> is cumulatively or selectively ACKed, to calculate an RTT sample
>>>>>>>> (rtt_sample = now - transmit_timestamp) if the sequence range was never
>>>>>>>> retransmitted.
>>>>>>>>
>>>>>>>> So, in Linux TCP, SACKed sequence ranges fail to generate an RTT
>>>>>>>> sample only when they were previously retransmitted.
>>>>>>>>
>>>>>>>> best regards,
>>>>>>>> neal
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> --
>>>>>>>>> Yoshi
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> tcpm mailing list -- [e-mail]
>>>>>>>>> To unsubscribe send an email to [e-mail]
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>> tcpm mailing list -- [e-mail]
>>>>>>> To unsubscribe send an email to [e-mail]
>>>>>>>
>>>>>>