Re: [tcpm] [EXTERNAL] Full review of: draft-ietf-tcpm-hystartplusplus-02 (technical)

[Praveen Balasubramanian <[e-mail]>]

Thanks Bob for the review. We published draft-03 addressing some of your comments.

Inline prefixed by >>>>

From: Bob Briscoe <[e-mail]>
Sent: Thursday, July 22, 2021 8:42 AM
To: Praveen Balasubramanian <[e-mail]>
Cc: [e-mail]; [e-mail]
Subject: [EXTERNAL] Full review of: draft-ietf-tcpm-hystartplusplus-02 (technical)

Praveen and co-authors.

Thank you for hystart++.
This one's my technical review. I've sent my editorial review in a separate posting.

I actually reviewed an earlier draft (balasubrmanian-02), but never got round to writing it up for you. I've just reviewed the latest (ietf-02) draft, and incorporated my comments about the older revision where still relevant.

Cheers


Bob

===Technical Comments==

4.2.  Algorithm Details
________________________________



 Round can be approximated using sequence numbers as follows:

   ...

   At the start of each round during normal slow start and CSS:
It might be worth stating that using sequence numbers is the /best/ way to measure rounds, not just one way. You want every round to contain at least one of the ping-ponged segments that were grandfathered by the very first segment of the IW (because that will be the segment most likely to have the min RTT). For instance, if you measure rounds in time, you might underestimate the duration of a round. Then, in some rounds, you would risk not seeing one of this ping-pong series at all.

>>>> Added a recommendation for using the sequence number approach.
________________________________

"where currRTT is the RTT sampled from the incoming ACK"
Need to make it clear this RTT is based on the latest ACK'd packet, not the earliest (in contrast to the RTT used for RTO).

>>>> Clarified.
________________________________

The somewhat opaque clamping of RTTThresh below needs to be justified:

      For rounds where cwnd is at or higher than LOW_CWND and

      N_RTT_SAMPLE RTT samples have been obtained, check if delay

      increase triggers slow start exit

      ...

            RttThresh = clamp(MIN_RTT_THRESH, lastRoundMinRTT / 8, MAX_RTT_THRESH)

            if (currentRoundMinRTT >= (lastRoundMinRTT + RttThresh))

               cssBaselineMinRtt = currentRoundMinRTT

               exit slow start and enter CSS

To aid comprehension for the benefit of the list, I'll separate the clamp line into two steps and substitute the default constants:
            RttThresh = clamp(32ms, lastRoundMinRTT, 128ms)
            RttThresh /= 8

To me, it makes good sense without the clamping. In english, that would be just:
    "if queuing due to slow start increases the min RTT by more than 12.5% from one round to the next, exit slow start"

However, taking the default values of the constants, the clamping expression adds the following conditions:
#1) If the base RTT is lower than 32ms, the queue has to be greater than 4ms to exit slow start.
#2) If the base RTT is higher than 128ms, the queue only has to exceed 16ms to exit slow start.
I.e. at both extremes, the threshold becomes absolute, rather than a fraction of the base RTT.

Possible rationales for the lower condition when the base RTT is small (#1):
a) an absolute Qdelay threshold is needed because small delay measurements are too noisy.
b) a relative amount of Qdelay would become smaller than humans can perceive, so an absolute floor is sufficient.
If (a), this needs to be stated, because different min delay measurement techniques can achieve different noise immunity.
I don't agree with (b), because unnecessary delay over a sequence of rounds can accumulate to become perceivable.

I can't think of a good rationale for the higher condition (#2). I'm sure it was the result of experiments, but it seems to be likely that as base RTT increases beyond 128ms, Hystart++ would become increasingly sensitive and exit slow start earlier and earlier - unnecessarily (?).

Whatever, the rationale for clamping RttThresh needs to be stated, so admins know how to set the parameters for other scenarios (for instance, in DCs excessive loss would occur with the given defaults).

>>>> This is worth discussing in the WG meeting tomorrow time permitting. These are a result of experiments. These are fixed clamps because queue build up at the bottleneck has no direct correlation with the base RTT. I agree though that stating the justification is useful and we will look into this for the next revision. We are open to ideas to make these thresholds dynamic.
________________________________

5.  Deployments and Performance Evaluations

It would be useful to see testbed results separated by whether the bottleneck before the flow arrives is empty, full, or part-full of traffic.

>>>> This is a good suggestion and can be implemented by staggered start of multiple flows. We will incorporate these tests.

Although Hystart is default enabled in Linux, it is invariably disabled. So, it's misleading to just say Hystart is default enabled, which implies it's widely used, when people clearly find it has problems (which motivates Hystart++). I found this out through an informal survey I did at the Mar'18 IETF in London by asking round the implementers of the most prevalent stacks (I would name names if I could find the note I later sent to someone or to some list, but I can't find it - sry).

>>>> I'd like some citations on this versus anecdata if possible before I add that caveat to the text. Do large deployments disable this? I haven't come across this suggestion in any Linux tuning guides to date.

________________________________

Regards


Bob

On 12/07/2021 16:46, [e-mail]<mailto:[e-mail]> wrote:



A New Internet-Draft is available from the on-line Internet-Drafts directories.

This draft is a work item of the TCP Maintenance and Minor Extensions WG of the IETF.



        Title           : HyStart++: Modified Slow Start for TCP

        Authors         : Praveen Balasubramanian

                          Yi Huang

                          Matt Olson

  Filename        : draft-ietf-tcpm-hystartplusplus-02.txt

  Pages           : 8

  Date            : 2021-07-12



Abstract:

   This doument describes HyStart++, a simple modification to the slow

   start phase of TCP congestion control algorithms.  Traditional slow

   start can cause overshotting of the ideal send rate and cause large

   packet loss within a round-trip time which results in poor

   performance.  HyStart++ is composed of the delay increase variant of

   HyStart to prevent overshooting of the ideal sending rate, while also

   mitigating poor performance which can result from false positives.





The IETF datatracker status page for this draft is:

https://datatracker.ietf.org/doc/draft-ietf-tcpm-hystartplusplus/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-tcpm-hystartplusplus%2F&data=04%7C01%7Cpravb%40microsoft.com%7Cea743d922a3b42d4af8f08d94d274b02%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625654367629970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=wnozkJrwBmG2cbtSB39sg0vYr8cP8%2BBqRnn7bNh6lbM%3D&reserved=0>



There is also an htmlized version available at:

https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-hystartplusplus-02<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-tcpm-hystartplusplus-02&data=04%7C01%7Cpravb%40microsoft.com%7Cea743d922a3b42d4af8f08d94d274b02%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625654367639937%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=SOKqs1Y%2Bi5cNPnn4rv1jlP7rRR%2Fy1iFs9eFlMtxd2ic%3D&reserved=0>



A diff from the previous version is available at:

https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-hystartplusplus-02<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Frfcdiff%3Furl2%3Ddraft-ietf-tcpm-hystartplusplus-02&data=04%7C01%7Cpravb%40microsoft.com%7Cea743d922a3b42d4af8f08d94d274b02%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625654367639937%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=GlGu0C5KV8%2BdYbVeoJzOszS3YmDCnqf5L5GNBEK9eWY%3D&reserved=0>





Internet-Drafts are also available by anonymous FTP at:

ftp://ftp.ietf.org/internet-drafts/<https://nam06.safelinks.protection.outlook.com/?url=ftp%3A%2F%2Fftp.ietf.org%2Finternet-drafts%2F&data=04%7C01%7Cpravb%40microsoft.com%7Cea743d922a3b42d4af8f08d94d274b02%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625654367649886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=ENURbF%2FuD14Os8t2xQ9oNQVCDITagh%2FAfznG30S%2BjTw%3D&reserved=0>





_______________________________________________

tcpm mailing list

[e-mail]<mailto:[e-mail]>

https://www.ietf.org/mailman/listinfo/tcpm<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cea743d922a3b42d4af8f08d94d274b02%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625654367649886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=pXpAocP3fib4kNDsuwgDZAqljB0TUWWJQB4RDWSkT3U%3D&reserved=0>



--

________________________________________________________________

Bob Briscoe                               http://bobbriscoe.net/<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbobbriscoe.net%2F&data=04%7C01%7Cpravb%40microsoft.com%7Cea743d922a3b42d4af8f08d94d274b02%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625654367649886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=rRojz5MR9vnVizaQBeqy%2FLdUifSnt%2FjBAD1TBXuukLA%3D&reserved=0>