Re: [tcpm] Hystart survey of large server operators

[Yuchung Cheng <[e-mail]>]

same experience with Chrisitian

For more than half a decade Google traffic used Linux CUBIC Hystart. This
includes google.com and YouTube public Internet traffic, as well as
internal Google TCP traffic. When we deployed pacing around 2013 we
disabled the Hystart ACK-train mechanism (that Hystart++ prohibits now) as
it causes false SS exit. But we continued to use the stock Hystart Delay
mechanism. ~2014-15 we switched to BBR which uses a different startup more
robust to delay jitters. But IMO the simpler hystart mitigating SS
over-shoot is still highly valuable for Cubic and other congestion
controls, as Internet bandwidth continues to hike.

Did those providers disable hystart b/c of its poor interaction between the
ack-train mode and pacing? Maybe it's time to disable the ack-train
approach in the upstream Linux when hystart++ is standardized.

On Wed, Jul 28, 2021 at 9:47 AM Christian Huitema <[e-mail]>
wrote:

> I certainly do not qualify as a "large operator", but when implementing
> Cubic for QUIC I found that Hystart was critical for performance.
> Specifically, the classical slow-start is very prone to overshooting the
> capacity of the path, which causes large batches of errors. With web-like
> traffic, these batches of errors cause increase latency for the first
> transactions in a session, and thus a drop in perceived quality. Hystart
> solves that. With Hystart, I observed that a large fraction of sessions do
> not experience any packet loss at all.
>
> As an aside, we should strive for this suppression of packet losses.
> That's actually a big reason for moving from Cubic to BBR. Hystart
> suppresses the losses during the initial phase of the connection, but Cubic
> still relies on periodically testing the limit of path capacity and causing
> losses during the subsequent phase. BBR's probing for bottleneck bandwidth
> is much more conservative, does not cause such losses. It might be possible
> to adapt Cubic to also not cause losses, for example by ending an epoch
> early if too many CE marks are received or if the RTT increases. That would
> be worth trying.
>
> -- Christian Huitema
> On 7/28/2021 3:19 AM, Bob Briscoe wrote:
>
> Yuchung,
>
> It was during a Mar 2018 ad hoc workshop Jana had organized at the London
> IETF entitled 'BBR and the intersection with other work". I can't remember
> why I needed to know at the time, but during the break I approached the
> major server operators individually, established whether they used Cubic,
> and if so asked whether they used Hystart or disabled it. It's hard to
> anonymize the results, because IMMSMC all those that used Cubic said they
> disabled Hystart.
>
> If this isn't correct, then maybe people who replied at the time thought
> they were being asked something else. Or maybe it was correct then but
> isn't now.
>
>
> Bob
>
>
> On 28/07/2021 00:46, Yuchung Cheng wrote:
>
> Wait -- how is hystart "invariably disabled" in Linux (cubic)!?
>
> What data indicates that
>
> On Tue, Jul 27, 2021 at 4:37 PM Bob Briscoe <[e-mail]
> <mailto:[e-mail]> <[e-mail]>> wrote:
>
>     Any large server operator out there who are using Cubic,
>     If you're willing to state whether or not your operations disable
>     Hystart, pls do so in reply.
>     Then Praveen can cite this mailing list thread in the Hystart++
>     draft, as requested below.
>
>     If you are willing to reply privately, I would be willing to keep
>     your confidences, and provide an anonymized result to the list.
>
>     Cheers
>
>
>
>     Bob
>
>     On 27/07/2021 01:50, Praveen Balasubramanian wrote:
>
>
>     Although Hystart is default enabled in Linux, it is invariably
>     disabled. So, it's misleading to just say Hystart is default
>     enabled, which implies it's widely used, when people clearly find
>     it has problems (which motivates Hystart++). I found this out
>     through an informal survey I did at the Mar'18 IETF in London by
>     asking round the implementers of the most prevalent stacks (I
>     would name names if I could find the note I later sent to someone
>     or to some list, but I can't find it - sry).
>
>
>     I'd like some citations on this versus anecdata if possible
>     before I add that caveat to the text. Do large deployments
>     disable this? I haven't come across this suggestion in any
>     Linux tuning guides to date.
>
>
>     --
> ________________________________________________________________
>     Bob Briscoehttp://bobbriscoe.net/  <http://bobbriscoe.net/>
> <http://bobbriscoe.net/>
>
>     _______________________________________________
>     tcpm mailing list
>     [e-mail] <mailto:[e-mail]> <[e-mail]>
>     https://www.ietf.org/mailman/listinfo/tcpm
>     <https://www.ietf.org/mailman/listinfo/tcpm>
> <https://www.ietf.org/mailman/listinfo/tcpm>
>
>
>
> _______________________________________________
> tcpm mailing [e-mail]://www.ietf.org/mailman/listinfo/tcpm
>
>