Re: [tcpm] [EXTERNAL] Re: Hystart and delay jitter

[Christian Huitema <[e-mail]>]

Back to Ingemar's point about delay measurement. In wireless networks, 
we should expect conditions to change when the device moves from cell to 
cell, when the number of devices in the cell changes, or even when the 
client stays in the same cell and moves closer or farther to the base 
station. In Wi-Fi network, I remember presentations showing that the 
quality of the radio signal could change radically if the device just 
moved a couple feet, maybe because it would then catch different 
reflections on walls of the signals sent by the base station. Conditions 
could also change if the device is rotated and the alignment of the 
device's antenna with the base station changes. Or if a big two-legged 
of water moves between device and base station, whether it is a big 
two-legged body or a smaller four-legged body with whiskers.

The congestion control algorithms that we currently use are not 
specifically designed for such changes in network conditions. They make 
hypotheses about min-rtt, max queue sizes, relation between rtt and 
congestion, or relative stability of the bottleneck bandwidth. As 
Ingemar points out, these hypotheses are not true in wireless networks.

If I look at simulations in classic papers, I almost always see networks 
made of static links, with fixed bandwidth and fixed delays. Such 
simulations are not going to point out the issues with wireless 
networks. Are there available link models that incorporate the typical 
behaviors of wireless networks? Better simulations would make the 
problems obvious, and researchers would probably invent some clever way 
of dealing with that. And if such simulation models were widely known, 
peer-reviewers would have something to say about papers based on fixed 
link simulations.

Ingemar, Mirja, can you point us to such to simulation models for 
wireless segments?

-- Christian Huitema

On 3/22/2021 8:17 AM, Mirja Kuehlewind wrote:
> Sorry, I was confused and thought this was a comment on the Cubic work. Would be really good to mention and discuss this in HyStart++ draft. Happy to discuss more if needed!
>
>
> On 22.03.21, 16:12, "tcpm on behalf of Mirja Kuehlewind" <[e-mail] on behalf of mirja.kuehlewind=[e-mail]> wrote:
>
>      Yes, we also reported on some problems/bugs in HyStart in maprg two meetings back:
>
>      https://datatracker.ietf.org/meeting/interim-2020-maprg-01/materials/slides-interim-2020-maprg-01-sessa-behavior-of-tcp-cubic-in-low-latency-mobile-radio-networks-00
>
>      This problem, where Slow Start is left wrongly and much too early, is particularly bad in combination with the TCP friendly region in Cubic. However, the actually problems lies in HyStart which is not covered in the Cubic RFC/bis-draft.
>
>
>      On 19.03.21, 10:16, "tcpm on behalf of Ingemar Johansson S" <[e-mail] on behalf of ingemar.s.johansson=[e-mail]> wrote:
>
>          Hi
>
>          My finding is that it is a bit problematic to rely on RTT estimates for path
>          capacity especially with cellular access . It was summarized in section 4 in
>          https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-02 and also
>          presented in
>          https://www.ietf.org/proceedings/96/slides/slides-96-iccrg-1.pdf
>
>          Regards
>          Ingemar
>
>          > -----Original Message-----
>          > From: tcpm <[e-mail]> On Behalf Of Praveen
>          > Balasubramanian
>          > Sent: den 18 mars 2021 23:37
>          > To: ncardwell=[e-mail]; Christian Huitema
>          > <[e-mail]>
>          > Cc: [e-mail]
>          > Subject: Re: [tcpm] [EXTERNAL] Re: Hystart and delay jitter
>          >
>          > Thanks Christian for the suggestions! We will include a discussion on
>          jitter in
>          > the next draft update and also experiment with the min of max function.
>          >
>          > -----Original Message-----
>          > From: tcpm <[e-mail]> On Behalf Of Neal Cardwell
>          > Sent: Friday, March 12, 2021 2:18 PM
>          > To: Christian Huitema <[e-mail]>
>          > Cc: [e-mail] Extensions <[e-mail]>
>          > Subject: [EXTERNAL] Re: [tcpm] Hystart and delay jitter
>          >
>          > On Fri, Mar 12, 2021 at 4:29 PM Christian Huitema <[e-mail]>
>          > wrote:
>          > >
>          > > Back in November 2019, when adding Cubic and Hystart to my
>          > > implementation of QUIC, I noticed that Hystart was sensitive to delay
>          > > jitter. Hystart detects the buildup of queues by monitoring the RTT.
>          > > Some links experience delay jitter, caused for example by access
>          > > protocol for shared radio links or possibly by link-local ARQ protocols.
>          > > The delay jitter can cause Hystart to make the wrong decision, in two
>          ways:
>          > >
>          > > 1) Delay jitter during a previous period could cause some packets to
>          > > be delivered "faster than usual", causing Hystart to under-estimate
>          > > the min RTT for that period.
>          > >
>          > > 2) Delay jitter during the currently measured period can cause packets
>          > > to be delivered "slower than usual",  causing Hystart to over-estimate
>          > > the min RTT for that period.
>          > >
>          > > The combination of these two issues may cause Hystart to make the
>          > > wrong decisions, and exit slow start at levels well below link capacity.
>          >
>          > Yes, we have found in both our production experience and controlled
>          > experiments that the Hystart-Delay algorithm is very susceptible to
>          spurious
>          > triggering from jitter, particularly in LTE and wifi paths.
>          >
>          > We discussed this a bit in Spring 2017 in the comparison of BBR's
>          bandwidth-
>          > based mechanism for exiting startup, vs Hystart-Delay's delay-based
>          > mechanism:
>          >   https://protect2.fireeye.com/v1/url?k=6c35f204-33aecb47-6c35b29f-
>          > 861fcb972bfc-c8f94340e007b7bd&q=1&e=fd13218f-2e70-4b39-a879-
>          > e9b53c01447d&u=https%3A%2F%2Fnam06.safelinks.protection.outlook.com
>          > %2F%3Furl%3Dhttps%253A%252F%252Fwww.ietf.org%252Fproceedings%25
>          > 2F98%252Fslides%252Fslides-98-iccrg-an-update-on-bbr-congestion-control-
>          > 00.pdf%2523page%253D8%26amp%3Bdata%3D04%257C01%257Cpravb%2540
>          > microsoft.com%257Cbc3399870c0d4105299408d8e5a4cbaa%257C72f988bf86f
>          > 141af91ab2d7cd011db47%257C1%257C0%257C637511843745826011%257CUn
>          > known%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
>          > iI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3D6SQWDb1
>          > mFH0cgrayEwiRYeccYvP0dv%252BAJ1eokyJF%252BKs%253D%26amp%3Brese
>          > rved%3D0
>          >
>          > > The draft-ietf-tcpm-hystartplusplus-01 does have some protection
>          > > against the second issue, because currentRoundMinRTT is computed on at
>          > > least N_RTT_SAMPLE. If that number is large enough, computing the min
>          > > over N samples should filter out "slower than usual" anomalies.
>          > > However, the draft does not include a protection against "faster than
>          > usual"
>          > > anomalies happening in the previous period. In my implementation, I
>          > > protected against that by computing a "min of max" function: compute a
>          > > rolling "MAX over N_RTT_SAMPLE", then compute the MIN value of that
>          > > during the reference period, and use that to set the reference value
>          > > "lastRoundMinRTT".
>          > >
>          > > I think it would be good to add a discussion of the effect of jitter
>          > > to the hystart++ draft. In addition, we may also want to mention
>          > > timestamps. The jitter on RTT may be caused by jitter on either
>          > > direction of transmission -- data path or ACK path. The effect of
>          > > jitter on the ACK path can be minimized if time stamps can be used to
>          > > monitor the variation of one-way delays. This is not discussed in the
>          > > current draft. Maybe it should be.
>          >
>          > A discussion of the effect of jitter sounds like a great idea.
>          >
>          > neal
>          >
>          > _______________________________________________
>          > tcpm mailing list
>          > [e-mail]
>          > https://protect2.fireeye.com/v1/url?k=032901f3-5cb238b0-03294168-
>          > 861fcb972bfc-7a6ccb41bd90a846&q=1&e=fd13218f-2e70-4b39-a879-
>          > e9b53c01447d&u=https%3A%2F%2Fnam06.safelinks.protection.outlook.com
>          > %2F%3Furl%3Dhttps%253A%252F%252Fwww.ietf.org%252Fmailman%252Flis
>          > tinfo%252Ftcpm%26amp%3Bdata%3D04%257C01%257Cpravb%2540microsof
>          > t.com%257Cbc3399870c0d4105299408d8e5a4cbaa%257C72f988bf86f141af91a
>          > b2d7cd011db47%257C1%257C0%257C637511843745826011%257CUnknown%
>          > 257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
>          > WwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3Doqte6wCaTBUvrw
>          > gXVRMcD13YgtspMJyIWMMLlwNuYdY%253D%26amp%3Breserved%3D0
>          >
>          > _______________________________________________
>          > tcpm mailing list
>          > [e-mail]
>          > https://www.ietf.org/mailman/listinfo/tcpm
>
>      _______________________________________________
>      tcpm mailing list
>      [e-mail]
>      https://www.ietf.org/mailman/listinfo/tcpm
>
> _______________________________________________
> tcpm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/tcpm