Re: [tcpm] CUBIC rfc8312bis / WGLC Issue 2

[Yoshifumi Nishida <[e-mail]>]

Hi folks,
I think I understand this issue, but I'm personally not sure how bad this
is.
Because this looks a rather pathological case to me, also I don't think
this can cause congestion collapse as this is still multicative decrease.
It seems to me that this is a kind of shooting in the foot, a suboptimal
case. However,  there are some advantages in the current logic.
I'm not very sure if we should sacrifice better results to address some
rare cases. I think we will need more analysis of the pros and cons for
this.

Thanks,
--
Yoshi

On Wed, Jul 13, 2022 at 7:17 AM Neal Cardwell <[e-mail]> wrote:

> Hi Markku and TCPMers,
>
> My understanding of Markku's concern here is that in slow start the cwnd
> can continue to grow in response to ACKs after the lost packet was sent, so
> that the cwnd is often twice the level of in-flight data at which the loss
> happened, by the time the loss is detected. So the cwnd ends up at 2 * 0.7
> = 1.4x the level at which losses happened, which causes an unnecessary
> follow-on round with losses, in order to again cut the cwnd, this time
> to 1.4 * 0.7 = 0.98x of the level that causes losses, which is likely to
> finally fit in the network path.
>
> However, there are two technical issues with this concern, as expressed in
> the proposed draft text in this thread:
>
> (1) The analysis for slow-start is not correct for the very common case
> where the flow is application-limited in slow-start, in which case the cwnd
> would not grow at all between the packet loss and the time the loss is
> detected. So the text is needlessly strict in this case.
>
> (2) For CUBIC the problematic dynamic (of cwnd growth between loss and
> loss detection exceeding the multiplicative decrease) can also occur
> outside of slow-start, in congestion avoidance. The CUBIC cwnd growth in
> congestion avoidance can be up to 1.5x per round trip. So after a packet
> loss the cwnd could grow by 1.5x before loss detection and then be cut in
> response to loss by 0.7, causing the ultimate cwnd to be 1.5 * 0.7 = 1.05x
> the volume of in-flight data at the time of the packet loss. This would
> likely cause an unnecessary follow-on round of packet loss due to failing
> to cut cwnd below the level that caused loss. So the problem is actually
> wider than slow-start.
>
> AFAICT a complete/general fix for this issue is best solved by recording
> the volume of inflight data at the point of each packet transmission, and
> then using that metric as the baseline for the multiplicative decrease when
> packet loss is detected, rather than using the current cwnd as the
> baseline. This is the approach that BBRv2 uses. Perhaps there are other,
> simpler approaches as well.
>
> I also agree with Vidhi's concern, that a change to the multiplicative
> decrease changes the algorithm substantially. To ensure that the draft/RFC
> is not recommending something that has unforeseen significant negative
> consequences, we shouldn't make such a significant change to the text until
> we get experience w/ the new variation.
>
> best regards,
> neal
>
>
> On Tue, Jul 12, 2022 at 6:08 PM Vidhi Goel <vidhi_goel=
> [e-mail]> wrote:
>
>> Hi Markku,
>>
>> I emailed about this to other co-authors and we think that this change is
>> completely untested for Cubic and we think that this could be considered of
>> a future version of Cubic, not the current rfc8312bis.
>> To change Beta from 0.7 to 0.5 during slow-start, we would at least need
>> some experience either from lab testing or deployment since all current
>> deployments of Cubic for both TCP and QUIC use 0.7 as Beta during slow
>> start. Since a lot of implementations currently use hystart(++) along with
>> Cubic, we don’t see any high risk of overaggressive sending rate and that
>> is what the current rfc8312bis suggests as well. In fact, changing Beta
>> from 0.7 to 0.5 can still be aggressive without using hystart.
>>
>> Thanks,
>> Vidhi
>>
>> > On Jul 11, 2022, at 5:55 PM, Markku Kojo <kojo=
>> [e-mail]> wrote:
>> >
>> > Hi all,
>> >
>> > below please find proposed text to solve the Issue 2 a). I will propose
>> text to solve 2 b) once we have come to conclusion with 2 a). For
>> description and arguments for issues 2 a) and 2 b), please see the original
>> issue descriptions below.
>> >
>> > Sec 4.6. Multiplicative Decrease
>> >
>> > Old:
>> >   The parameter Beta__cubic_ SHOULD be set to 0.7, which is different
>> >   from the multiplicative decrease factor used in [RFC5681] (and
>> >   [RFC6675]) during fast recovery.
>> >
>> >
>> > New:
>> >   If the sender is not in slow start when the congestion event is
>> >   detected, the parameter Beta__cubic_ SHOULD be set to 0.7, which
>> >   is different from the multiplicative decrease factor used in
>> >   [RFC5681] (and [RFC6675].
>> >   This change is justified in the Reno-friendly region during
>> >   congestion avoidance because a CUBIC sender compensates the higher
>> >   multiplicative decrease factor than that of Reno by applying
>> >   a lower additive increase factor during congestion avoidance.
>> >
>> >   However, if the sender is in slow start when the congestion event is
>> >   detected, the parameter Beta__cubic_ MUST be set to 0.5 [Jacob88].
>> >   This results in the sender continuing to transmit data at the maximum
>> >   rate that the slow start determined to be available for the flow.
>> >   Using Beta__cubic_ with a value larger than 0.5 when the congestion
>> >   event is detected in slow start would result in an overagressive send
>> >   rate where the sender injects excess packets into the network and
>> >   each such packet is guaranteed to be dropped or force a packet from
>> >   a competing flow to be dropped at a tail-drop bottleneck router.
>> >   Furthermore, injecting such undelivered packets creates a danger of
>> >   congestion collapse (of some degree) "by delivering packets through
>> >   the network that are dropped before reaching their ultimate
>> >   destination." [RFC 2914]
>> >
>> >
>> >   [Jacob88] V. Jacobson, Congestion avoidance and control, SIGCOMM '88.
>> >
>> > Thanks,
>> >
>> > /Markku
>> >
>> > On Tue, 14 Jun 2022, Markku Kojo wrote:
>> >
>> >> Hi all,
>> >>
>> >> this thread starts the discussion on the issue 2: CUBIC is specified
>> to use incorrect multiplicative-decrease factor for a congestion event that
>> occurs when operating in slow start. And, applying HyStart++ does not
>> remove the problem, it only mitigates it in some percentage of cases.
>> >>
>> >> I think it is useful to discuss this in two phases: 2 a) and 2 b)
>> below.
>> >> For anyone commenting/arguing on the part 2 b), it is important to
>> first
>> >> acknowledge whether (s)he thinks the original design and logic by Van
>> Jacobson is correct. If not, one should explain why Van's design logic is
>> incorrect.
>> >>
>> >> Issue 2 a)
>> >> ----------
>> >>
>> >> To begin with, let's but aside a potential use of HyStart++ (also
>> assume tail drop router unless otherwise mentioned).
>> >>
>> >> The use of an MD factor larger than 0.5 is against the theory and
>> original design by Van Jacobson as explained in the congavoid paper
>> [Jacob88]. Any MD factor value larger then 0.5 will result sending extra
>> packets during Fast Recovery following the congestion event (drop). All
>> extra packets will become dropped at a tail-drop bottleneck (if a lonely
>> flow).
>> >>
>> >> Note that at the time when the drop becomes signalled at the TCP
>> sender, the size of the cwnd is double the available network capacity that
>> slow start determined for the flow. That is, using MD=0.5 is already as
>> aggressive as possible, leaving no slack. Therefore, if MD=0.7 is used, the
>> TCP sender enters fast recovery with cwnd that is 40% larger that the
>> determined network capacity and all excess packets are guaranteed to become
>> dropped, or even worse, the excess packets are likely to force packets for
>> any competing flows to become unfairly be dropped.
>> >>
>> >> Moreover, if NewReno loss recovery is in use, a CUBIC sender will
>> >> operate overagressively for a very long time. For example, if the
>> >> available network capacity for the flow is 100 packets, cwnd will have
>> >> value 200 when the congestion is signalled and the CUBIC sender enters
>> >> fast recovery with cwnd=140 and injects 40 excess packets for each of
>> >> the subsequent 100 RTTs it stays in fast recovery, forcing 4000
>> packets to become inevitably and totally unnecessarily dropped.
>> >>
>> >> Even worse, this behaviour of sending 'undelivered packets' is against
>> >> the congestion control principles as it creates a danger of congestion
>> >> collapse (of some degree) "by delivering packets through the network
>> >> that are dropped before reaching their ultimate destination." [RFC
>> 2914]
>> >>
>> >> Such undelivered packets unnecessarily eat capacity from other flows
>> >> sharing the path before the bottleneck.
>> >>
>> >> RFC 2914 emphasises:
>> >>
>> >> "This is probably the largest unresolved danger with respect to
>> >> congestion collapse in the Internet today."
>> >>
>> >> It is very easy to envision a realistic network setup where this
>> creates a degree of congestion collapse where a notable portion of useful
>> network capacity is wasted due to the undelivered packets.
>> >>
>> >>
>> >> [Jacob88] V. Jacobson, Congestion avoidance and control, SIGCOMM '88.
>> >>
>> >>
>> >> Issue 2 b)
>> >> ----------
>> >>
>> >> The CUBIC draft suggests that HyStart++ should be used *everywhere*
>> instead of the traditional Slow Start (see section 4.10).
>> >>
>> >> Although the draft does not say it, seemingly the authors suggest
>> using HyStart++ instead of traditional Slow Start in order to avoid the
>> problem of over-aggressive behaviour discussed above. This, however, has
>> several issues.
>> >>
>> >> First. it is directly in conflict with HyStart++ specification which
>> says that HyStart++ should be used only for the initial Slow Start.
>> However, the overaggressive behaviour after slow start is also a potential
>> problem with slow start during an RTO recovery; in case of sudden
>> congestion that reduces available capacity for a flow down to a fraction of
>> the currently available capacity, it is very likely that an RTO occurs. In
>> such a case the RTO recovery in slow start inevitably overshoots and it is
>> crucial for all flows not to be overaggressive.
>> >>
>> >> Second, the experimental results for initial slow start in HyStart++
>> draft suggest that while HyStart++ achieves good results HyStart++ is
>> unable to exit slow start early and avoid overshoot in a significant
>> percentage of cases.
>> >>
>> >> Given the above issues, the CUBIC draft must require that MD of 0.5 is
>> used when the congestion event occurs while the sender is (still) in slow
>> start. The use of MD=0.5 is an obvious stumble in the original CUBIC and
>> the original CUBIC authors have already acknowledged this. It seems also
>> obvious that instead of correcting the actual problem (use of MD other than
>> 0.5), HyStart and HyStart++ have been proposed to address the design
>> mistake. While HyStart++ is a useful method also when used with MD=0.5,
>> when used alone it only mitigates the impact of the actual problem rather
>> than solves the problem.
>> >>
>> >> What should be done for the cases where HyStart++ exits slow start but
>> >> is not able to avoid (some level of) overshoot and dropped packets is
>> IMO an open issue. Resolving it requires additional experiments and it
>> should be resolved separately when we have more data. For now when we do
>> not have enough data and understanding of the behaviour we should IMO
>> follow the general IETF guideline "be conservative in what you send" and
>> specify that MD = 0.5 should be used for a congestion event that occurs for
>> a packet sent in slow start.
>> >>
>> >> Thanks,
>> >>
>> >> /Markku
>> >>
>> >
>> > _______________________________________________
>> > tcpm mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/tcpm
>>
>> _______________________________________________
>> tcpm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>