Re: [tcpm] Proceeding CUBIC draft - thoughts and late follow-up

[Markku Kojo <[e-mail]>]

Hi Yoshi,

On Wed, 20 Jul 2022, Yoshifumi Nishida wrote:

> Hi Markku,
> 
> Sorry for not being very clear.
> What I meant was even though Reno increases cwnd every RTT and Cubic increases it every other RTT (which can
> indicate the presumption of the model is not very accurate), I think it doesn't necessarily mean Reno has
> higher packet loss rate than Cubic. 

First, the original mathematic model has been derived to be exact as 
this kind mathematical models typically are. So, the problem is not that 
it is not very accurate, it is simply incorrect as the logic behind the 
model does not match with the reality (of deterministic network behaviour 
that basically all TCP models are based on).

The original congestion control design and logic by Van Jacobson is based 
on the pkt conservation principle in deterministic network conditions. 
AFAIK (almost) all mathemathical TCP models are also based on the same 
assumption, including the model in [FHP00] that CUBIC's Reno-friendly 
operation is based on. That model assumes that both Reno and creno 
*inject an additional pkt every RTT* (definitely both Reno and creno 
increase cwnd every RTT but that is not the key). According to the pkt 
conservation principle a flow that continues with the same rate, that is, 
does not inject an additional pkt to the network won't encounter/incur a 
pkt loss as it won't send more than the network can buffer. Of course, in 
real networks this does not hold always because not all nets have such 
exactly deterministic behaviour, but the point is that the mathemetical
basis for the model is incorrect and as a result there definitely are a 
significant number of network paths in the Internet that have such a 
deterministic behaviour, ensuring the problem shows up somewhere as 
described; CC algos are required to operate correctly and be evaluated in 
a wide range of environments exactly because they basically need to 
operate correctly in any network conditions.

Having an algo that is based on an incorrect and unvalidated model in a 
standards track document is problematic, isn't it?

Second, packet drop rate means distance between drops. If the incorrect 
model results in creno not incurring a pkt drop when it is supposed to 
happen then the distance between drops automatically increases, meaning 
that pkt drop rate decreases for creno from what was assumed.

> They might not be equal, but the gaps might not be that big. Well, I'm not sure at this point. 

The difference is likely to depend on net conditions. In deterministic 
conditions it is quite easy to see that the bias is systematic and 
relatively significant. IMHO, it is very problematic if we don't 
understand how creno actually operates compared to Reno. It is like 
basing the design on somewhat random operation. Or at least we don't know 
what is expected behaviour and performance is. How can such algo be even 
considered as the new CC base as some have wished to happen?

> So, while I agree this should not be required, I still believe we'll need more analysis on this. 
> I think it'll be difficult to think about the best way to fix it until we understand the details of the issue

Certainly more analysis in needed to understand it. When we don't know, 
the old IETF "rule" should be applied: "be concervative in what you send".

That said, a reasonable solution for a standards track document would be 
specifying conservatively that in the TCP-friendly region CUBIC uses Reno 
unmodified (and the model-based creno is allowed for experimental use 
only, if a stds track doc could say something like this). After all the 
only difference in the TCP-friendly region is the lower saw tooth due to 
Beta=0.7 and alpha 0.53 but in the current Internet AFAIK we have few 
such shallow bottleneck buffers where creno shallower saw tooth is really 
needed; AFAIK we still suffer buffer bloat widely in the Internet. The 
shallower sawtooth is needed for efficient operation in high BDP 
environments, i.e., with the genuine CUBIC mode.

In the future, yes, shallower sawtooth is desireable, but by then we may 
well have an appropriate solution that does not need to be based on any 
mathematical model mimicking Reno CC. We may simply start working on 
modifying RFC 5681 towards a shallower sawtooth by defining somewhat 
larger Beta when operating in the "normal" CA cycle but keeping Beta=0.5 
in the other CC phases.

Thanks,

/Markku

> --
> Yoshi
> 
> On Tue, Jul 19, 2022 at 4:56 PM Markku Kojo <[e-mail]> wrote:
>       Hi Yoshi,
>
>       See below inline.
>
>       On Thu, 14 Jul 2022, Yoshifumi Nishida wrote:
>
>       > Hi Markku,
>       >
>       > Thanks for the comments. I think we're mostly on the same page for RFC9002. 
>       > I put some comments on the issue 1 part.
>       >
>       > On Mon, Jul 11, 2022 at 7:54 PM Markku Kojo <[e-mail]> wrote:
>       >
>       >       > BTW, my intention is not saying RFC9002 is overly aggressive. I just meant comparing
>       with reno
>       >       precisely might
>       >       > not be a very good idea for modern networks. 
>       >
>       >       Agreed, it is not overly aggressive.
>       >
>       >       I am not saying that CUBIC should be precisely as aggressive as Reno even
>       >       though its design goal has been to be equally aggressive in the
>       >       Reno-friendly region.
>       >
>       >       Instead, with the issue 1, for example, a CUBIC flow competing with a
>       >       Reno flow will incorrectly and against its design goal opt out
>       >       roughly every second cwnd decrease that the Reno flow executes. Doesn't
>       >       that result in systematic and significant difference in peformance like
>       >       the measuement results that I've pointed to also confirm?
>       >
>       > I think we'll need more analysis on this point. 
>       > Let's say a CUBIC connection has 50 packets cwnd and a Reno connection has 51 packets cwnd, the
>       available
>       > buffer is only 100 packets.
>       > In this case, I'm not very sure which connection of packets will be dropped hence not sure about
>       the
>       > consequences.
>
>       I am not quite sure what you are looking after here? It does not matter
>       what are the cwnd sizes. They may differ. What matters is what happens on
>       the RTT when the buffer capacity has been exhausted and a flow (or flows)
>       injects an additional packet once cwnd has increased by one full MSS.
>       For Reno this occurs every RTT but for CUBIC (creno = CUBIC in the
>       Reno-friendly region) roughly every second RTT (for simplicity I use here
>       Beta = 0.5 for CUBIC). When I say creno opts out (roughly) every second
>       cwnd decrease that is true when the flows are synchronized. When they are
>       not, things are a bit more complex and I didn't want to go there for
>       simplicity and because for that we would need more analysis and
>       measurements to learn the exact impact of the incorrect formula.
>       In addition, we shold not consider only the case of two competing flows
>       but any number with different proportion of Reno and creno.
>       I used the simple two-flow case to illustrate why the model (and formula)
>       are incorrect.
>
>       Of course also two Reno flows may become desynchronized (and they do in
>       real networks) and then their cwnds are not the same each CA cycle but at
>       times only one of the flows gets a drop and continues with a smaller
>       cwnd. However, this happens randomly to either of the flows and there is
>       no systematic bias. And, when it happens the flow with the smaller cwnd
>       (statistically) catches up with the larger cwnd in some number of CA
>       cycles. creno that increases its cwnd by full MSS only every second RTT
>       creates a systematic bias that results in unjustified performance gain
>       because of systematically not being candidate for a pkt drop each time
>       the buffer becomes full. I must also say that the problem is present in
>       full as described in deterministic network environments but it does not
>       become negligible when the network does not behave in such deterministic
>       way due to varying pkt timings. Nonetheless, we have plenty of
>       deterministic enough network environments today and the CC algos must work
>       correctly also in these environments.
>
>       > I still believe we cannot be sure until we see some deep analysis for this.
>
>       The problem is true but the exact impact is unknown indeed.
>
>       > This may take time, but I think it's necessary to get the conclusion on this point. 
>
>       Whatever the conclusion is, it must not ignore the problem.
>
>       Thanks,
>
>       /Markku
>
>       > --
>       > Yoshi  
>       >
>       >
> 
> 
>