Re: [tcpm] CUBIC rfc8312bis / WGLC Issue 1 (Was: Re: ProceedingCUBIC draft - thoughts and late follow-up)

[Markku Kojo <[e-mail]>]

Hi Yoshi,

I moved your comment and the discussion on your reply under this thread on 
the Issue 1 (see below)

On Tue, 14 Jun 2022, Markku Kojo wrote:

> Hi all,
>
> this thread starts the discussion on the issue 1: the incorrect model for 
> determining CUBIC alpha for the congestion avoidance (CA) phase (Issue 1 a) 
> and the inadequate validation of a proper constant C for the CUBIC window 
> increase function (Issue 1 b).
>
>
> Issue 1 a)
> ----------
>
> The model that CUBIC uses to be fair to Reno CC (in Reno-friendly region) is 
> unvalidated and actually incorrect.
>
> A more detailed description of the issue:
>
> The original paper manuscript that CUBIC bases its behaviour in the 
> Reno-friendly region did a preliminary attempt to validate the model but 
> failed (and the paper never got published). This is the only known attempt to 
> validate the model and even this failed validation attempt was quite light, 
> consisting of only a couple of network settings and obviously did not use any 
> replications for the results shown in the paper. Hence, even the statistical 
> validity of the results remains questionable. Results were shown only for a 
> setting with AQM enabled at the bottleneck router. The results for a 
> tail-drop case are missing in the paper manuscript.
>
> The report (creno.pdf, see a pointer to the doc in the email pointed to 
> below) that Bob wrote provides some explanation why the model does not give 
> correct results and thereby the resulting behaviour presented in the original 
> paper notably deviates from that of Reno CC. The email that I wrote to the wg 
> list
>
> https://mailarchive.ietf.org/arch/msg/tcpm/bds-h_a6-NliTjx-ZqUSaFpSSnA/
>
> complements Bob's explanation for the AQM case and corrects Bob's analysis 
> for the tail-drop case, explaining why the model is incorrect for the 
> traditional and still today prevailing tail-drop router case.
>
> Consequently, the use of the incorect model results in unknown behaviour of 
> CUBIC when in the Reno-friendly region. Moreover, it is quite likely that the 
> behaviour is different with different AQM implementations at the bottleneck, 
> resulting in even more random behavior. This alone is very problematic and 
> becomes more problematic when considering how moving out from the 
> Reno-friendly region is specified: when the genuine CUBIC formula gives a 
> larger cwnd than the cwnd that the Reno-friendly model gives, CUBIC moves to 
> the genuine CUBIC mode that is significantly more aggressive than Reno CC.
>
> Therefore, if the incorrect model gives too low cwnd for mimicked Reno CC, 
> CUBIC moves too early to the genuine CUBIC mode and becomes too agggressive 
> too early even though it should behave equally aggressive as Reno CC. On the 
> other hand, if the incorrect model gives too large cwnd, CUBIC is too 
> aggressive throughout the Reno-friendly region.
> In summary, if the model is not correct, it results in more aggressive 
> behaviour than Reno CC no matter which direction the model fails.
>
> And very importantly: some people have suggested that CUBIC should replace 
> the current stds track CC algos and become the default. The behaviour of Reno 
> CC is very thoroughly studied and very well understood. If we replace it with 
> *unknown* behaviour, how can we anymore specify what is the correct and 
> allowed aggressiveness for any upcoming CC when the behaviour of the new 
> default itself is unknown, making comparative analysis of other CCs against 
> CUBIC in the Reno-frindly region very difficult? The behaviour is assumed to 
> be the same as Reno CC but the actual behaviour is random, it may be 2 times 
> or 8 times more aggressive than Reno, for example.


On Tue, 7 Jun 2022, Yoshifumi Nishida wrote:

> Hi Markku,
>
> Thanks for the detailed feedback. This is very useful.
> One thing I would like to clarify is that we’ve already acknowledged the
> TCP friendly  model in the draft has some unsolved discussions. But, I
> believe our > current > consensus is to not change the logics for it in
> the current draft as it will require long term evaluations.
>
> So, I would like to check if you’re suggesting we should update the
> draft against it or you have some ideas to address these issues in
> some ways (e.g adding more clarification in the draft, mentioning it in 
> the write-up, etc)
> 
> Thanks,
> --
> Yoshi

I think the problem is even trickier because it is hard to see how it 
would be possible to correct the model that is based on wrong 
assumptions. This said, it is important for the wg to consider whether it 
is ready to suggest publishing a congestion control algorithm that is not 
correct and has not been validated. And, if the answer is yes, how to 
justify it and what would be the appropriate status for the RFC as well 
as the way forward after publishing the draft.

I fully symphatize those who have deployed CUBIC and understand that 
there is a pressure to publish the draft with no modifications to what has 
been implemented. However, RFC 5033 was written specifically to avoid this 
kind of situation where an CC algo has been (widely) deployed and only 
then brought to IETF standardization. It is understandable that those who 
have deployed the CC algo would be very reluctant to modify the algo. On 
the other hand, AFAIK all current stds track CC algos have had various 
issues that have been brought up during the standardization process but 
these issues have been resolved before publishing the draft. So, why we 
should make an exception? IMO, wide deployment cannot be the answer 
because it does not automatically reveal the negative impact to other 
traffic but specific comparative measurements must be carried out.
Also, why should the IETF set a precedent for any future congestion 
control drafts, implying that it is ok to first deploy a CC algo and then 
bring it to IETF and use the (wide) deployment as an argument against 
modifying it regardless of whatever issues it might have?

So, I don't have a good answer. IMO, if the draft is published with 
unresolved issues, the draft itself must clearly identify and document 
the issues and give some kind of justification and a clear way forward. 
That is, we must ensure there is an initiative set and path to follow in 
order to correct any shortcomings in a published RFC. Otherwise, the 
issues are very likely ignored and forgotten forever.

Thanks,

/Markku



> Issue 1 b)
> ----------
>
> Another issue related to the operating in the Reno-friendly region is the 
> question when CUBIC should operate in the Reno-friendly region and when it 
> may move out of it. Obviously CUBIC should stay in the Reno-friendly region 
> when Reno CC would be able to fully utilize the available network capacity. 
> In practice, this is specified by selecting the value for constant C in the 
> formula that is used to determine cwnd in the "genuine" CUBIC mode. However, 
> selecting a proper value for C has not been properly validated in a wide 
> range of environments as required in RFC 5033.
>
> Preliminary validation of constant C has been done for the original CUBIC 
> paper. That is good enough for a scientific paper but not adequate for an 
> IETF stds track algo. There seems to be no additional evaluation since the 
> timeframe of the CUBIC paper publication around 15 years ago. Particularly, 
> there seems to be no evaluation with AQM at the bottleneck router or with a 
> buffer-bloated bottleneck router, not to mention many other network 
> environments. Nor is there any data available for a non-SACK TCP sender.
>
> The evaluation of 1 a) and 1 b) must be done separately. Othserwise, it is 
> very hard to tell whether any deviations are due to the incorrect model or 
> incorrect value of C. The original CUBIC paper and some other papers show 
> that CUBIC is not fair to Reno CC in certain network conditions where Reno CC 
> has no problems in utilizing the available network capacity; instead, CUBIC 
> steals capacity from Reno CC.
>
> Thanks,
>
> /Markku
>