Re: [tcpm] 2nd WGLC for draft-ietf-tcpm-rfc8312bis

[Markku Kojo <[e-mail]>]

Hi Bob, Yoshi, all,

I'm afraid Bobs statements about the process do not match what RFC 2119 
and RFC 8174 state about using the keywords and what is normative text.
Bob also seem to have missed the crucial role of RFC 5681 that has been 
intentionally and clearly set in RFC 2914 and RFC 5033.

In addition, I'd like to clarify my points about fairness that Bob seems 
to have interpreted incorrectly.

See inline below.


On Tue, 22 Feb 2022, Bob Briscoe wrote:

> Yoshi, Markku,
> 
> Many of the technical points Markku raised were important omissions from the rfc8312bis draft.
> Whatever status we give the eventual RFC, they should be highlighted as areas that need
> further work, either known weaknesses or just areas where we don't have enough data to know
> whether they are weaknesses. I tried to help find useful references for experiments already
> done around those points, and I tried to help write such text.
> 
> However, I cannot agree with some of Markku's statements about process.
>
>   #1: The rules can change
> 
> It's OK to quote RFCs, but it's also OK for us to reach consensus that certain aspects of old
> RFCs are outdated, as long as we give our reasons, and there is rough consensus around that.
> We are all consenting adults - we don't have to be limited by RFCs from the past if we
> collectively don't want to be.

I fully agree. I'd also like to note that if we want to divert from what 
has currently been documented we need to do that in a controlled manner, 
not by publishing an RFC that creates confusion but have a thorough 
discussion and modify these BCPs in those part it is seen appropriate.

> I have not noticed a need for the above "rule #1" in this discussion yet, but I'm just putting
> it out there, so if anyone needs to invoke it later, they can't be accused of inventing a rule
> for their own convenience.

Agreed.

>   #2: There is no TCP-compatibility requirement
> 
> Yes, CUBIC is not TCP-compatible. But that's a feature, not a bug. Words like 'TCP-compatible'
> or 'fair' sound nice, but do not be fooled into thinking that an RFC recommends or requires
> you to behave in a certain way just because it has used a nice-sounding word for it.

RFC 2914, RFC 5033, and RFC 5681 are all in line and compatible with 
each other and that is intentianal, and that is the current consensus 
until we decide to change it. See below that this is also currently normative.

> For an RFC to recommend or require something, it has to say SHOULD or MUST. So when BCP 41
> [RFC2914] defines 'TCP-compatible', and talks about what is necessary to be 'fair', it implies
> nothing about whether you should or must be TCP-compatible, or fair; unless it actually says
> you SHOULD or MUST be TCP-compatible, or fair. No RFC says that.

My long term understanding as well as reading of RFC 2119 and RFC 8174 
gives very different view.

RFC 2119 (the one that was valid at the time RFC 2914 was published) says:

  "These words are often capitalized."

And continues:

                                  Authors who follow these guidelines
  should incorporate this phrase near the beginning of their document:

       The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
       NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and
       "OPTIONAL" in this document are to be interpreted as described in
       RFC 2119.

RFC 2914 and RFC 5033 do not include the above phrase, so the text in 
these RFCs is all *normative* without using the keywords "MUST" or 
"SHOULD". This is further clarified in RFC 8174:

  These words can be used as defined here, but using them is not
  required. Specifically, normative text does not require the use
  of these key words. They are used for clarity and consistency
  when that is what's wanted, but a lot of normative text does not
  use them and is still normative.

There are plenty of RFCs that do not use RFC 2119 language and the text is 
fully normative, including RFC 2914 and RFC 5033.

> BCP 133 [RFC5033] and others have written careful words about straying from equal flow rates
> (quoted below). That gives a checklist for people like us who come along later. But it also
> respects our right to make a decision on whether/how to publish a CC RFC based on our own
> present-day knowledge and experience. To quote the relevant guideline in BCP 133 [RFC5033]:
>
>    (1) Impact on Standard TCP, SCTP [RFC2960], and DCCP [RFC4340].
>
>        Proposed congestion control mechanisms should be evaluated when
>        competing with standard IETF congestion control [RFC2581,
>        RFC2960, RFC4340].  Alternate congestion controllers that have a
>        significantly negative impact on traffic using standard
>        congestion control may be suspect and this aspect should be part
>        of the community's decision making with regards to the
>        suitability of the alternate congestion control mechanism.
>
>        We note that this bullet is not a requirement for strict TCP-
>        friendliness as a prerequisite for an alternate congestion
>        control mechanism to advance to Experimental.  As an example,
>        HighSpeed TCP is a congestion control mechanism that is
>        Experimental, but that is not TCP-friendly in all environments.
>        We also note that this guideline does not constrain the fairness
>        offered for non-best-effort traffic.
>
>        As an example from an Experimental RFC, fairness with standard
>        TCP is discussed in Sections 4 and 6 of [RFC3649] (HighSpeed TCP)
>        and using spare capacity is discussed in Sections 6, 11.1, and 12
>        of [RFC3649].
> The rfc8312bis draft has a section on 'using spare capacity' precisely because that's what was
> considered acceptable about CUBIC and the new generation of hi-BDP CCs developed at that time,
> as I've emphasized in the last para above.

Yes, 'using spare capacity' is all fine, if it is shown to be the case. 
However, providing data and showing that capacity is not stolen from Reno 
CC when Reno CC is capable of fully using the available capacity is 
exactly the concern I have raised. This section 5.2 in rfc8312bis draft is 
not convincing at all. It cites just one paper [HKLRX06] and at least I 
have very hard time to find it providing any evidence that "indicate that 
CUBIC uses the spare bandwidth left unused by existing Reno TCP flows". 
[HKLRX06] focuses on comparing various high-speed TCP alternatives, not 
specifically comparing Reno CC and CUBIC. It has Fig 2 that shows one 
Reno CC (SACK) competing with CUBIC. The results in the figure are hard 
to interpret as distinguishing some of the graphs is quite impossible. 
However, the results seem to show that the link utilization with two SACK 
flows is roughly at the same level or even higher than the link 
utilization with SACK and CUBIC, that is, CUBIC does not even improve 
link utilization when RTT (and BDP) get higher?
In Fig 7., for example at 40ms RTT, where SACK is able to fully utilize 
the available capacity, the fairness index for CUBIC is not really 
supporting the sentence above either. So, it is hard to see how the claim
above could be backed up by data in [HKLRX06]. Maybe I'm missing 
something? 
And, definitely we are lacking results in a wide range of environments, 
just one set of network parameters are used in those tests (only RTT is 
varied).

In addition, as I have pointed out, in [HRX08] Figs 10a and 10c 
clearly demonstrate that CUBIC steals capacity from SACK (Reno) CC.

>   #3: A process for publishing EXP RFCs doesn't cover publishing non-EXP RFCs (by
>   definition)
> 
> This IESG statement on "Experimental Specification of New Congestion Control Algorithms" says
> it defines the process for a CC to be published as an Experimental RFC. It does not say it is
> the process that /all/ CCs have to follow to enter the standards track.
> 
> Before invoking that process, one has to determine whether it is applicable to this case. The
> IESG statement contains a sentence about how to determine its applicability:
>       The decision of whether a specific new congestion control schemes goes beyond the
>       principles of [RFC2914] and hence will undergo the process described in this
>       document lies with the transport area directors, who will use the expertise of the
>       transport area directorate and other congestion control experts to aid their
>       decision.
> 
> Since RFC2914, we've had RFC3246 explaining why Reno is unscalable, and RFC5033 (quoted above)
> giving guidelines for designers of new CCs. So we can debate whether the process that Markku
> quoted is relevant to this case, to help the AD to decide whether to follow the process or
> not.

The IESG statement on "Experimental Specification of New Congestion 
Control Algorithms" is fully inline with RFC 5033 which, in turn, is 
consistent with RFC 2914. It seems that Bob and many others are missing 
(have forgotten?) the long-term policy and consensus that the IETF has 
followed when new/alternate CC proposals have been introduced to the 
IETF; all such proposals are first given experimental status. This is 
also very clearly documented in RFC 5033:

   "Following the lead of HighSpeed TCP [RFC3649], alternate congestion
    control algorithms are expected to be published as "Experimental"
    RFCs until such time that the community better understands the
    solution space."

Bob is all correct when pointing out the problems with Reno CC in high 
BDP environments and what is documented in RFC 3649. RFC 5033 also 
acknowledges this and points to RFC 3649 on how to evaluate such 
proposals. RFC 3649 itself is experimental and it does not claim it is 
ready but recognizes that more work in evaluating it is needed, 
particularly in areas other than the steady-state behavior that RFC 3649 
focuses on. E.g., RFC 3649 points out a crucial test to evaluate: whether 
an alternative CC uses just the spare capacity of Reno CC or does it 
steal capacity from Reno CC when Reno CC (alone) is able to utilize the 
available capacity.

Moreover, as Bob pointed RFC 5033 gives quite clear guidelines and 
criteria to evaluate the alternate CCs and points to RFC 3649 for 
evaluating "high-speed" variants like CUBIC. If we compare the level of 
evaluation in RFC 3649 and rfc8312bis draft, the latter is way behind the 
level of justification and evaluation even though RFC 3649 is just 
experimental.

>   #4 EXP or STD? My view
> 
> Reno is worse than other CCs in some respects. Nearly everyone has abandoned it. So, if
> someone asked you why Reno is the IETF's only stds track CC, what would you say ...?

Has anyone ever thought of any alternative reasons why Reno CC has been 
largely abandoned? Might it be possible that the widely deployed CUBIC 
(and maybe some other alternatives) is much more aggressive than Reno CC 
particularly in environments where Reno CC has no problems in utilizing 
the available bandwidth and thereby CUBIC throttles competing Reno CC 
performance such that nobody finds any variant of Reno CC competitive 
enough anymore?

Note that for the global average access speeds Reno CC has hardly any 
problems in fully utilizing the available capacity, particularly when the 
queue sizes still today tend to be relatively large, if not 
buffer-bloated. There are high BDP environments, for sure, where we have 
a need for CCs that better use the avaiable capacity but such CCs still 
need to compete fairly with Reno CC in lower BDP environments. Otherwise, 
we end up exactly to the "arms race" seriously warned in RFC 2914 and 
reiterated in RFC 5033.

> As I already said, we have to stop thinking that the stds track requires a CC algorithm to be
> perfect. A CC certainly mustn't have failure or collapse modes, but it's performance doesn't
> have to be perfect in all cases.

Sure, Reno CC is not perfect, nor is CUBIC.

> For instance, CUBIC has known poor utilization over radio
> links. That didn't stop Reno from being on the stds track. For me, the IETF needs a new CC on
> the stds track, and CUBIC is the most natural candidate.
> 
> However, we must not set a precedent for any old CC to move to the stds track. So we must
> document it's weaknesses and failings, and explain why they are not serious (if that is the
> case).

Fully agree.

> New CCs need to interoperate with the most prevalent CC on the Internet (CUBIC). The IETF
> cannot even say that as long as it hasn't formally recognized CUBIC on the stds track. That is
> a failing of *us* in the tcpm WG. We can move CUBIC to stds track via the exp track if we have
> to, or we can go straight to stds track. The only difference is that the first option makes us
> look like jobsworths for longer.


What we need to understand is that whichever CC we accept to the stds 
track, we need to have a very good understanding of its performance, 
particularly when competing with the current stds track CC that has a lot 
of dependences in the RFC series. We must not create confusion by having 
a CC on stds track that deviates from that of the current stds track CC 
but we don't know why, when and how much and thereby we are not able to 
document and justify it clearly at the time of publishing such an 
alternate CC. That's why we have created and agreed on the process how to 
bring alternative CCs to the IETF such that they should undergo a thorough 
evaluation.

Thanks,

/Markku

> jobsworth
> /ˈdʒɒbzwəːθ/
> noun  derogatory•informal
> an official who upholds petty rules even at the expense of humanity or common sense.
> 1970s: from "I can't do that, it's more than my job's worth."
> 
> 
> Bob
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
> 
>