Re: [tsvwg] links to Canary methods for roll-out of new transport features

[Jonathan Morton <[e-mail]>]

On Friday, 30 July 2021, Gorry (erg) wrote:
> I am not sure though, I think many CC- related topics can have potential collateral damage, and we have managed to deploy these gradually and improve the transport were necessary. So, what is different here to exploring methods such as larger Initial Window, BBR, Hystart, etc.

Hystart makes a transport strictly less aggressive, by exiting the exponential growth phase early and continuing with much slower linear or polynomial growth.  There is no possibility of collateral damage except in case of implementation bugs.  This is an excellent use case for canary testing, and I would endorse its use with Hystart++.

Large IWs do have potential for collateral damage, but the conditions that would trigger it (small buffers) result in effects (high loss to IW) that are easily noticed by the transport employing it.  This is therefore also suitable for canary testing.

BBR is not so clear a case.  I am pleased that Google is taking a relatively cautious approach to deploying it in an Internet facing context, has designed it with standards-track CC coexistence in mind, and seeks to improve it when problems are reported and verified.  However, the ECN response introduced with BBRv2 is not standards-track compatible, and the likely collateral damage when operating in a shared AQM bottleneck is not easily noticed by the BBR transport itself, especially when those circumstances arise only infrequently.  Canary testing is thus an incomplete solution there.

The entire problem here is that L4S is likely to cause *externalised* collateral damage which is not easily noticed by the L4S transport itself.  Unless great care is taken to watch for such problems, canary testing will therefore fail to find them.

What is more, canary testing has as a prerequisite the confidence of correct operation and design gained though lab testing.  Lab testing of L4S has not given any of that confidence, thus progression to canary testing would be inappropriate.

 - Jonathan Morton