[tsvwg] CE-marking observations

["Holland, Jake" <[e-mail]>]

Hi everyone,

I have a few numbers to report about some CE-marking queue
deployment observations over the last few months.

I've urged that we not break normal ECN, and I'll repeat that
urging here.  To me this looks like the early stages of rollout
of a technology that's finally getting a little bit of traction,
and breaking it over the next couple of years seems like a
mistake, and worth avoiding if possible.

(I still think regular AQM deployed in the right places solves
most of the problem, and that ECN helps plenty at the application
level for TCP, so starting a rollout of something incompatible
right when it finally starts happening would be harmful to the
goal of improving latency, regardless of the potential benefits.)

This is all based on observations from production traffic to end
users, since we've turned on ECN support by default since
mid-February.

There were 3 regional consumer ISPs with strong evidence of
ubiquitous marking queue deployment (above 65% of their IPs). One
of these 3 may have deployed something within the last month, as
we saw no such evidence from late February, but it was clearly
present in late March (it also could have been higher congestion
that wasn't ever hit in Feb, since there was substantially more
traffic as well).

There were 7 such ISPs that had marking queues on at least about 15%
of their IPs.

Those numbers seemed to me probably ISP-deployed, and could plausibly
be either shared queue devices or a managed deployment of CPEs with
something like the cake/codel implementations.  I have no visibility
at this point into which they used, only that there were notably more
CE-marking IPs within the AS than what's expected from individuals buying
home gateways with ECN support.

There were 70 or so ASs with observed marking capability prevalence
above 5% across their client IPs, and 30 or so ASs with observed
marking capability over 10%, but many of these were quite small.  I
again have no visibility into whether they were shared queues or no,
and at the lower end it's much harder to make a good guess at whether
it's a managed deployment or individual actions that happen to have
some coherence.  There might be a mix, especially across the lower
end of the 70, of AS-owned vs. not.

Globally I saw a lower bound on client IPs that exhibit marking at
about 0.1%.  The vast majority seemed isolated individuals most
likely using one of the various available home router installs for
their gateway, so to me the more interesting part was the early
edge of ISPs that seem to be doing something that might end up
scaling.

All these are lower bounds, in that I can only tell it's marking if
we saw CE or ECE, and in some cases we plausibly never had congestion
during the observation periods I used.  Also, it of course only has a
chance to get marked when the clients have it enabled, which happens
at quite a small rate in some areas.

These numbers also have a chance at being falsely low for the general
case because it was filtered to nearby connections with enough
connections to usefully examine their latency variation.  But it's at
least something of a lower bound.

HTH.

-Jake