Re: [tsvwg] [iccrg] IETF104 ICCRG TCP Prague presentation

[Jonathan Morton <[e-mail]>]

> On 3 Apr, 2019, at 8:40 pm, Bob Briscoe <[e-mail]> wrote:
> 
> To save those who weren't in the room from having to sit through the audio, I said that there are two sets of contradictory measurements: 
> 	• those from 'academic' active measurement studies that have found hardly any CE
> 	• those from Apple's passive measurements on Apple devices that have found significant localized amounts of CE.
> My hypothesis is that the active measurements aren't seeing much CE, cos they don't generate enough traffic themselves to congestion the link and they would have to coincide with other traffic to see it passively.
> 
> I said, in short, I believe the active measurements are more likely wrong, and passive more likely right.

This is actually reasonable.  However your slides were very unclear about this, which I think led to unnecessary confusion.

> Nonetheless the CE that Apple measured could be consistent with classic ECN marking from FQ-CoDel, e.g. in OpenWRT devices, which would protect competing flows from L4S flows. Apple saw CE "mainly in the uplink", which is likely to be FQ-CoDel. And the smaller amount seen in the downlink could have been peer-to-peer flows that had been marked on entry to the uplink by an FQ-CoDel home router.

It is actually common for OpenWRT devices to include "ingress shaping" to exert some control over bufferbloat on downloads.  The terminology, in fact, comes from this practice, as the port in question is facing the WAN link and has two qdiscs attached, one for egress traffic towards the Internet, and one for ingress traffic received from the Internet.

A lower rate of observed marking on downloads could be due to the greater difficulty of driving the download direction into saturation, especially given a backhaul network that may be oversubscribed in many ISPs during peak hours, and thus the greater difficulty of properly configuring an ingress shaper.  It is of course better for AQM to be installed on the upstream side of where the congestion actually occurs.

> In the [Internet -> home] direction, you are right that FQ on the home end would not isolate the access link from a misunderstanding of any CE marking. But I'm not suggesting it could or would. The question is about the machine at the Internet side of the access link (BRAS, CMTS, etc) that is queuing traffic into the access link towards the home. We can be nearly certain it is not doing FQ (these machines serve thousands of customers, so I don't believe any use FQ). The question is whether we can find one of these non-FQ machines that has an AQM enabled that does CE marking.

The motivating reason for ingress shaping is that, generally, the BRAS or equivalent device in the ISP does not perform AQM at all (though some do policing, which doesn't involve ECN).

In my experience there is often a "provisioning shaper" that is essentially a TBF with a dumb tail-dropping FIFO attached; I once suffered one on a rate-limited 3G service which had about 40 seconds of effective bloat.  If that had any sort of AQM installed, ECN or otherwise, flow-isolating or not, it would have resulted in vastly better service at essentially no cost to the ISP.  Indeed it may have saved them some money in reduced memory requirements on their provisioning shaper box.

On my present LTE connection, which varies wildly in available capacity on a minute-to-minute basis, I've set my ingress shaper to a flat 10Mbps rate.  It makes an enormous improvement to the quality of service I receive, even though it reduces my throughput to a fraction of what's available during off-hours.  Yes, it is a flow-isolating AQM shaper (Cake) which is broadly similar to fq_codel.

I did spend a few minutes today sketching out what happens if an L4S flow hits fq_codel on an ingress shaper.  The theoretical results are that, in congestion-avoidance mode, the L4S flow takes about twice as long to drain the dumb bottleneck queue as a Classic ECN flow after the onset of saturation.  But the real shocker is what happens in the terminal phase of slow-start, where a rapid correction is required after the minimum 2x overshoot in exponential growth (because it takes an RTT for the CE/ECE/CWR signal to circulate and take effect).

How many RTTs does it take for the cwnd to be halved in L4S, when the only CE signal is ramping up linearly at one-per-RTT-per-RTT?  Hopefully enough for L4S to detect the delay induced by the dumb FIFO and fall back to RFC-compliant behaviour.

(Yes, we have a real solution sketched out for SCE.  It will be tested soon.  We already have running code.)

 - Jonathan Morton