Re: [iccrg] IETF104 ICCRG TCP Prague presentation

[Sebastian Moeller <[e-mail]>]

Hi Bob,

thanks for your detailed response.


> On Apr 3, 2019, at 19:40, Bob Briscoe <[e-mail]> wrote:
> 
> Sebastien, (adding tsvwg, given this is all part of the decision over L4S/SCE),
> 
> On 02/04/2019 12:38, Sebastian Moeller wrote:
>> I finally got around to see the TCP Prague presentation, nice, especially the part around slide 11 "Fallback to Reno-friendly on Classic ECN" caught my interest.
>> 
>> The claim made there is that L4S's inability to properly deduce the type of CE signal (normal ECN response expected versus DCTCP-like ECN response expected) would only matter in the case of ECN-marking single queue AQMs and not for flow-queueung AQMs, and I would like to argue that that should be measured rather than solved by declaration.
>> 
> [BB] I don't know whether you just looked at my slides without the voice track. But the voice track makes it clear that is exactly what I've been setting up.

	I also listened to the video, but it seems I was confused (I am easily confused). So I whole heartedly agree with this approach.

> 
> To save those who weren't in the room from having to sit through the audio, I said that there are two sets of contradictory measurements: 
> 	• those from 'academic' active measurement studies that have found hardly any CE
> 	• those from Apple's passive measurements on Apple devices that have found significant localized amounts of CE.
> My hypothesis is that the active measurements aren't seeing much CE, cos they don't generate enough traffic themselves to congestion the link and they would have to coincide with other traffic to see it passively.

	I also note, that SQM for example defaults to ECN off on the path from the router towards the internet (rationale, with slow uplinks often seen at internet access links the serialization delay from a single packet can be painful already, this rationale might up for re-evaluation in the light of the newer graded types of responses to ECN marks at the core of the L4S/SCE proposals, but I digress). The consequence of this decision is that the likelyhood of seeing ECN marks is reduced making it harder to measure. If we had a significant number of sqm-devices in the field we might help with switching ECN on on the uplink as well, but we have no reliable way to estimate and no "telemetrics" from deployed instances

> 
> I said, in short, I believe the active measurements are more likely wrong, and passive more likely right.
> 
> Nonetheless the CE that Apple measured could be consistent with classic ECN marking from FQ-CoDel, e.g. in OpenWRT devices, which would protect competing flows from L4S flows.
> Apple saw CE "mainly in the uplink", which is likely to be FQ-CoDel. And the smaller amount seen in the downlink could have been peer-to-peer flows that had been marked on entry to the uplink by an FQ-CoDel home router.
> 
> With the help of others, I've devised a test for FQ vs FIFO, but we need to find where the CE-marking routers are before we apply the test (a scatter-gun approach would be infeasible given the need to congest the link). In discussions with Apple folks, we think it's best to run tests at the CDN end, to identify the network prefixes where CE is appearing (the client end used in the original tests usually doesn't know its public IP address).

	Sounds reasonable, with the potential unhappyness of ISPs for the congestion, but then this is similar to what a speedtest does (or netalyzr).

> 
> If we find any CE in the downlink from the CDN, that will disprove my peer-peer FQ hypothesis. But it won't prove that it's from a FIFO. The test for FQ vs FIFO is not straightforward, and I haven't yet worked out how we run it in the relevant direction over the paths where we find CE (not least 'cos that would involve collecting and identifying client IP addresses, which raises privacy concerns). I suspect we'll need to co-opt some volunteers in the affected localities, somehow.
> 
> An alternative to inline testing will be to find out whether the networks where CE is being seen have supplied their customers with routers with FQ-CoDel enabled. That won't prove conclusively that all the CE is coming from FQ-CoDel, but we can only go so far in trying to prove a negative.

	I wonder how are reading out the CE marking at all? Is the idea to place measurement devices between CDN end-hosts and the CPE and look at the IP headers, or is the plan to use a specific application?

> 
>> 
>> 
>> Consider the following case, that get's more and more common: traffic shaping for both ingress and egress somewhere on the end-customer side of an internet access link. For egress I agree with the rationale, that flow-queueing/fair-queueung should deal with flows that appear unresponsive sooner or later (but I note that this still will increase the memory required for the queueing and should not be considered ideal behavior).
>> But for ingress shaping downstream of the "true" bottleneck (I am simplifying over some details here) this rationale falls apart, the unexpected packets the L4S sender kept sending since it misinterpreted the CE mark, will still hog valuable bandwidth of the internet access link, and hence fair queuing on the home end of the true bottleneck will not isolate that homenet from the L4S misunderstanding.
>> 
> [BB] For the avoidance of doubt, I think you're using 'egress' to mean [home -> Internet] (I use it for the other direction, but I can work with your terminology).

	Sorry, this really is not my field so my nomenclature is probably idiosyncratic, but yes, I see this from the vantage point of the home network, and upload traffic egress from my network, while download traffic ingresses, from the other side of the access links, these direction can be defines the other way around.

> 
> In the [Internet -> home] direction, you are right that FQ on the home end would not isolate the access link from a misunderstanding of any CE marking. But I'm not suggesting it could or would.

	But that is the use-case I am concerned about, as with all the promises (better) AQM at the head-end brings, it will IMHO not eradicate the use of "ingress"-shaping on the customer side, as that is required to allow meaning-full prioritization and to establish the bandwidth-sharing model for the home network (whre typically per-flow-fairness is more desirable than no fairness, but where per-internal-IP-address-fairness might be even more desirable, Jonathan's cake actually offers that capability, but since this absolutely requires access to the internal-IP-addresses will require the AQM ti live inside the NAT typical for IPv4 access links).

> The question is about the machine at the Internet side of the access link (BRAS, CMTS, etc) that is queuing traffic into the access link towards the home. We can be nearly certain it is not doing FQ (these machines serve thousands of customers, so I don't believe any use FQ). The question is whether we can find one of these non-FQ machines that has an AQM enabled that does CE marking.
> 
> That's what we're trying to find. It's of course hard to prove a negative - that none exist. All we can hope to do is measure a sample of the paths we find.

	Okay, but any idea what to do about my hobbyhorse, short of completely disabling L4S for all nodes in the home network (which seems heavy-handed, given that L4S's goal "low latency networking" is IMHO desirable for all end-users)? This kind of set-up is not as unique as it might seem, and typically users employing it are latency sensitive and hence certainly part of the audience which might like the L4S approach other-wise.

> 
> 
> A nit: DCTCP is not /unresponsive/. It responds to congestion, but as if it is equivalent to a number of 'classic' (Reno-friendly) TCP flows.

	I agree, hence the term "appear unresponsive", the problem isn't DCTCP here, but a misunderstanding between AQM and involved TCP.

> 
>> 
>> I would be delighted to learn where my assumptions are wrong...
>> 
> Regards

	Thanks for your kind and informative response.

Best Regards
	Sebastian

> 
> 
> 
> Bob
> 
>> 
>> Regards
>> 	Sebastian Moeller
>> 
>> _______________________________________________
>> iccrg mailing list
>> 
>> [e-mail]
>> https://www.irtf.org/mailman/listinfo/iccrg
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               
> http://bobbriscoe.net/