Re: [tsvwg] https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03#page-10

[<[e-mail]>]

Hi Sebastian,

it is hard to engineer hardware around cross layer functionalities. So reading and setting IP ECN bits behind an MPLS label stack is no good idea, I think. Then all in all, there are 8 codepoints to address DiffServ and ECN in MPLS. I think, spending a single ECN codepoint may be feasible (there's an RFC specifying MPLS/IP ECN interoperartion). Getting 4 ECN codepoints for a single PHB in MPLS will not work.

I personally like the idea of signaling congestion by packet marks rather than by drops. To get MPLS support, one must be sure it is honored E2E. 

AQM can be deployed to manage queue drops. It's deployed in the sections of the backbone that I am aware of (flow there is referring to a Diffserv aggregate, not to anything address-related). Looking at what's offered by our vendors, I think WRED dropping (or the like) is a commodity in the backbone.

ECN should first be deployed, where congestion appears most frequent. That's the Internet access, I think.

Regards,

Ruediger



-----Ursprüngliche Nachricht-----
Von: Sebastian Moeller <[e-mail]> 
Gesendet: Donnerstag, 28. März 2019 13:10
An: Geib, Rüdiger <[e-mail]>
Cc: [e-mail]; [e-mail]
Betreff: Re: [tsvwg] https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03#page-10

Hi Ruediger,

> On Mar 28, 2019, at 12:10, <[e-mail]> <[e-mail]> wrote:
> 
> Hi Gorry,
> 
> If I get you right, you are asking whether all routers along an end to end path should generally deploy per flow queuing or whether that's more suitable for edge routers. 
> 
> I don't recommend per IP flow queuing in MPLS backbone networks. I also don't recommend per top label or per label stack queuing in MPLS backbones.
> 
> Some routers support Virtual Output Queuing, but this more refers to an incoming to outgoing interface multipoint to point relation. 

Do you recommend to deploy AQM in those routers at all? In other words would you bother to look into a MPLS encapsulated IP header to get access to the ECN bits in your MPLS cloud at all, and how about at the edges of the MPLS network?
I naively thought that such an AQM only is needed the edges (access links, and transit/peering points, basically the only places where congestion is likely enough to justify the cost of doing AQM in the first place).

Best Regards
	Sebastian

> 
> Regards,
> 
> Ruediger
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: tsvwg <[e-mail]> Im Auftrag von G Fairhurst
> Gesendet: Donnerstag, 28. März 2019 10:08
> An: Sebastian Moeller <[e-mail]>
> Cc: tsvwg IETF list <[e-mail]>
> Betreff: Re: [tsvwg] https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03#page-10
> 
> On 28/03/2019, 09:59, Sebastian Moeller wrote:
>> Dear Gorry,
>> 
>>> On Mar 28, 2019, at 09:52, G Fairhurst<[e-mail]>  wrote:
>>> 
>>> On 28/03/2019, 09:25, Sebastian Moeller wrote:
>>>> So the L4S architecture description has the following rationale, why L4S opted for the dual-queue approach instead of mandatinf flow queueing:
>>>> 
>>>> https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-03#page-10:
>>>> 
>>>>  Per-flow queuing:  Similarly per-flow queuing is not incompatible
>>>>       with the L4S approach.  However, one queue for every flow can be
>>>>       thought of as overkill compared to the minimum of two queues for
>>>>       all traffic needed for the L4S approach.  The overkill of per-flow
>>>>       queuing has side-effects:
>>>> 
>>>>      A.  fq makes high performance networking equipment costly
>>>>           (processing and memory) - in contrast dual queue code can be
>>>>           very simple;
>>>> 
>>>>       B.  fq requires packet inspection into the end-to-end transport
>>>>           layer, which doesn't sit well alongside encryption for privacy
>>>>           - in contrast the use of ECN as the classifier for L4S
>>>>           requires no deeper inspection than the IP layer;
>>>> 
>>>>       C.  fq isolates the queuing of each flow from the others but not
>>>>           from itself so, unlike L4S, it does not support applications
>>>>           that need both capacity-seeking behaviour and very low
>>>>           latency.
>>>> 
>>>>           It might seem that self-inflicted queuing delay should not
>>>>           count, because if the delay wasn't in the network it would
>>>>           just shift to the sender.  However, modern adaptive
>>>>           applications, e.g.  HTTP/2 [RFC7540] or the interactive media
>>>>           applications described in Section 6, can keep low latency
>>>>           objects at the front of their local send queue by shuffling
>>>>           priorities of other objects dependent on the progress of other
>>>>           transfers.  They cannot shuffle packets once they have
>>>>           released them into the network.
>>>> 
>>>>       D.  fq prevents any one flow from consuming more than 1/N of the
>>>>           capacity at any instant, where N is the number of flows.  This
>>>>           is fine if all flows are elastic, but it does not sit well
>>>>           with a variable bit rate real-time multimedia flow, which
>>>>           requires wriggle room to sometimes take more and other times
>>>>           less than a 1/N share.
>>>> 
>>>>           It might seem that an fq scheduler offers the benefit that it
>>>>           prevents individual flows from hogging all the bandwidth.
>>>>           However, L4S has been deliberately designed so that policing
>>>>           of individual flows can be added as a policy choice, rather
>>>>           than requiring one specific policy choice as the mechanism
>>>>           itself.  A scheduler (like fq) has to decide packet-by-packet
>>>>           which flow to schedule without knowing application intent.
>>>>           Whereas a separate policing function can be configured less
>>>>           strictly, so that senders can still control the instantaneous
>>>>           rate of each flow dependent on the needs of each application
>>>>           (e.g. variable rate video), giving more wriggle-room before a
>>>>           flow is deemed non-compliant.  Also policing of queuing and of
>>>>           flow-rates can be applied independently.
>>>> 
>>>> 
>>>> 
>>>> And then cablelabs, the organization closest to deploying L4S, in Data-Over-Cable Service Interface Specifications DOCSIS® 3.1, MAC and Upper Layer Protocols Interface SpecificationCM-SP-MULPIv3.1-I17-190121, Annex P Queue Protection Algorithm (Normative) says:
>>>> 
>>>> "This annex defines the Queue Protection algorithm that is required to be supported by the CM in the upstream (see Section 7.7.6.1). It is also the Queue Protection algorithm that CMTS Queue Protection algorithms are required to support (see Section 7.7.6.2). In either direction, this algorithm is intended to be applied solely to a Low Latency Service Flow. It detects queue- building Microflows and redirects some or all of their packets to the Classic Service Flow in order to protect the Latency Service Flow from excessive queuing. A Microflow is defined in Section P.3, but typically it is an end-to- end transport layer data flow."
>>>> 
>>>> To me this looks like the introduction of flow-queuing through a backdoor, instead of doing it upfront this tries to measure whether flows behave well enough to keep enjoying the L4S special treatment and demotes non-compliant flows to the TCP-friendly queue. I would be intersted to learn how this mandatory requirement can be implemented without at least having similar cost like flow queueung (at least I fear it will drag the identifies issues A. and B. from above into the system running the L4S-compliant AQM). I would love to learn where my interpretation is wrong.
>>>> 
>>>> Many Thanks in advance
>>>> 
>>>> Regards
>>>> 	Sebastian Moeller
>>> I'm not sure I agree this is necessarily a change of direction - I personally see queue protection as about a control plane optimisation to identify and mitigate traffic from non-conformant flows.
>> 	Sure, but it is telling that the one organization that is in the process of rolling-out L4S makes this a _mandatory_ requirement, no?
> I'll let those who worked on this spec comment upon this.
>>> Per-flow accounting is not the same as per-flow queueing. (From a different perspective: a circuit-breaker that monitors an envelope is not the same as a transport congestion controller or the same as a network traffic shaper/scheduler).
>> 	Sure, but if "Per-flow accounting" has similar computational cost " per-flow queueing" the rationale for not directly mandating  per-flow queueing seems somewhat diminished.
> Unpicking this and asking for help:
> 
> Is it really is the same "cost"?
> 
> And... do people see it as something that all routers along the path should do, or something that an edge router may do?
>>> My expectation would be that someone implementing L4S doesn't need to do this?
>> 	I agree, it is not in the L4S RFCs, so it is not a MUST from the RFCs perspective.
>> 
>> 
>>> ... unless someone wants to offer this sort of protection. In the same way that not all current Ethernet switches protect from overload, but it is important that those people who care about these things can acquire this when they needs it.
>> Fair enough, point taken.
>> 
>> Thanks
>> 	Sebastian
>> 
>>> Gorry
>>> 
> Gorry
>