Re: [aqm] FQ-PIE kernel module implementation
"Agarwal, Anil" <Anil.Agarwal@viasat.com> Tue, 07 July 2015 17:31 UTC
Return-Path: <prvs=86303aa29a=anil.agarwal@viasat.com>
X-Original-To: aqm@ietfa.amsl.com
Delivered-To: aqm@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 05F901ACEDD for <aqm@ietfa.amsl.com>; Tue, 7 Jul 2015 10:31:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.609
X-Spam-Level:
X-Spam-Status: No, score=-1.609 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DC_PNG_UNO_LARGO=0.001, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K3RnmL6qTuSm for <aqm@ietfa.amsl.com>; Tue, 7 Jul 2015 10:31:30 -0700 (PDT)
Received: from mta-us-west-01.viasat.com (mta-us-west-01.viasat.com [8.37.96.47]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 425CF1ACED7 for <aqm@ietf.org>; Tue, 7 Jul 2015 10:31:30 -0700 (PDT)
Received: from pps.filterd (VCASPAM01.hq.corp.viasat.com [127.0.0.1]) by VCASPAM01.hq.corp.viasat.com (8.14.5/8.14.5) with SMTP id t67HVRGJ014417; Tue, 7 Jul 2015 17:31:27 GMT
From: "Agarwal, Anil" <Anil.Agarwal@viasat.com>
To: Polina Goltsman <polina.goltsman@student.kit.edu>, "Bless, Roland (TM)" <roland.bless@kit.edu>, "Fred Baker (fred)" <fred@cisco.com>, Toke Høiland-Jørgensen <toke@toke.dk>
Thread-Topic: [aqm] FQ-PIE kernel module implementation
Thread-Index: AQHQnxLHo+U4nRH5VEqK1N9uhqwYK52l3TeAgCN6BID//6Y4WYAAZwmAgABQInOAAHv8gIABp/TwgARlo4CAABFUAIAAIxjQ
Date: Tue, 07 Jul 2015 17:31:25 +0000
Message-ID: <7A2801D5E40DD64A85E38DF22117852C70AD366D@wdc1exchmbxp05.hq.corp.viasat.com>
References: <D1961A16.1087%hokano@cisco.com> <5577FBD3.5000804@student.kit.edu> <97EDD2D8-CC0A-4AFA-9A74-3F2C282CF5C2@cisco.com> <87mvzem9i9.fsf@alrua-karlstad.karlstad.toke.dk> <7E6C797B-EE6F-4390-BC8F-606FDD8D5195@cisco.com> <559659A8.9030104@student.kit.edu> <87fv55mtpz.fsf@alrua-karlstad.karlstad.toke.dk> <559674B7.5050004@kit.edu> <7A2801D5E40DD64A85E38DF22117852C70AD0859@wdc1exchmbxp05.hq.corp.viasat.com> <559B889B.4060409@kit.edu> <559B9724.6090902@student.kit.edu>
In-Reply-To: <559B9724.6090902@student.kit.edu>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
Content-Type: multipart/related; boundary="_004_7A2801D5E40DD64A85E38DF22117852C70AD366Dwdc1exchmbxp05h_"; type="multipart/alternative"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151, 1.0.33, 0.0.0000 definitions=2015-07-07_06:2015-07-07,2015-07-07,1970-01-01 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 kscore.is_bulkscore=0 kscore.compositescore=1 compositescore=0.9 suspectscore=0 phishscore=0 bulkscore=0 kscore.is_spamscore=0 rbsscore=0.9 spamscore=0 urlsuspectscore=0.9 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1506180000 definitions=main-1507070266
Archived-At: <http://mailarchive.ietf.org/arch/msg/aqm/g0OKCv9Porn9Y1lzV4D9hsaM5EA>
Cc: "draft-ietf-aqm-pie@tools.ietf.org" <draft-ietf-aqm-pie@tools.ietf.org>, "Hironori Okano -X (hokano - AAP3 INC at Cisco)" <hokano@cisco.com>, AQM IETF list <aqm@ietf.org>
Subject: Re: [aqm] FQ-PIE kernel module implementation
X-BeenThere: aqm@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for active queue management and flow isolation." <aqm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/aqm>, <mailto:aqm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/aqm/>
List-Post: <mailto:aqm@ietf.org>
List-Help: <mailto:aqm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/aqm>, <mailto:aqm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Jul 2015 17:31:33 -0000
Polina, Roland, This is good info. So, here is a short summary of our analysis - For FQ-PIE with aggregate-queue AQM - 1. In the presence of unresponsive flows, FQ-PIE has similar properties as single-queue AQMs - the responsive flows are squeezed down to use leftover bandwidth, if any. FQ-AQM with per-queue AQM performs better. 2. In the presence of flows that do not use their fairshare (temporarily or permanently), FQ-PIE has similar properties as single-queue AQMs - the flows, that do not use their fairshare, experience non-zero packet drops. FQ-AQM with per-queue AQM performs better. 3. In the presence of flows that do not use their fairshare (temporarily or permanently), the queue size and queuing delay of flows that use their fairshare can grow above the desired target value. #2 and #3 are probably not major issues - especially in a network bottleneck with a large number of diverse flows. But it is worth pointing out and documenting these properties (somewhere). Regards, Anil From: Polina Goltsman [mailto:polina.goltsman@student.kit.edu] Sent: Tuesday, July 07, 2015 5:09 AM To: Bless, Roland (TM); Agarwal, Anil; Fred Baker (fred); Toke Høiland-Jørgensen Cc: draft-ietf-aqm-pie@tools.ietf.org; Hironori Okano -X (hokano - AAP3 INC at Cisco); AQM IETF list Subject: Re: [aqm] FQ-PIE kernel module implementation Hello all, Here are my thoughts about interaction of AQM and fair-queueing system. I think I will start with a figure. I have started a tcp flow with netperf, and 15 seconds later unresponsive UDP flow with iperf with a send rate a little bit above bottleneck link capacity. Both flows run together for 50 seconds. This figure plots the throughput of UDP flow that was reported by iperf server. (Apparently netperf doesn't produce any output if throughput is below some value, so I can't plot TCP flow.). The bottleneck is 100Mb/s and RTT is 100ms. All AQMs were configured with their default values and noecn flag. [cid:image001.png@01D0B8B7.713A8190] Here is my example in theory. A link with capacity is C is shared between two flows - a non-application-limited TCP flow and unresponsive UDP flow with send rate 105%C. Both flows send max-sized packets, so round robin can be used instead of fair-queueing scheduler. Per definition of max-min fair share both flows are supposed to get 50% of link capacity. (1) Taildrop queues: UDP packets will be dropped when its queue is full, TCP packets will be dropped when its queue is full. As long as there are packets in TCP flow queue, TCP should receive its fair share. ( As far as I understand, this depends on the size of the queue) (2) AQM with state per queue: Drop probability of UDP flow will always be non-zero and should stabilize around approximately 0.5. Drop probability of TCP flow will be non-zero only when it starts sending above 50%C. Thus, while TCP recovers from packet drops, it should not receive another drop. (3) AQM with state per aggregate: UDP flow always creates a standing queue, so drop probability of aggregate is always non-zero. Let's call it p_aqm. The share of TCP packets in the aggregate p_tcp = TCP send rate / (TCP send rate + UDP send rate) and the probability of dropping a TCP packet is p_aqm * p_tcp. This probability is non-zero unless TCP doesn't send at all. In (3) drop probability is at least different. I assume that it is larger than in (2), which will cause more packet drops for TCP flow, and as result the flow will reduce its sending rate below its fair share. Regards, Polina On 07/07/2015 10:06 AM, Bless, Roland (TM) wrote: Hi, thanks for your analysis. Indeed, Polina came up with a similar analysis for an unresponsive UDP flow and a TCP flow. Flow queueing can achieve link share fairness despite the presence of unresponsive flows, but is ineffective if the AQM is applied to the aggregate and not to the individual flow queue. Polina used the FQ-PIE implementation to verify this behavior (post will follow). Regards, Roland Am 04.07.2015 um 22:12 schrieb Agarwal, Anil: Roland, Fred, Here is a simple example to illustrate the differences between FQ-AQM with AQM per queue vs AQM per aggregate queue. Let's take 2 flows, each mapped to separate queues in a FQ-AQM system. Link rate = 100 Mbps Flow 1 rate = 50 Mbps, source rate does not go over 50 Mbps Flow 2 rate >= 50 Mbps, adapts based on AQM. FQ-Codel, AQM per queue: Flow 1 delay is minimal Flow 1 packet drops = 0 Flow 2 delay is close to target value FQ-Codel, AQM for aggregate queue: Does not work at all Packets are dequeued alternatively from queue 1 and queue 2 Packets from queue 1 experience very small queuing delay Hence, CoDel does not enter dropping state, queue 2 is not controlled :( FQ-PIE, AQM per queue: Flow 1 delay is minimal Flow 1 packet drops = 0 Flow 2 delay is close to target value FQ-PIE, AQM for aggregate queue: Flow 1 delay and queue 1 length are close to zero. Flow 2 delay is close to 2 * target_del :( qlen2 = target_del * aggregate_depart_rate Flow 1 experiences almost the same number of drops or ECNs as flow 2 :( Same drop probability and almost same packet rate for both flows (If flow 1 drops its rate because of packet drops or ECNs, the analysis gets slightly more complicated). See if this makes sense. If the analysis is correct, then it illustrates that flow behaviors are quite different between AQM per queue and AQM per aggregate queue schemes. In FQ-PIE for aggregate queue, - The total number of queued bytes will slosh between queues depending on the nature and data rates of the flows. - Flows with data rates within their fair share value will experience non-zero packet drops (or ECN marks). - Flows that experience no queuing delay will increase queuing delay of other flows. - In general, the queuing delay for any given flow will not be close to target_delay and can be much higher
- [aqm] FQ-PIE kernel module implementation Hironori Okano -X (hokano - AAP3 INC at Cisco)
- Re: [aqm] FQ-PIE kernel module implementation Dave Taht
- Re: [aqm] [Bloat] FQ-PIE kernel module implementa… Simon Barber
- Re: [aqm] [Bloat] FQ-PIE kernel module implementa… Bill Ver Steeg (versteb)
- Re: [aqm] [Bloat] FQ-PIE kernel module implementa… Bill Ver Steeg (versteb)
- Re: [aqm] FQ-PIE kernel module implementation Polina Goltsman
- Re: [aqm] [Bloat] FQ-PIE kernel module implementa… Fred Baker (fred)
- Re: [aqm] FQ-PIE kernel module implementation Fred Baker (fred)
- Re: [aqm] FQ-PIE kernel module implementation Toke Høiland-Jørgensen
- Re: [aqm] FQ-PIE kernel module implementation Fred Baker (fred)
- Re: [aqm] FQ-PIE kernel module implementation Fred Baker
- Re: [aqm] FQ-PIE kernel module implementation Toke Høiland-Jørgensen
- Re: [aqm] FQ-PIE kernel module implementation Bless, Roland (TM)
- Re: [aqm] FQ-PIE kernel module implementation Polina Goltsman
- Re: [aqm] FQ-PIE kernel module implementation Simon Barber
- Re: [aqm] FQ-PIE kernel module implementation Dave Taht
- Re: [aqm] FQ-PIE kernel module implementation Michael Welzl
- Re: [aqm] FQ-PIE kernel module implementation Fred Baker (fred)
- Re: [aqm] FQ-PIE kernel module implementation Dave Taht
- Re: [aqm] FQ-PIE kernel module implementation Agarwal, Anil
- Re: [aqm] FQ-PIE kernel module implementation Fred Baker (fred)
- Re: [aqm] FQ-PIE kernel module implementation Bless, Roland (TM)
- Re: [aqm] FQ-PIE kernel module implementation Polina Goltsman
- Re: [aqm] FQ-PIE kernel module implementation Dave Taht
- Re: [aqm] FQ-PIE kernel module implementation Agarwal, Anil
- Re: [aqm] FQ-PIE kernel module implementation Francini, Andrea (Andrea)
- Re: [aqm] FQ-PIE kernel module implementation Rong Pan (ropan)
- Re: [aqm] FQ-PIE kernel module implementation Francini, Andrea (Andrea)
- Re: [aqm] FQ-PIE kernel module implementation Rong Pan (ropan)
- Re: [aqm] FQ-PIE kernel module implementation Agarwal, Anil
- Re: [aqm] FQ-PIE kernel module implementation Agarwal, Anil
- Re: [aqm] FQ-PIE kernel module implementation Polina Goltsman
- Re: [aqm] FQ-PIE kernel module implementation Francini, Andrea (Andrea)
- Re: [aqm] FQ-PIE kernel module implementation Rong Pan (ropan)
- Re: [aqm] FQ-PIE kernel module implementation Agarwal, Anil
- Re: [aqm] FQ-PIE kernel module implementation Rong Pan (ropan)
- Re: [aqm] FQ-PIE kernel module implementation Agarwal, Anil
- Re: [aqm] FQ-PIE kernel module implementation Francini, Andrea (Andrea)
- Re: [aqm] FQ-PIE kernel module implementation Rong Pan (ropan)
- Re: [aqm] FQ-PIE kernel module implementation Agarwal, Anil
- Re: [aqm] FQ-PIE kernel module implementation Francini, Andrea (Andrea)