< draft-ietf-aqm-eval-guidelines-11.txt		draft-ietf-aqm-eval-guidelines-12.txt >
	Internet Engineering Task Force N. Kuhn, Ed.		Internet Engineering Task Force N. Kuhn, Ed.
	Internet-Draft CNES, Telecom Bretagne		Internet-Draft CNES, Telecom Bretagne
	Intended status: Informational P. Natarajan, Ed.		Intended status: Informational P. Natarajan, Ed.
	Expires: August 4, 2016 Cisco Systems		Expires: November 2, 2016 Cisco Systems
	N. Khademi, Ed.		N. Khademi, Ed.
	University of Oslo		University of Oslo
	D. Ros		D. Ros
	Simula Research Laboratory AS		Simula Research Laboratory AS
	February 2016		May 2016
	AQM Characterization Guidelines		AQM Characterization Guidelines
	draft-ietf-aqm-eval-guidelines-11		draft-ietf-aqm-eval-guidelines-12
	Abstract		Abstract
	Unmanaged large buffers in today's networks have given rise to a slew		Unmanaged large buffers in today's networks have given rise to a slew
	of performance issues. These performance issues can be addressed by		of performance issues. These performance issues can be addressed by
	some form of Active Queue Management (AQM) mechanism, optionally in		some form of Active Queue Management (AQM) mechanism, optionally in
	combination with a packet scheduling scheme such as fair queuing.		combination with a packet scheduling scheme such as fair queuing.
	This document describes various criteria for performing precautionary		This document describes various criteria for performing
	characterizations of AQM schemes.		characterizations of AQM schemes, that can be used in lab testing
			during development, prior to deployment.
	Status of This Memo		Status of This Memo
	This Internet-Draft is submitted in full conformance with the		This Internet-Draft is submitted in full conformance with the
	provisions of BCP 78 and BCP 79.		provisions of BCP 78 and BCP 79.
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF). Note that other groups may also distribute		Task Force (IETF). Note that other groups may also distribute
	working documents as Internet-Drafts. The list of current Internet-		working documents as Internet-Drafts. The list of current Internet-
	Drafts is at http://datatracker.ietf.org/drafts/current/.		Drafts is at http://datatracker.ietf.org/drafts/current/.
	Internet-Drafts are draft documents valid for a maximum of six months		Internet-Drafts are draft documents valid for a maximum of six months
	and may be updated, replaced, or obsoleted by other documents at any		and may be updated, replaced, or obsoleted by other documents at any
	time. It is inappropriate to use Internet-Drafts as reference		time. It is inappropriate to use Internet-Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	This Internet-Draft will expire on August 4, 2016.		This Internet-Draft will expire on November 2, 2016.
	Copyright Notice		Copyright Notice
	Copyright (c) 2016 IETF Trust and the persons identified as the		Copyright (c) 2016 IETF Trust and the persons identified as the
	document authors. All rights reserved.		document authors. All rights reserved.
	This document is subject to BCP 78 and the IETF Trust's Legal		This document is subject to BCP 78 and the IETF Trust's Legal
	Provisions Relating to IETF Documents		Provisions Relating to IETF Documents
	(http://trustee.ietf.org/license-info) in effect on the date of		(http://trustee.ietf.org/license-info) in effect on the date of
	publication of this document. Please review these documents		publication of this document. Please review these documents
	carefully, as they describe your rights and restrictions with respect		carefully, as they describe your rights and restrictions with respect
	to this document. Code Components extracted from this document must		to this document. Code Components extracted from this document must
	include Simplified BSD License text as described in Section 4.e of		include Simplified BSD License text as described in Section 4.e of
	the Trust Legal Provisions and are provided without warranty as		the Trust Legal Provisions and are provided without warranty as
	described in the Simplified BSD License.		described in the Simplified BSD License.
	Table of Contents		Table of Contents
	1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3		1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
	1.1. Goals of this document . . . . . . . . . . . . . . . . . 5		1.1. Reducing the latency and maximizing the goodput . . . . . 5
	1.2. Requirements Language . . . . . . . . . . . . . . . . . . 6		1.2. Goals of this document . . . . . . . . . . . . . . . . . 5
	1.3. Glossary . . . . . . . . . . . . . . . . . . . . . . . . 6		1.3. Requirements Language . . . . . . . . . . . . . . . . . . 6
	2. End-to-end metrics . . . . . . . . . . . . . . . . . . . . . 6		1.4. Glossary . . . . . . . . . . . . . . . . . . . . . . . . 6
			2. End-to-end metrics . . . . . . . . . . . . . . . . . . . . . 7
	2.1. Flow completion time . . . . . . . . . . . . . . . . . . 7		2.1. Flow completion time . . . . . . . . . . . . . . . . . . 7
	2.2. Flow start up time . . . . . . . . . . . . . . . . . . . 7		2.2. Flow start up time . . . . . . . . . . . . . . . . . . . 8
	2.3. Packet loss . . . . . . . . . . . . . . . . . . . . . . . 7		2.3. Packet loss . . . . . . . . . . . . . . . . . . . . . . . 8
	2.4. Packet loss synchronization . . . . . . . . . . . . . . . 8		2.4. Packet loss synchronization . . . . . . . . . . . . . . . 9
	2.5. Goodput . . . . . . . . . . . . . . . . . . . . . . . . . 9		2.5. Goodput . . . . . . . . . . . . . . . . . . . . . . . . . 9
	2.6. Latency and jitter . . . . . . . . . . . . . . . . . . . 9		2.6. Latency and jitter . . . . . . . . . . . . . . . . . . . 10
	2.7. Discussion on the trade-off between latency and goodput . 10		2.7. Discussion on the trade-off between latency and goodput . 10
	3. Generic setup for evaluations . . . . . . . . . . . . . . . . 10		3. Generic setup for evaluations . . . . . . . . . . . . . . . . 11
	3.1. Topology and notations . . . . . . . . . . . . . . . . . 11		3.1. Topology and notations . . . . . . . . . . . . . . . . . 11
	3.2. Buffer size . . . . . . . . . . . . . . . . . . . . . . . 12		3.2. Buffer size . . . . . . . . . . . . . . . . . . . . . . . 13
	3.3. Congestion controls . . . . . . . . . . . . . . . . . . . 12		3.3. Congestion controls . . . . . . . . . . . . . . . . . . . 13
	4. Methodology, Metrics, AQM Comparisons, Packet Sizes,		4. Methodology, Metrics, AQM Comparisons, Packet Sizes,
	Scheduling and ECN . . . . . . . . . . . . . . . . . . . . . 13		Scheduling and ECN . . . . . . . . . . . . . . . . . . . . . 14
	4.1. Methodology . . . . . . . . . . . . . . . . . . . . . . . 13		4.1. Methodology . . . . . . . . . . . . . . . . . . . . . . . 14
	4.2. Comments on metrics measurement . . . . . . . . . . . . . 13		4.2. Comments on metrics measurement . . . . . . . . . . . . . 14
	4.3. Comparing AQM schemes . . . . . . . . . . . . . . . . . . 14		4.3. Comparing AQM schemes . . . . . . . . . . . . . . . . . . 15
	4.3.1. Performance comparison . . . . . . . . . . . . . . . 14		4.3.1. Performance comparison . . . . . . . . . . . . . . . 15
	4.3.2. Deployment comparison . . . . . . . . . . . . . . . . 15		4.3.2. Deployment comparison . . . . . . . . . . . . . . . . 16
	4.4. Packet sizes and congestion notification . . . . . . . . 15		4.4. Packet sizes and congestion notification . . . . . . . . 16
	4.5. Interaction with ECN . . . . . . . . . . . . . . . . . . 15		4.5. Interaction with ECN . . . . . . . . . . . . . . . . . . 17
	4.6. Interaction with Scheduling . . . . . . . . . . . . . . . 16		4.6. Interaction with Scheduling . . . . . . . . . . . . . . . 17
	5. Transport Protocols . . . . . . . . . . . . . . . . . . . . . 16		5. Transport Protocols . . . . . . . . . . . . . . . . . . . . . 18
	5.1. TCP-friendly sender . . . . . . . . . . . . . . . . . . . 17		5.1. TCP-friendly sender . . . . . . . . . . . . . . . . . . . 18
	5.1.1. TCP-friendly sender with the same initial congestion		5.1.1. TCP-friendly sender with the same initial congestion
	window . . . . . . . . . . . . . . . . . . . . . . . 17		window . . . . . . . . . . . . . . . . . . . . . . . 18
	5.1.2. TCP-friendly sender with different initial congestion		5.1.2. TCP-friendly sender with different initial congestion
	windows . . . . . . . . . . . . . . . . . . . . . . . 17		windows . . . . . . . . . . . . . . . . . . . . . . . 19
	5.2. Aggressive transport sender . . . . . . . . . . . . . . . 18		5.2. Aggressive transport sender . . . . . . . . . . . . . . . 19
	5.3. Unresponsive transport sender . . . . . . . . . . . . . . 18		5.3. Unresponsive transport sender . . . . . . . . . . . . . . 19
	5.4. Less-than Best Effort transport sender . . . . . . . . . 19		5.4. Less-than Best Effort transport sender . . . . . . . . . 20
	6. Round Trip Time Fairness . . . . . . . . . . . . . . . . . . 19		6. Round Trip Time Fairness . . . . . . . . . . . . . . . . . . 21
	6.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 19		6.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 21
	6.2. Recommended tests . . . . . . . . . . . . . . . . . . . . 20		6.2. Recommended tests . . . . . . . . . . . . . . . . . . . . 21
	6.3. Metrics to evaluate the RTT fairness . . . . . . . . . . 20		6.3. Metrics to evaluate the RTT fairness . . . . . . . . . . 21
			7. Burst Absorption . . . . . . . . . . . . . . . . . . . . . . 22
	7. Burst Absorption . . . . . . . . . . . . . . . . . . . . . . 20		7.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 22
	7.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 20		7.2. Recommended tests . . . . . . . . . . . . . . . . . . . . 22
	7.2. Recommended tests . . . . . . . . . . . . . . . . . . . . 21		8. Stability . . . . . . . . . . . . . . . . . . . . . . . . . . 23
	8. Stability . . . . . . . . . . . . . . . . . . . . . . . . . . 22		8.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 23
	8.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 22		8.2. Recommended tests . . . . . . . . . . . . . . . . . . . . 24
	8.2. Recommended tests . . . . . . . . . . . . . . . . . . . . 23		8.2.1. Definition of the congestion Level . . . . . . . . . 24
	8.2.1. Definition of the congestion Level . . . . . . . . . 23		8.2.2. Mild congestion . . . . . . . . . . . . . . . . . . . 25
	8.2.2. Mild congestion . . . . . . . . . . . . . . . . . . . 23		8.2.3. Medium congestion . . . . . . . . . . . . . . . . . . 25
	8.2.3. Medium congestion . . . . . . . . . . . . . . . . . . 23		8.2.4. Heavy congestion . . . . . . . . . . . . . . . . . . 25
	8.2.4. Heavy congestion . . . . . . . . . . . . . . . . . . 24		8.2.5. Varying the congestion level . . . . . . . . . . . . 25
	8.2.5. Varying the congestion level . . . . . . . . . . . . 24		8.2.6. Varying available capacity . . . . . . . . . . . . . 25
	8.2.6. Varying available capacity . . . . . . . . . . . . . 24		8.3. Parameter sensitivity and stability analysis . . . . . . 26
	8.3. Parameter sensitivity and stability analysis . . . . . . 25		9. Various Traffic Profiles . . . . . . . . . . . . . . . . . . 27
	9. Various Traffic Profiles . . . . . . . . . . . . . . . . . . 26		9.1. Traffic mix . . . . . . . . . . . . . . . . . . . . . . . 27
	9.1. Traffic mix . . . . . . . . . . . . . . . . . . . . . . . 26		9.2. Bi-directional traffic . . . . . . . . . . . . . . . . . 28
	9.2. Bi-directional traffic . . . . . . . . . . . . . . . . . 26		10. Example of multi-AQM scenario . . . . . . . . . . . . . . . . 28
	10. Multi-AQM Scenario . . . . . . . . . . . . . . . . . . . . . 27		10.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 28
	10.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 27		10.2. Details on the evaluation scenario . . . . . . . . . . . 28
	10.2. Details on the evaluation scenario . . . . . . . . . . . 27		11. Implementation cost . . . . . . . . . . . . . . . . . . . . . 29
	11. Implementation cost . . . . . . . . . . . . . . . . . . . . . 27		11.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 29
	11.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 27		11.2. Recommended discussion . . . . . . . . . . . . . . . . . 29
	11.2. Recommended discussion . . . . . . . . . . . . . . . . . 28		12. Operator Control and Auto-tuning . . . . . . . . . . . . . . 30
	12. Operator Control and Auto-tuning . . . . . . . . . . . . . . 28		12.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 30
	12.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 28		12.2. Recommended discussion . . . . . . . . . . . . . . . . . 30
	12.2. Recommended discussion . . . . . . . . . . . . . . . . . 29		13. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
	13. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 29		14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 32
	14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 30		15. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 32
	15. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 30		16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32
	16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 31		17. Security Considerations . . . . . . . . . . . . . . . . . . . 32
	17. Security Considerations . . . . . . . . . . . . . . . . . . . 31		18. References . . . . . . . . . . . . . . . . . . . . . . . . . 32
	18. References . . . . . . . . . . . . . . . . . . . . . . . . . 31		18.1. Normative References . . . . . . . . . . . . . . . . . . 32
	18.1. Normative References . . . . . . . . . . . . . . . . . . 31
	18.2. Informative References . . . . . . . . . . . . . . . . . 33		18.2. Informative References . . . . . . . . . . . . . . . . . 33
	Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 34		Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 35
	1. Introduction		1. Introduction
	Active Queue Management (AQM) [RFC7567] addresses the concerns		Active Queue Management (AQM) addresses the concerns arising from
	arising from using unnecessarily large and unmanaged buffers to		using unnecessarily large and unmanaged buffers to improve network
	improve network and application performance. Several AQM algorithms		and application performance, such as presented in the section 1.2 of
			the AQM recommendations document [RFC7567]. Several AQM algorithms
	have been proposed in the past years, most notably Random Early		have been proposed in the past years, most notably Random Early
	Detection (RED), BLUE, and Proportional Integral controller (PI), and		Detection (RED), BLUE, and Proportional Integral controller (PI), and
	more recently CoDel [NICH2012] and PIE [PAN2013]. In general, these		more recently CoDel [I-D.ietf-aqm-codel] and PIE [I-D.ietf-aqm-pie].
	algorithms actively interact with the Transmission Control Protocol		In general, these algorithms actively interact with the Transmission
	(TCP) and any other transport protocol that deploys a congestion		Control Protocol (TCP) and any other transport protocol that deploys
	control scheme to manage the amount of data they keep in the network.		a congestion control scheme to manage the amount of data they keep in
	The available buffer space in the routers and switches should be		the network. The available buffer space in the routers and switches
	large enough to accommodate the short-term buffering requirements.		should be large enough to accommodate the short-term buffering
			requirements. AQM schemes aim at reducing buffer occupancy, and
	AQM schemes aim at reducing buffer occupancy, and therefore the end-		therefore the end-to-end delay. Some of these algorithms, notably
	to-end delay. Some of these algorithms, notably RED, have also been		RED, have also been widely implemented in some network devices.
	widely implemented in some network devices. However, the potential		However, the potential benefits of the RED scheme have not been
	benefits of the RED scheme have not been realized since RED is		realized since RED is reported to be usually turned off.
	reported to be usually turned off. The main reason of this
	reluctance to use RED in today's deployments comes from its
	sensitivity to the operating conditions in the network and the
	difficulty of tuning its parameters.
	A buffer is a physical volume of memory in which a queue or set of		A buffer is a physical volume of memory in which a queue or set of
	queues are stored. When speaking of a specific queue in this		queues are stored. When speaking of a specific queue in this
	document, "buffer occupancy" refers to the amount of data (measured		document, "buffer occupancy" refers to the amount of data (measured
	in bytes or packets) that are in the queue, and the "maximum buffer		in bytes or packets) that are in the queue, and the "maximum buffer
	size" refers to the maximum buffer occupancy. In real		size" refers to the maximum buffer occupancy. In switches and
	implementations of switches, a global memory is often shared between		routers, a global memory space is often shared between the available
	the available devices, and thus, the maximum buffer size may vary		interfaces, and thus, the maximum buffer size for any given interface
	over the time.		may vary over the time.
	Bufferbloat [BB2011] is the consequence of deploying large unmanaged		Bufferbloat [BB2011] is the consequence of deploying large unmanaged
	buffers on the Internet -- the buffering has often been measured to		buffers on the Internet -- the buffering has often been measured to
	be ten times or hundred times larger than needed. Large buffer sizes		be ten times or hundred times larger than needed. Large buffer sizes
	in combination with TCP and/or unresponsive flows increases end-to-		in combination with TCP and/or unresponsive flows increases end-to-
	end delay. This results in poor performance for latency-sensitive		end delay. This results in poor performance for latency-sensitive
	applications such as real-time multimedia (e.g., voice, video,		applications such as real-time multimedia (e.g., voice, video,
	gaming, etc). The degree to which this affects modern networking		gaming, etc). The degree to which this affects modern networking
	equipment, especially consumer-grade equipment's, produces problems		equipment, especially consumer-grade equipment's, produces problems
	even with commonly used web services. Active queue management is		even with commonly used web services. Active queue management is
	thus essential to control queuing delay and decrease network latency.		thus essential to control queuing delay and decrease network latency.
	The Active Queue Management and Packet Scheduling Working Group (AQM		The Active Queue Management and Packet Scheduling Working Group (AQM
	WG) was chartered to address the problems with large unmanaged		WG) was chartered to address the problems with large unmanaged
	buffers in the Internet. Specifically, the AQM WG is tasked with		buffers in the Internet. Specifically, the AQM WG is tasked with
	standardizing AQM schemes that not only address concerns with such		standardizing AQM schemes that not only address concerns with such
	buffers, but also are robust under a wide variety of operating		buffers, but also are robust under a wide variety of operating
	conditions. This document provides characterization guidelines that		conditions. This document provides characterization guidelines that
	can be used to assess the deployability of an AQM, whether it is		can be used to assess the applicability, performance and
	candidate for standardization at IETF or not.		deployability of an AQM, whether it is candidate for standardization
			at IETF or not.
	[RFC7567] separately describes the AQM algorithm implemented in a		AQM algorithm implemented in a router can be separated from the
	router from the scheduling of packets sent by the router. The rest		scheduling of packets sent out by the router as discussed in the AQM
	of this memo refers to the AQM as a dropping/marking policy as a		recommendations document [RFC7567]. The rest of this memo refers to
	separate feature to any interface scheduling scheme. This document		the AQM as a dropping/marking policy as a separate feature to any
	may be complemented with another one on guidelines for assessing		interface scheduling scheme. This document may be complemented with
	combination of packet scheduling and AQM. We note that such a		another one on guidelines for assessing combination of packet
	document will inherit all the guidelines from this document plus any		scheduling and AQM. We note that such a document will inherit all
	additional scenarios relevant for packet scheduling such as flow		the guidelines from this document plus any additional scenarios
	starvation evaluation or impact of the number of hash buckets.		relevant for packet scheduling such as flow starvation evaluation or
			impact of the number of hash buckets.
	1.1. Goals of this document		1.1. Reducing the latency and maximizing the goodput
	The trade-off between reducing the latency and maximizing the goodput		The trade-off between reducing the latency and maximizing the goodput
	is intrinsically linked to each AQM scheme and is key to evaluating		is intrinsically linked to each AQM scheme and is key to evaluating
	its performance. Whenever possible, solutions ought to aim at both		its performance. To ensure the safety deployment of an AQM, its
	maximizing goodput and minimizing latency. Moreover, to ensure the		behaviour should be assessed in a variety of scenarios. Whenever
	safety deployment of an AQM, its behaviour should be assessed in a		possible, solutions ought to aim at both maximizing goodput and
	variety of scenarios.		minimizing latency.
			1.2. Goals of this document
	This document recommends a generic list of scenarios against which an		This document recommends a generic list of scenarios against which an
	AQM proposal should be evaluated, considering both potential		AQM proposal should be evaluated, considering both potential
	performance gain and safety of deployment. The guidelines help to		performance gain and safety of deployment. The guidelines help to
	quantify performance of AQM schemes in terms of latency reduction,		quantify performance of AQM schemes in terms of latency reduction,
	goodput maximization and the trade-off between these two. The		goodput maximization and the trade-off between these two. The
	document presents central aspects of an AQM algorithm that should be		document presents central aspects of an AQM algorithm that should be
	considered whatever the context, such as burst absorption capacity,		considered whatever the context, such as burst absorption capacity,
	RTT fairness or resilience to fluctuating network conditions. The		RTT fairness or resilience to fluctuating network conditions. The
	guidelines also discuss methods to understand the various aspects		guidelines also discuss methods to understand the various aspects
	associated with safely deploying and operating the AQM scheme. Thus,		associated with safely deploying and operating the AQM scheme. Thus,
	one of the key objectives behind formulating the guidelines is to		one of the key objectives behind formulating the guidelines is to
	help ascertain whether a specific AQM is not only better than drop-		help ascertain whether a specific AQM is not only better than drop-
	tail (i.e. without AQM and with a BDP-sized buffer) but also safe to		tail (i.e. without AQM and with a BDP-sized buffer) but also safe to
	deploy: the guidelines can be used to compare several AQM proposals		deploy: the guidelines can be used to compare several AQM proposals
	with each other, and should be used to compare a proposal with drop-		with each other, but should be used to compare a proposal with drop-
	tail.		tail.
			This memo details generic characterization scenarios against which
			any AQM proposal should be evaluated, irrespective of whether or not
			an AQM is standardized by the IETF. This documents recommends the
			relevant scenarios and metrics to be considered. The document
			presents central aspects of an AQM algorithm that should be
			considered whatever the context, such as burst absorption capacity,
			RTT fairness or resilience to fluctuating network conditions.
	These guidelines do not define and are not bound to a particular		These guidelines do not define and are not bound to a particular
	environment or evaluation toolset. Instead the guidelines can be		deployment scenario or evaluation toolset. Instead the guidelines
	used to assert the potential gain of introducing an AQM for the		can be used to assert the potential gain of introducing an AQM for
	particular environment, which is of interest to the testers. These		the particular environment, which is of interest to the testers.
	guidelines do not cover every possible aspect of a particular		These guidelines do not cover every possible aspect of a particular
	algorithm. These guidelines do not present context-dependent		algorithm. These guidelines do not present context-dependent
	scenarios (such as 802.11 WLANs, data-centers or rural broadband		scenarios (such as 802.11 WLANs, data-centers or rural broadband
	networks). To keep the guidelines generic, a number of potential		networks). To keep the guidelines generic, a number of potential
	router components and algorithms (such as DiffServ) are omitted.		router components and algorithms (such as DiffServ) are omitted.
	The goals of this document can thus be summarized as follows:		The goals of this document can thus be summarized as follows:
	o The present characterization guidelines provide a non-exhaustive		o The present characterization guidelines provide a non-exhaustive
	list of scenarios to help ascertain whether an AQM is not only		list of scenarios to help ascertain whether an AQM is not only
	better than drop-tail (with a BDP-sized buffer), but also safe to		better than drop-tail (with a BDP-sized buffer), but also safe to
	skipping to change at page 6, line 12 ¶		skipping to change at page 6, line 24 ¶
	particular evaluation toolset and (2) can be used for various		particular evaluation toolset and (2) can be used for various
	deployment contexts; testers are free to select a toolset that is		deployment contexts; testers are free to select a toolset that is
	best suited for the environment in which their proposal will be		best suited for the environment in which their proposal will be
	deployed.		deployed.
	o The present characterization guidelines are intended to provide		o The present characterization guidelines are intended to provide
	guidance for better selecting an AQM for a specific environment;		guidance for better selecting an AQM for a specific environment;
	it is not required that an AQM proposal is evaluated following		it is not required that an AQM proposal is evaluated following
	these guidelines for its standardization.		these guidelines for its standardization.
	1.2. Requirements Language		1.3. Requirements Language
	The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",		The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
	"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this		"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
	document are to be interpreted as described in RFC 2119 [RFC2119].		document are to be interpreted as described in RFC 2119 [RFC2119].
	1.3. Glossary		1.4. Glossary
	o AQM: [RFC7567] separately describes the Active Queue Managment		o application-limited traffic: a type of traffic that does not have
	(AQM) algorithm implemented in a router from the scheduling of		an unlimited amount of data to transmit.
	packets sent by the router. The rest of this memo refers to the
	AQM as a dropping/marking policy as a separate feature to any		o AQM: the Active Queue Managment (AQM) algorithm implemented in a
	interface scheduling scheme.		router can be separated from the scheduling of packets sent by the
			router. The rest of this memo refers to the AQM as a dropping/
			marking policy as a separate feature to any interface scheduling
			scheme [RFC7567].
			o BDP: Bandwidth Delay Product.
	o buffer: a physical volume of memory in which a queue or set of		o buffer: a physical volume of memory in which a queue or set of
	queues are stored.		queues are stored.
	o buffer occupancy: amount of data that are stored in a buffer,		o buffer occupancy: amount of data that are stored in a buffer,
	measured in bytes or packets.		measured in bytes or packets.
	o buffer size: maximum buffer occupancy, that is the maximum amount		o buffer size: maximum buffer occupancy, that is the maximum amount
	of data that may be stored in a buffer, measured in bytes or		of data that may be stored in a buffer, measured in bytes or
	packets.		packets.
			o IW10: TCP initial congestion window set to 10 packets.
			o latency: one-way delay of packets across Internet paths. This
			definition suits transport layer definition of the latency, that
			shall not be confused with an application layer view of the
			latency.
	o goodput: goodput is defined as the number of bits per unit of time		o goodput: goodput is defined as the number of bits per unit of time
	forwarded to the correct destination minus any bits lost or		forwarded to the correct destination minus any bits lost or
	retransmitted [RFC2647].		retransmitted [RFC2647]. The goodput should be determined for
			each flow and not for aggregates of flows.
	o SQRT: the square root function.		o SQRT: the square root function.
	o ROUND: the round function.		o ROUND: the round function.
	2. End-to-end metrics		2. End-to-end metrics
	End-to-end delay is the result of propagation delay, serialization		End-to-end delay is the result of propagation delay, serialization
	delay, service delay in a switch, medium-access delay and queuing		delay, service delay in a switch, medium-access delay and queuing
	delay, summed over the network elements along the path. AQM schemes		delay, summed over the network elements along the path. AQM schemes
	skipping to change at page 7, line 19 ¶		skipping to change at page 7, line 44 ¶
	Some metrics listed in this section are not suited to every type of		Some metrics listed in this section are not suited to every type of
	traffic detailed in the rest of this document. It is therefore not		traffic detailed in the rest of this document. It is therefore not
	necessary to measure all of the following metrics: the chosen metric		necessary to measure all of the following metrics: the chosen metric
	may not be relevant to the context of the evaluation scenario (e.g.,		may not be relevant to the context of the evaluation scenario (e.g.,
	latency vs. goodput trade-off in application-limited traffic		latency vs. goodput trade-off in application-limited traffic
	scenarios). Guidance is provided for each metric.		scenarios). Guidance is provided for each metric.
	2.1. Flow completion time		2.1. Flow completion time
	The flow completion time is an important performance metric for the		The flow completion time is an important performance metric for the
	end-user when the flow size is finite. Considering the fact that an		end-user when the flow size is finite. The definition of the flow
	AQM scheme may drop/mark packets, the flow completion time is		size may be source of contradictions, thus, this metric can consider
	directly linked to the dropping/marking policy of the AQM scheme.		a flow as a single file. Considering the fact that an AQM scheme may
	This metric helps to better assess the performance of an AQM		drop/mark packets, the flow completion time is directly linked to the
	depending on the flow size. The Flow Completion Time (FCT) is		dropping/marking policy of the AQM scheme. This metric helps to
	related to the flow size (Fs) and the goodput for the flow (G) as		better assess the performance of an AQM depending on the flow size.
	follows:		The Flow Completion Time (FCT) is related to the flow size (Fs) and
			the goodput for the flow (G) as follows:
	FCT [s] = Fs [Byte] / ( G [Bit/s] / 8 [Bit/Byte] )		FCT [s] = Fs [Byte] / ( G [Bit/s] / 8 [Bit/Byte] )
			Where flow size is the size of the application-level flow in bits and
			goodput is the application-level transfer time (described in
			Section 2.5).
	If this metric is used to evaluate the performance of web transfers,		If this metric is used to evaluate the performance of web transfers,
	it is suggested to rather consider the time needed to download all		it is suggested to rather consider the time needed to download all
	the objects that compose the web page, as this makes more sense in		the objects that compose the web page, as this makes more sense in
	terms of user experience than assessing the time needed to download		terms of user experience than assessing the time needed to download
	each object.		each object.
	2.2. Flow start up time		2.2. Flow start up time
	The flow start up time is the time between the request has been sent		The flow start up time is the time between the request has been sent
	from the client and the server starts to transmit data. The amount		from the client and the server starts to transmit data. The amount
	of packets dropped by an AQM may seriously affect the waiting period		of packets dropped by an AQM may seriously affect the waiting period
	during which the data transfer has not started. This metric would		during which the data transfer has not started. This metric would
	specifically focus on the operations such as DNS lookups, TCP opens		specifically focus on the operations such as DNS lookups, TCP opens
	of SSL handshakes.		and SSL handshakes.
	2.3. Packet loss		2.3. Packet loss
	Packet loss can occur en-route, this can impact the end-to-end		Packet loss can occur en-route, this can impact the end-to-end
	performance measured at receiver.		performance measured at receiver.
	The tester SHOULD evaluate loss experienced at the receiver using one		The tester should evaluate loss experienced at the receiver using one
	of the two metrics:		of the two metrics:
	o the packet loss ratio: this metric is to be frequently measured		o the packet loss ratio: this metric is to be frequently measured
	during the experiment. The long-term loss ratio is of interest		during the experiment. The long-term loss ratio is of interest
	for steady-state scenarios only;		for steady-state scenarios only;
	o the interval between consecutive losses: the time between two		o the interval between consecutive losses: the time between two
	losses is to be measured.		losses is to be measured.
	The packet loss ratio can be assessed by simply evaluating the loss		The packet loss ratio can be assessed by simply evaluating the loss
	ratio as a function of the number of lost packets and the total		ratio as a function of the number of lost packets and the total
	number of packets sent. This might not be easily done in laboratory		number of packets sent. This might not be easily done in laboratory
	testing, for which these guidelines advice the tester:		testing, for which these guidelines advice the tester:
	o to check that for every packet, a corresponding packet was		o to check that for every packet, a corresponding packet was
	received within a reasonable time, as explained in [RFC2680].		received within a reasonable time, as presented in the document
			that proposes a metric for one-way packet loss across Internet
			paths [RFC2680].
	o to keep a count of all packets sent, and a count of the non-		o to keep a count of all packets sent, and a count of the non-
	duplicate packets received, as explained in the section 10 of		duplicate packets received, as discussed in RFC that presents a
	[RFC2544].		benchmarking methodology [RFC2544].
	The interval between consecutive losses, which is also called a gap,		The interval between consecutive losses, which is also called a gap,
	is a metric of interest for VoIP traffic and, as a result, has been		is a metric of interest for VoIP traffic [RFC3611].
	further specified in [RFC3611].
	2.4. Packet loss synchronization		2.4. Packet loss synchronization
	One goal of an AQM algorithm is to help to avoid global		One goal of an AQM algorithm is to help to avoid global
	synchronization of flows sharing a bottleneck buffer on which the AQM		synchronization of flows sharing a bottleneck buffer on which the AQM
	operates ([RFC2309],[RFC7567]). The "degree" of packet-loss		operates ([RFC2309],[RFC7567]). The "degree" of packet-loss
	synchronization between flows SHOULD be assessed, with and without		synchronization between flows should be assessed, with and without
	the AQM under consideration.		the AQM under consideration.
	As discussed e.g., in [HASS2008], loss synchronization among flows		Loss synchronization among flows may be quantified by several
	may be quantified by several slightly different metrics that capture		slightly different metrics that capture different aspects of the same
	different aspects of the same issue. However, in real-world		issue [HASS2008]. However, in real-world measurements the choice of
	measurements the choice of metric could be imposed by practical		metric could be imposed by practical considerations -- e.g., whether
	considerations -- e.g., whether fine-grained information on packet		fine-grained information on packet losses at the bottleneck is
	losses in the bottleneck available or not. For the purpose of AQM		available or not. For the purpose of AQM characterization, a good
	characterization, a good candidate metric is the global		candidate metric is the global synchronization ratio, measuring the
	synchronization ratio, measuring the proportion of flows losing		proportion of flows losing packets during a loss event. This metric
	packets during a loss event. [JAY2006] used this metric in real-		can be used in real-world experiments to characterize synchronization
	world experiments to characterize synchronization along arbitrary		along arbitrary Internet paths [JAY2006].
	Internet paths; the full methodology is described in [JAY2006].
	If an AQM scheme is evaluated using real-life network environments,		If an AQM scheme is evaluated using real-life network environments,
	it is worth pointing out that some network events, such as failed		it is worth pointing out that some network events, such as failed
	link restoration may cause synchronized losses between active flows		link restoration may cause synchronized losses between active flows
	and thus confuse the meaning of this metric.		and thus confuse the meaning of this metric.
	2.5. Goodput		2.5. Goodput
	The goodput has been defined in section 3.17 of [RFC2647] as the		The goodput has been defined as the number of bits per unit of time
	number of bits per unit of time forwarded to the correct destination		forwarded to the correct destination interface, minus any bits lost
	interface, minus any bits lost or retransmitted. This definition		or retransmitted, such as proposed in the secton 3.17 of the RFC
	induces that the test setup needs to be qualified to assure that it		describing the benchmarking terminology for firewall performances
	is not generating losses on its own.		[RFC2647]. This definition requires that the test setup needs to be
			qualified to assure that it is not generating losses on its own.
	Measuring the end-to-end goodput provides an appreciation of how well		Measuring the end-to-end goodput provides an appreciation of how well
	an AQM scheme improves transport and application performance. The		an AQM scheme improves transport and application performance. The
	measured end-to-end goodput is linked to the dropping/marking policy		measured end-to-end goodput is linked to the dropping/marking policy
	of the AQM scheme -- e.g., the fewer the number of packet drops, the		of the AQM scheme -- e.g., the fewer the number of packet drops, the
	fewer packets need retransmission, minimizing the impact of AQM on		fewer packets need retransmission, minimizing the impact of AQM on
	transport and application performance. Additionally, an AQM scheme		transport and application performance. Additionally, an AQM scheme
	may resort to Explicit Congestion Notification (ECN) marking as an		may resort to Explicit Congestion Notification (ECN) marking as an
	initial means to control delay. Again, marking packets instead of		initial means to control delay. Again, marking packets instead of
	dropping them reduces the number of packet retransmissions and		dropping them reduces the number of packet retransmissions and
	increases goodput. End-to-end goodput values help to evaluate the		increases goodput. End-to-end goodput values help to evaluate the
	AQM scheme's effectiveness of an AQM scheme in minimizing packet		AQM scheme's effectiveness of an AQM scheme in minimizing packet
	drops that impact application performance and to estimate how well		drops that impact application performance and to estimate how well
	the AQM scheme works with ECN.		the AQM scheme works with ECN.
	The measurement of the goodput allows the tester evaluate to which		The measurement of the goodput allows the tester to evaluate to which
	extent an AQM is able to maintain a high bottleneck utilization.		extent an AQM is able to maintain a high bottleneck utilization.
	This metric should be also obtained frequently during an experiment		This metric should also be obtained frequently during an experiment
	as the long-term goodput is relevant for steady-state scenarios only		as the long-term goodput is relevant for steady-state scenarios only
	and may not necessarily reflect how the introduction of an AQM		and may not necessarily reflect how the introduction of an AQM
	actually impacts the link utilization during at a certain period of		actually impacts the link utilization during at a certain period of
	time. Fluctuations in the values obtained from these measurements		time. Fluctuations in the values obtained from these measurements
	may depend on other factors than the introduction of an AQM, such as		may depend on other factors than the introduction of an AQM, such as
	link layer losses due to external noise or corruption, fluctuating		link layer losses due to external noise or corruption, fluctuating
	bandwidths (802.11 WLANs), heavy congestion levels or transport		bandwidths (802.11 WLANs), heavy congestion levels or transport
	layer's rate reduction by congestion control mechanism.		layer's rate reduction by congestion control mechanism.
	2.6. Latency and jitter		2.6. Latency and jitter
	The latency, or the one-way delay metric, is discussed in [RFC2679].		The latency, or the one-way delay metric, is discussed in [RFC2679].
	There is a consensus on an adequate metric for the jitter, that		There is a consensus on an adequate metric for the jitter, that
	represents the one-way delay variations for packets from the same		represents the one-way delay variations for packets from the same
	flow: the Packet Delay Variation (PDV), detailed in [RFC5481], serves		flow: the Packet Delay Variation (PDV) serves well all use cases
	well all use cases.		[RFC5481].
	The end-to-end latency includes components other than just the		The end-to-end latency includes components other than just the
	queuing delay, such as the signal processing delay, transmission		queuing delay, such as the signal processing delay, transmission
	delay and the processing delay. Moreover, the jitter is caused by		delay and the processing delay. Moreover, the jitter is caused by
	variations in queuing and processing delay (e.g., scheduling		variations in queuing and processing delay (e.g., scheduling
	effects). The introduction of an AQM scheme would impact these		effects). The introduction of an AQM scheme would impact end-to-end
	metrics (end-to-end latency and jitter) and therefore they should be		latency and jitter, and therefore these metrics should be considered
	considered in the end-to-end evaluation of performance.		in the end-to-end evaluation of performance.
	2.7. Discussion on the trade-off between latency and goodput		2.7. Discussion on the trade-off between latency and goodput
	The metrics presented in this section may be considered as explained		The metrics presented in this section may be considered in order to
	in the rest of this document, in order to discuss and quantify the		discuss and quantify the trade-off between latency and goodput.
	trade-off between latency and goodput.
	With regards to the goodput, and in addition to the long-term		With regards to the goodput, and in addition to the long-term
	stationary goodput value, it is RECOMMENDED to take measurements		stationary goodput value, it is recommended to take measurements
	every multiple of the minimum RTT (minRTT) between A and B. It is		every multiple of the minimum RTT (minRTT) between A and B. It is
	suggested to take measurements at least every K x minRTT (to smooth		suggested to take measurements at least every K x minRTT (to smooth
	out the fluctuations), with K=10. Higher values for K are encouraged		out the fluctuations), with K=10. Higher values for K can be
	whenever it is more appropriate for the presentation of the results.		considered whenever it is more appropriate for the presentation of
	The value for K may depend on the network's path characteristics.		the results, since the value for K may depend on the network's path
	The measurement period MUST be disclosed for each experiment and when		characteristics. The measurement period must be disclosed for each
	results/values are compared across different AQM schemes, the		experiment and when results/values are compared across different AQM
	comparisons SHOULD use exactly the same measurement periods. With		schemes, the comparisons should use exactly the same measurement
	regards to latency, it is RECOMMENDED to take the samples on per-		periods. With regards to latency, it is recommended to take the
	packet basis whenever possible depending on the features provided by		samples on per-packet basis whenever possible depending on the
	hardware/software and the impact of sampling itself on the hardware		features provided by hardware/software and the impact of sampling
	performance. It is generally RECOMMENDED to provide at least 10		itself on the hardware performance.
	samples per RTT.
	From each of these sets of measurements, the cumulative density		From each of these sets of measurements, the cumulative density
	function (CDF) of the considered metrics SHOULD be computed. If the		function (CDF) of the considered metrics should be computed. If the
	considered scenario introduces dynamically varying parameters,		considered scenario introduces dynamically varying parameters,
	temporal evolution of the metrics could also be generated. For each		temporal evolution of the metrics could also be generated. For each
	scenario, the following graph may be generated: the x-axis shows		scenario, the following graph may be generated: the x-axis shows
	queuing delay (that is the average per-packet delay in excess of		queuing delay (that is the average per-packet delay in excess of
	minimum RTT), the y-axis the goodput. Ellipses are computed such as		minimum RTT), the y-axis the goodput. Ellipses are computed such as
	detailed in [WINS2014]: "We take each individual [...] run [...] as		detailed in [WINS2014]: "We take each individual [...] run [...] as
	one point, and then compute the 1-epsilon elliptic contour of the		one point, and then compute the 1-epsilon elliptic contour of the
	maximum-likelihood 2D Gaussian distribution that explains the points.		maximum-likelihood 2D Gaussian distribution that explains the points.
	[...] we plot the median per-sender throughput and queueing delay as		[...] we plot the median per-sender throughput and queueing delay as
	a circle. [...] The orientation of an ellipse represents the		a circle. [...] The orientation of an ellipse represents the
	covariance between the throughput and delay measured for the		covariance between the throughput and delay measured for the
	protocol." This graph provides part of a better understanding of (1)		protocol." This graph provides part of a better understanding of (1)
	the delay/goodput trade-off for a given congestion control mechanism		the delay/goodput trade-off for a given congestion control mechanism
	Section 5, and (2) how the goodput and average queue delay vary as a		(Section 5), and (2) how the goodput and average queue delay vary as
	function of the traffic load Section 8.2.		a function of the traffic load (Section 8.2).
	3. Generic setup for evaluations		3. Generic setup for evaluations
	This section presents the topology that can be used for each of the		This section presents the topology that can be used for each of the
	following scenarios, the corresponding notations and discusses		following scenarios, the corresponding notations and discusses
	various assumptions that have been made in the document.		various assumptions that have been made in the document.
	3.1. Topology and notations		3.1. Topology and notations
	+---------+ +-----------+
	|senders A| |receivers B|
	+---------+ +-----------+
	+--------------+ +--------------+		+--------------+ +--------------+
	|traffic class1| |traffic class1|		|sender A_i | |receive B_i |
	|--------------| |--------------|		|--------------| |--------------|
	| SEN.Flow1.1 +---------+ +-----------+ REC.Flow1.1 |		| SEN.Flow1.1 +---------+ +-----------+ REC.Flow1.1 |
	| + | | | | + |		| + | | | | + |
	| | | | | | | |		| | | | | | | |
	| + | | | | + |		| + | | | | + |
	| SEN.Flow1.X +-----+ | | +--------+ REC.Flow1.X |		| SEN.Flow1.X +-----+ | | +--------+ REC.Flow1.X |
	+--------------+ | | | | +--------------+		+--------------+ | | | | +--------------+
	+ +-+---+---+ +--+--+---+ +		+ +-+---+---+ +--+--+---+ +
	| |Router L | |Router R | |		| |Router L | |Router R | |
	| |---------| |---------| |		| |---------| |---------| |
	| | AQM | | | |		| | AQM | | | |
	| | BuffSize| | BuffSize| |		| | BuffSize| | BuffSize| |
	| | (Bsize) +-----+ (Bsize) | |		| | (Bsize) +-----+ (Bsize) | |
	| +-----+--++ ++-+------+ |		| +-----+--++ ++-+------+ |
	+ | | | | +		+ | | | | +
	+--------------+ | | | | +--------------+		+--------------+ | | | | +--------------+
	|traffic classN| | | | | |traffic classN|		|sender A_n | | | | | |receive B_n |
	|--------------| | | | | |--------------|		|--------------| | | | | |--------------|
	| SEN.FlowN.1 +---------+ | | +-----------+ REC.FlowN.1 |		| SEN.FlowN.1 +---------+ | | +-----------+ REC.FlowN.1 |
	| + | | | | + |		| + | | | | + |
	| | | | | | | |		| | | | | | | |
	| + | | | | + |		| + | | | | + |
	| SEN.FlowN.Y +------------+ +-------------+ REC.FlowN.Y |		| SEN.FlowN.Y +------------+ +-------------+ REC.FlowN.Y |
	+--------------+ +--------------+		+--------------+ +--------------+
	Figure 1: Topology and notations		Figure 1: Topology and notations
	Figure 1 is a generic topology where:		Figure 1 is a generic topology where:
	o sender with different traffic characteristics (i.e., traffic		o traffic profile is a set of flows with similar characteristics -
			RTT, congestion control scheme, transport protocol, etc.;
			o senders with different traffic characteristics (i.e., traffic
	profiles) can be introduced;		profiles) can be introduced;
	o the timing of each flow could be different (i.e., when does each		o the timing of each flow could be different (i.e., when does each
	flow start and stop);		flow start and stop);
	o each traffic profile can comprise various number of flows;		o each traffic profile can comprise various number of flows;
	o each link is characterized by a couple (one-way delay, capacity);		o each link is characterized by a couple (one-way delay, capacity);
	o flows are generated at A and sent to B, sharing a bottleneck (the
	link between routers L and R);
	o the tester SHOULD consider both scenarios of asymmetric and		o sender A_i is instantiated for each traffic profile. A
			corresponding receiver B_i is instantiated for receiving the flows
			in the profile;
			o flows sharing a bottleneck (the link between routers L and R);
			o the tester should consider both scenarios of asymmetric and
	symmetric bottleneck links in terms of bandwidth. In case of		symmetric bottleneck links in terms of bandwidth. In case of
	asymmetric link, the capacity from senders to receivers is higher		asymmetric link, the capacity from senders to receivers is higher
	than the one from receivers to senders; the symmetric link		than the one from receivers to senders; the symmetric link
	scenario provides a basic understanding of the operation of the		scenario provides a basic understanding of the operation of the
	AQM mechanism whereas the asymmetric link scenario evaluates an		AQM mechanism whereas the asymmetric link scenario evaluates an
	AQM mechanism in a more realistic setup;		AQM mechanism in a more realistic setup;
	o in asymmetric link scenarios, the tester SHOULD study the bi-		o in asymmetric link scenarios, the tester should study the bi-
	directional traffic between A and B (downlink and uplink) with the		directional traffic between A and B (downlink and uplink) with the
	AQM mechanism deployed on one direction only. The tester MAY		AQM mechanism deployed on one direction only. The tester may
	additionally consider a scenario with AQM mechanism being deployed		additionally consider a scenario with AQM mechanism being deployed
	on both directions. In each scenario, the tester SHOULD		on both directions. In each scenario, the tester should
	investigate the impact of drop policy of the AQM on TCP ACK		investigate the impact of drop policy of the AQM on TCP ACK
	packets and its impact on the performance.		packets and its impact on the performance (Section 9.2).
	Although this topology may not perfectly reflect actual topologies,		Although this topology may not perfectly reflect actual topologies,
	the simple topology is commonly used in the world of simulations and		the simple topology is commonly used in the world of simulations and
	small testbeds. It can be considered as adequate to evaluate AQM		small testbeds. It can be considered as adequate to evaluate AQM
	proposals, similarly to the topology proposed in		proposals [I-D.irtf-iccrg-tcpeval]. Testers ought to pay attention
	[I-D.irtf-iccrg-tcpeval]. Testers ought to pay attention to the		to the topology that has been used to evaluate an AQM scheme when
	topology that has been used to evaluate an AQM scheme when comparing		comparing this scheme with a newly proposed AQM scheme.
	this scheme with a newly proposed AQM scheme.
	3.2. Buffer size		3.2. Buffer size
	The size of the buffers should be carefully chosen, and MAY be set to		The size of the buffers should be carefully chosen, and may be set to
	the bandwidth-delay product; the bandwidth being the bottleneck		the bandwidth-delay product; the bandwidth being the bottleneck
	capacity and the delay the largest RTT in the considered network.		capacity and the delay the largest RTT in the considered network.
	The size of the buffer can impact the AQM performance and is a		The size of the buffer can impact the AQM performance and is a
	dimensioning parameter that will be considered when comparing AQM		dimensioning parameter that will be considered when comparing AQM
	proposals.		proposals.
	If a specific buffer size is required, the tester MUST justify and		If a specific buffer size is required, the tester must justify and
	detail the way the maximum queue size is set. Indeed, the maximum		detail the way the maximum queue size is set. Indeed, the maximum
	size of the buffer may affect the AQM's performance and its choice		size of the buffer may affect the AQM's performance and its choice
	SHOULD be elaborated for a fair comparison between AQM proposals.		should be elaborated for a fair comparison between AQM proposals.
	While comparing AQM schemes the buffer size SHOULD remain the same		While comparing AQM schemes the buffer size should remain the same
	across the tests.		across the tests.
	3.3. Congestion controls		3.3. Congestion controls
	This document considers running three different congestion control		This document considers running three different congestion control
	algorithms between A and B		algorithms between A and B
	o Standard TCP congestion control: the base-line congestion control		o Standard TCP congestion control: the base-line congestion control
	is TCP NewReno with SACK, as explained in [RFC5681].		is TCP NewReno with SACK [RFC5681].
	o Aggressive congestion controls: a base-line congestion control for		o Aggressive congestion controls: a base-line congestion control for
	this category is TCP Cubic [I-D.ietf-tcpm-cubic].		this category is TCP Cubic [I-D.ietf-tcpm-cubic].
	o Less-than Best Effort (LBE) congestion controls: an LBE congestion		o Less-than Best Effort (LBE) congestion controls: an LBE congestion
	control 'results in smaller bandwidth and/or delay impact on		control 'results in smaller bandwidth and/or delay impact on
	standard TCP than standard TCP itself, when sharing a bottleneck		standard TCP than standard TCP itself, when sharing a bottleneck
	with it.' [RFC6297]		with it.': a base-line congestion control for this category is
			LEDBAT [RFC6817].
	Other transport congestion controls can OPTIONALLY be evaluated in		Other transport congestion controls can OPTIONALLY be evaluated in
	addition. Recent transport layer protocols are not mentioned in the		addition. Recent transport layer protocols are not mentioned in the
	following sections, for the sake of simplicity.		following sections, for the sake of simplicity.
	4. Methodology, Metrics, AQM Comparisons, Packet Sizes, Scheduling and		4. Methodology, Metrics, AQM Comparisons, Packet Sizes, Scheduling and
	ECN		ECN
	4.1. Methodology		4.1. Methodology
	A description of each test setup SHOULD be detailed to allow this		A description of each test setup should be detailed to allow this
	test to be compared with other tests. This also allows others to		test to be compared with other tests. This also allows others to
	replicate the tests if needed. This test setup SHOULD detail		replicate the tests if needed. This test setup should detail
	software and hardware versions. The tester could make its data		software and hardware versions. The tester could make its data
	available.		available.
	The proposals SHOULD be evaluated on real-life systems, or they MAY		The proposals should be evaluated on real-life systems, or they may
	be evaluated with event-driven simulations (such as ns-2, ns-3,		be evaluated with event-driven simulations (such as ns-2, ns-3,
	OMNET, etc). The proposed scenarios are not bound to a particular		OMNET, etc). The proposed scenarios are not bound to a particular
	evaluation toolset.		evaluation toolset.
	The tester is encouraged to make the detailed test setup and the		The tester is encouraged to make the detailed test setup and the
	results publicly available.		results publicly available.
	4.2. Comments on metrics measurement		4.2. Comments on metrics measurement
	The document presents the end-to-end metrics that ought to be used to		The document presents the end-to-end metrics that ought to be used to
	evaluate the trade-off between latency and goodput in Section 2. In		evaluate the trade-off between latency and goodput in Section 2. In
	addition to the end-to-end metrics, the queue-level metrics (normally		addition to the end-to-end metrics, the queue-level metrics (normally
	collected at the device operating the AQM) provide a better		collected at the device operating the AQM) provide a better
	understanding of the AQM behavior under study and the impact of its		understanding of the AQM behavior under study and the impact of its
	internal parameters. Whenever it is possible (e.g., depending on the		internal parameters. Whenever it is possible (e.g., depending on the
	features provided by the hardware/software), these guidelines advice		features provided by the hardware/software), these guidelines advise
	to consider queue-level metrics, such as link utilization, queuing		to consider queue-level metrics, such as link utilization, queuing
	delay, queue size or packet drop/mark statistics in addition to the		delay, queue size or packet drop/mark statistics in addition to the
	AQM-specific parameters. However, the evaluation MUST be primarily		AQM-specific parameters. However, the evaluation must be primarily
	based on externally observed end-to-end metrics.		based on externally observed end-to-end metrics.
	These guidelines do not aim to detail on the way these metrics can be		These guidelines do not aim to detail on the way these metrics can be
	measured, since the way these metrics are measured is expected to		measured, since the way these metrics are measured is expected to
	depend on the evaluation toolset.		depend on the evaluation toolset.
	4.3. Comparing AQM schemes		4.3. Comparing AQM schemes
	This document recognizes that these guidelines may be used for		This document recognizes that these guidelines may be used for
	comparing AQM schemes.		comparing AQM schemes.
	AQM schemes need to be compared against both performance and		AQM schemes need to be compared against both performance and
	deployment categories. In addition, this section details how best to		deployment categories. In addition, this section details how best to
	achieve a fair comparison of AQM schemes by avoiding certain		achieve a fair comparison of AQM schemes by avoiding certain
	pitfalls.		pitfalls.
	4.3.1. Performance comparison		4.3.1. Performance comparison
	AQM schemes should be compared against the generic scenarios that are		AQM schemes should be compared against the generic scenarios that are
	summarized in Section 13. AQM schemes MAY be compared for specific		summarized in Section 13. AQM schemes may be compared for specific
	network environments such as data centers, home networks, etc. If an		network environments such as data centers, home networks, etc. If an
	AQM scheme has parameter(s) that were externally tuned for		AQM scheme has parameter(s) that were externally tuned for
	optimization or other purposes, these values MUST be disclosed.		optimization or other purposes, these values must be disclosed.
	AQM schemes belong to different varieties such as queue-length based		AQM schemes belong to different varieties such as queue-length based
	schemes (ex. RED) or queueing-delay based scheme (ex. CoDel, PIE).		schemes (ex. RED) or queueing-delay based scheme (ex. CoDel, PIE).
	AQM schemes expose different control knobs associated with different		AQM schemes expose different control knobs associated with different
	semantics. For example, while both PIE and CoDel are queueing-delay		semantics. For example, while both PIE and CoDel are queueing-delay
	based schemes and each expose a knob to control the queueing delay --		based schemes and each expose a knob to control the queueing delay --
	PIE's "queueing delay reference" vs. CoDel's "queueing delay target",		PIE's "queueing delay reference" vs. CoDel's "queueing delay target",
	the two tuning parameters of the two schemes have different		the two tuning parameters of the two schemes have different
	semantics, resulting in different control points. Such differences		semantics, resulting in different control points. Such differences
	in AQM schemes can be easily overlooked while making comparisons.		in AQM schemes can be easily overlooked while making comparisons.
	This document RECOMMENDS the following procedures for a fair		This document recommends the following procedures for a fair
	performance comparison between the AQM schemes:		performance comparison between the AQM schemes:
	1. comparable control parameters and comparable input values:		1. similar control parameters and implications: Testers should be
	carefully identify the set of parameters that control similar		aware of the control parameters of the different schemes that
	behavior between the two AQM schemes and ensure these parameters		control similar behavior. Testers should also be aware of the
	have comparable input values. For example, to compare how well a		input value ranges and corresponding implications. For example,
	queue-length based AQM scheme controls queueing delay vs. a		consider two different schemes - (A) queue-length based AQM
	queueing-delay based AQM scheme, a tester can identify the		scheme, and (B) queueing-delay based scheme. A and B are likely
	parameters of the schemes that control queue delay and ensure		to have different kinds of control inputs to control the target
	that their input values are comparable. Similarly, to compare		delay - target queue length in A vs. target queuing delay in B,
	how well two AQM schemes accommodate packet bursts, the tester		for example. Setting parameter values such as 100MB for A vs.
	can identify burst-related control parameters and ensure they are		10ms for B will have different implications depending on
	configured with similar values. Additionally, it would be		evaluation context. Such context-dependent implications must be
	preferable if an AQM proposal listed such parameters and		considered before drawing conclusions on performance comparisons.
	discussed how each relates to network characteristics such as		Also, it would be preferable if an AQM proposal listed such
	capacity, average RTT etc.		parameters and discussed how each relates to network
			characteristics such as capacity, average RTT etc.
	2. compare over a range of input configurations: there could be		2. compare over a range of input configurations: there could be
	situations when the set of control parameters that affect a		situations when the set of control parameters that affect a
	specific behavior have different semantics between the two AQM		specific behavior have different semantics between the two AQM
	schemes. As mentioned above, PIE has tuning parameters to		schemes. As mentioned above, PIE has tuning parameters to
	control queue delay that has a different semantics from those		control queue delay that has a different semantics from those
	used in CoDel. In such situations, these schemes need to be		used in CoDel. In such situations, these schemes need to be
	compared over a range of input configurations. For example,		compared over a range of input configurations. For example,
	compare PIE vs. CoDel over the range of target delay input		compare PIE vs. CoDel over the range of target delay input
	configurations.		configurations.
	4.3.2. Deployment comparison		4.3.2. Deployment comparison
	AQM schemes MUST be compared against deployment criteria such as the		AQM schemes must be compared against deployment criteria such as the
	parameter sensitivity (Section 8.3), auto-tuning (Section 12) or		parameter sensitivity (Section 8.3), auto-tuning (Section 12) or
	implementation cost (Section 11).		implementation cost (Section 11).
	4.4. Packet sizes and congestion notification		4.4. Packet sizes and congestion notification
	An AQM scheme may be considering packet sizes while generating		An AQM scheme may be considering packet sizes while generating
	congestion signals. [RFC7141] discusses the motivations behind this.		congestion signals [RFC7141]. For example, control packets such as
	For example, control packets such as DNS requests/responses, TCP		DNS requests/responses, TCP SYNs/ACKs are small, but their loss can
	SYNs/ACKs are small, but their loss can severely impact the		severely impact application performance. An AQM scheme may therefore
	application performance. An AQM scheme may therefore be biased		be biased towards small packets by dropping them with lower
	towards small packets by dropping them with smaller probability		probability compared to larger packets. However, such an AQM scheme
	compared to larger packets. However, such an AQM scheme is unfair to		is unfair to data senders generating larger packets. Data senders,
	data senders generating larger packets. Data senders, malicious or		malicious or otherwise, are motivated to take advantage of such AQM
	otherwise, are motivated to take advantage of such AQM scheme by		scheme by transmitting smaller packets, and could result in unsafe
	transmitting smaller packets, and could result in unsafe deployments		deployments and unhealthy transport and/or application designs.
	and unhealthy transport and/or application designs.
	An AQM scheme SHOULD adhere to the recommendations outlined in		An AQM scheme should adhere to the recommendations outlined in the
	[RFC7141], and SHOULD NOT provide undue advantage to flows with		best current practive for dropping and marking packets document
	smaller packets [RFC7567].		[RFC7141], and should not provide undue advantage to flows with
			smaller packets, such as discussed in the section 4.4 of the AQM
			recommendation document [RFC7567]. In order to evaluate if an AQM
			scheme is biased towards flows with smaller size packets, traffic can
			be generated, such as defined in Section 8.2.2, where half of the
			flows have smaller packets (e.g. 500 bytes packets) than the other
			half of the flow (e.g. 1500 bytes packets). In this case, the
			metrics reported could be the same as in Section 6.3, where Category
			I is the set of flows with smaller packets and Category II the one
			with larger packets. The bidirectional scenario could also be
			considered (Section 9.2).
	4.5. Interaction with ECN		4.5. Interaction with ECN
	Deployed AQM algorithms SHOULD implement Explicit Congestion		ECN [RFC3168] is an alternative that allows AQM schemes to signal
	Notification (ECN) as well as loss to signal congestion to endpoints		receivers about network congestion that does not use packet drop.
	[RFC7567]. ECN [RFC3168] is an alternative that allows AQM schemes		There are benefits of providing ECN support for an AQM scheme
	to signal receivers about network congestion that does not use packet		[WELZ2015].
	drop. The benefits of providing ECN support for an AQM scheme are
	described in [WELZ2015]. Section 3 of [WELZ2015] describes expected
	operation of routers enabling ECN. AQM schemes SHOULD NOT drop or
	remark packets solely because the ECT(0) or ECT(1) codepoints are
	used, and when ECN-capable SHOULD set a CE-mark on ECN-capable
	packets in the presence of incipient congestion.
	If the tested AQM scheme can support ECN [RFC7567], the testers MUST		If the tested AQM scheme can support ECN, the testers must discuss
	discuss and describe the support of ECN. Since these guidelines can		and describe the support of ECN, such as discussed in the AQM
	be used to evaluate the performance of the tested AQM with and		recommendation [RFC7567]. Also, the AQM's ECN support can be studied
	without ECN markings, they could also be used to quantify the		and verified by replicating tests in Section 8.1 with ECN turned ON
	interest of enabling ECN.		at the TCP senders. The results can be used to not only evaluate the
			performance of the tested AQM with and without ECN markings, but also
			quantify the interest of enabling ECN.
	4.6. Interaction with Scheduling		4.6. Interaction with Scheduling
	A network device may use per-flow or per-class queuing with a		A network device may use per-flow or per-class queuing with a
	scheduling algorithm to either prioritize certain applications or		scheduling algorithm to either prioritize certain applications or
	classes of traffic, limit the rate of transmission, or to provide		classes of traffic, limit the rate of transmission, or to provide
	isolation between different traffic flows within a common class		isolation between different traffic flows within a common class, such
			as discussed in the section 2.1 of the AQM recommendation document
	[RFC7567].		[RFC7567].
	The scheduling and the AQM conjointly impact on the end-to-end		The scheduling and the AQM conjointly impact on the end-to-end
	performance. Therefore, the AQM proposal MUST discuss the		performance. Therefore, the AQM proposal must discuss the
	feasibility to add scheduling combined with the AQM algorithm. This		feasibility to add scheduling combined with the AQM algorithm. It
	discussion as an instance, MAY explain whether the dropping policy is		can be explained whether the dropping policy is applied when packets
	applied when packets are being enqueued or dequeued.		are being enqueued or dequeued.
	These guidelines do not propose guidelines to assess the performance		These guidelines do not propose guidelines to assess the performance
	of scheduling algorithms. Indeed, as opposed to characterizing AQM		of scheduling algorithms. Indeed, as opposed to characterizing AQM
	schemes that is related to their capacity to control the queuing		schemes that is related to their capacity to control the queuing
	delay in a queue, characterizing scheduling schemes is related to the		delay in a queue, characterizing scheduling schemes is related to the
	scheduling itself and its interaction with the AQM scheme. As one		scheduling itself and its interaction with the AQM scheme. As one
	example, the scheduler may create sub-queues and the AQM scheme may		example, the scheduler may create sub-queues and the AQM scheme may
	be applied on each of the sub-queues, and/or the AQM could be applied		be applied on each of the sub-queues, and/or the AQM could be applied
	on the whole queue. Also, schedulers might, such as FQ-CoDel		on the whole queue. Also, schedulers might, such as FQ-CoDel
	[HOEI2015] or FavorQueue [ANEL2014], introduce flow prioritization.		[HOEI2015] or FavorQueue [ANEL2014], introduce flow prioritization.
	skipping to change at page 17, line 13 ¶		skipping to change at page 18, line 24 ¶
	Control Protocol (TCP) [RFC0793] with a limited number of		Control Protocol (TCP) [RFC0793] with a limited number of
	applications. TCP is a widely deployed transport. It fills up		applications. TCP is a widely deployed transport. It fills up
	available buffers until a sender transfering a bulk flow with TCP		available buffers until a sender transfering a bulk flow with TCP
	receives a signal (packet drop) that reduces the sending rate. The		receives a signal (packet drop) that reduces the sending rate. The
	larger the buffer, the higher the buffer occupancy, and therefore the		larger the buffer, the higher the buffer occupancy, and therefore the
	queuing delay. An efficient AQM scheme sends out early congestion		queuing delay. An efficient AQM scheme sends out early congestion
	signals to TCP to bring the queuing delay under control.		signals to TCP to bring the queuing delay under control.
	Not all endpoints (or applications) using TCP use the same flavor of		Not all endpoints (or applications) using TCP use the same flavor of
	TCP. Variety of senders generate different classes of traffic which		TCP. Variety of senders generate different classes of traffic which
	may not react to congestion signals (aka non-responsive flows		may not react to congestion signals (aka non-responsive flows in the
	[RFC7567]) or may not reduce their sending rate as expected (aka		section 3 of the AQM recommendation document [RFC7567]) or may not
	Transport Flows that are less responsive than TCP[RFC7567], also		reduce their sending rate as expected (aka Transport Flows that are
	called "aggressive flows"). In these cases, AQM schemes seek to		less responsive than TCP, such as proposed in the section 3 of the
	control the queuing delay.		AQM recommendation document [RFC7567], also called "aggressive
			flows"). In these cases, AQM schemes seek to control the queuing
			delay.
	This section provides guidelines to assess the performance of an AQM		This section provides guidelines to assess the performance of an AQM
	proposal for various traffic profiles -- different types of senders		proposal for various traffic profiles -- different types of senders
	(with different TCP congestion control variants, unresponsive,		(with different TCP congestion control variants, unresponsive,
	aggressive).		aggressive).
	5.1. TCP-friendly sender		5.1. TCP-friendly sender
	5.1.1. TCP-friendly sender with the same initial congestion window		5.1.1. TCP-friendly sender with the same initial congestion window
	This scenario helps to evaluate how an AQM scheme reacts to a TCP-		This scenario helps to evaluate how an AQM scheme reacts to a TCP-
	friendly transport sender. A single long-lived, non application-		friendly transport sender. A single long-lived, non application-
	limited, TCP NewReno flow, with an Initial congestion Window (IW) set		limited, TCP NewReno flow, with an Initial congestion Window (IW) set
	to 3 packets, transfers data between sender A and receiver B. Other		to 3 packets, transfers data between sender A and receiver B. Other
	TCP friendly congestion control schemes such as TCP-friendly rate		TCP friendly congestion control schemes such as TCP-friendly rate
	control [RFC5348] etc MAY also be considered.		control [RFC5348] etc may also be considered.
	For each TCP-friendly transport considered, the graph described in		For each TCP-friendly transport considered, the graph described in
	Section 2.7 could be generated.		Section 2.7 could be generated.
	5.1.2. TCP-friendly sender with different initial congestion windows		5.1.2. TCP-friendly sender with different initial congestion windows
	This scenario can be used to evaluate how an AQM scheme adapts to a		This scenario can be used to evaluate how an AQM scheme adapts to a
	traffic mix consisting of TCP flows with different values of the IW.		traffic mix consisting of TCP flows with different values of the IW.
	For this scenario, two types of flows MUST be generated between		For this scenario, two types of flows must be generated between
	sender A and receiver B:		sender A and receiver B:
	o A single long-lived non application-limited TCP NewReno flow;		o A single long-lived non application-limited TCP NewReno flow;
	o A single application-limited TCP NewReno flow, with an IW set to 3		o A single application-limited TCP NewReno flow, with an IW set to 3
	or 10 packets. The size of the data transferred must be strictly		or 10 packets. The size of the data transferred must be strictly
	higher than 10 packets and should be lower than 100 packets.		higher than 10 packets and should be lower than 100 packets.
	The transmission of the non application-limited flow must start		The transmission of the non application-limited flow must start first
	before the transmission of the application-limited flow and only		and the transmission of the application-limited flow starts after the
	after the steady state has been reached by non application-limited		non application-limited flow has reached steady state. The steady
	flow.		state can be assumed when the goodput is stable.
	For each of these scenarios, the graph described in Section 2.7 could		For each of these scenarios, the graph described in Section 2.7 could
	be generated for each class of traffic (application-limited and non		be generated for each class of traffic (application-limited and non
	application-limited). The completion time of the application-limited		application-limited). The completion time of the application-limited
	TCP flow could be measured.		TCP flow could be measured.
	5.2. Aggressive transport sender		5.2. Aggressive transport sender
	This scenario helps testers to evaluate how an AQM scheme reacts to a		This scenario helps testers to evaluate how an AQM scheme reacts to a
	transport sender that is more aggressive than a single TCP-friendly		transport sender that is more aggressive than a single TCP-friendly
	sender. We define 'aggressiveness' as a higher increase factor than		sender. We define 'aggressiveness' as a higher increase factor than
	standard upon a successful transmission and/or a lower than standard		standard upon a successful transmission and/or a lower than standard
	decrease factor upon a unsuccessful transmission (e.g., in case of		decrease factor upon a unsuccessful transmission (e.g., in case of
	congestion controls with Additive-Increase Multiplicative-Decrease		congestion controls with Additive-Increase Multiplicative-Decrease
	(AIMD) principle, a larger AI and/or MD factors). A single long-		(AIMD) principle, a larger AI and/or MD factors). A single long-
	lived, non application-limited, TCP Cubic flow transfers data between		lived, non application-limited, TCP Cubic flow transfers data between
	sender A and receiver B. Other aggressive congestion control schemes		sender A and receiver B. Other aggressive congestion control schemes
	MAY also be considered.		may also be considered.
	For each flavor of aggressive transports, the graph described in		For each flavor of aggressive transports, the graph described in
	Section 2.7 could be generated.		Section 2.7 could be generated.
	5.3. Unresponsive transport sender		5.3. Unresponsive transport sender
	This scenario helps testers to evaluate how an AQM scheme reacts to a		This scenario helps testers to evaluate how an AQM scheme reacts to a
	transport sender that is less responsive than TCP. Note that faulty		transport sender that is less responsive than TCP. Note that faulty
	transport implementations on an end host and/or faulty network		transport implementations on an end host and/or faulty network
	elements en-route that "hide" congestion signals in packet headers		elements en-route that "hide" congestion signals in packet headers
	[RFC7567] may also lead to a similar situation, such that the AQM		may also lead to a similar situation, such that the AQM scheme needs
	scheme needs to adapt to unresponsive traffic. To this end, these		to adapt to unresponsive traffic (see the section 3 of the AQM
	guidelines propose the two following scenarios.		recommendation document [RFC7567]). To this end, these guidelines
			propose the two following scenarios.
	The first scenario can be used to evaluate queue build up. It		The first scenario can be used to evaluate queue build up. It
	considers unresponsive flow(s) whose sending rate is greater than the		considers unresponsive flow(s) whose sending rate is greater than the
	bottleneck link capacity between routers L and R. This scenario		bottleneck link capacity between routers L and R. This scenario
	consists of a long-lived non application limited UDP flow transmits		consists of a long-lived non application limited UDP flow transmits
	data between sender A and receiver B. Graphs described in		data between sender A and receiver B. Graphs described in
	Section 2.7 could be generated.		Section 2.7 could be generated.
	The second scenario can be used to evaluate if the AQM scheme is able		The second scenario can be used to evaluate if the AQM scheme is able
	to keep the responsive fraction under control. This scenario		to keep the responsive fraction under control. This scenario
	skipping to change at page 19, line 16 ¶		skipping to change at page 20, line 31 ¶
	the first scenario, the rate of the UDP traffic should not be greater		the first scenario, the rate of the UDP traffic should not be greater
	than the bottleneck capacity, and should be higher than half of the		than the bottleneck capacity, and should be higher than half of the
	bottleneck capacity. For each type of traffic, the graph described		bottleneck capacity. For each type of traffic, the graph described
	in Section 2.7 could be generated.		in Section 2.7 could be generated.
	5.4. Less-than Best Effort transport sender		5.4. Less-than Best Effort transport sender
	This scenario helps to evaluate how an AQM scheme reacts to LBE		This scenario helps to evaluate how an AQM scheme reacts to LBE
	congestion controls that 'results in smaller bandwidth and/or delay		congestion controls that 'results in smaller bandwidth and/or delay
	impact on standard TCP than standard TCP itself, when sharing a		impact on standard TCP than standard TCP itself, when sharing a
	bottleneck with it.' [RFC6297]. The potential fateful interaction		bottleneck with it.' [RFC6297]. There are potential fateful
	when AQM and LBE techniques are combined has been shown in		interactions when AQM and LBE techniques are combined [GONG2014];
	[GONG2014]; this scenario helps to evaluate whether the coexistence		this scenario helps to evaluate whether the coexistence of the
	of the proposed AQM and LBE techniques may be possible.		proposed AQM and LBE techniques may be possible.
	A single long-lived non application-limited TCP NewReno flow		A single long-lived non application-limited TCP NewReno flow
	transfers data between sender A and receiver B. Other TCP-friendly		transfers data between sender A and receiver B. Other TCP-friendly
	congestion control schemes MAY also be considered. Single long-lived		congestion control schemes may also be considered. Single long-lived
	non application-limited LEDBAT [RFC6817] flows transfer data between		non application-limited LEDBAT [RFC6817] flows transfer data between
	sender A and receiver B. We recommend to set the target delay and		sender A and receiver B. We recommend to set the target delay and
	gain values of LEDBAT respectively to 5 ms and 10 [TRAN2014]. Other		gain values of LEDBAT respectively to 5 ms and 10 [TRAN2014]. Other
	LBE congestion control schemes, any of those listed in [RFC6297], MAY		LBE congestion control schemes may also be considered and are listed
	also be considered.		in the IETF survey of LBE protocols [RFC6297].
	For each of the TCP-friendly and LBE transports, the graph described		For each of the TCP-friendly and LBE transports, the graph described
	in Section 2.7 could be generated.		in Section 2.7 could be generated.
	6. Round Trip Time Fairness		6. Round Trip Time Fairness
	6.1. Motivation		6.1. Motivation
	An AQM scheme's congestion signals (via drops or ECN marks) must		An AQM scheme's congestion signals (via drops or ECN marks) must
	reach the transport sender so that a responsive sender can initiate		reach the transport sender so that a responsive sender can initiate
	its congestion control mechanism and adjust the sending rate. This		its congestion control mechanism and adjust the sending rate. This
	procedure is thus dependent on the end-to-end path RTT. When the RTT		procedure is thus dependent on the end-to-end path RTT. When the RTT
	varies, the onset of congestion control is impacted, and in turn		varies, the onset of congestion control is impacted, and in turn
	impacts the ability of an AQM scheme to control the queue. It is		impacts the ability of an AQM scheme to control the queue. It is
	therefore important to assess the AQM schemes for a set of RTTs		therefore important to assess the AQM schemes for a set of RTTs
	between A and B (e.g., from 5 ms to 200 ms).		between A and B (e.g., from 5 ms to 200 ms).
	The asymmetry in terms of difference in intrinsic RTT between various		The asymmetry in terms of difference in intrinsic RTT between various
	paths sharing the same bottleneck SHOULD be considered so that the		paths sharing the same bottleneck should be considered, so that the
	fairness between the flows can be discussed since in this scenario, a		fairness between the flows can be discussed. In this scenario, a
	flow traversing on shorter RTT path may react faster to congestion		flow traversing on shorter RTT path may react faster to congestion
	and recover faster from it compared to another flow on a longer RTT		and recover faster from it compared to another flow on a longer RTT
	path. The introduction of AQM schemes may potentially improve this		path. The introduction of AQM schemes may potentially improve the
	type of fairness.		RTT fairness.
	Introducing an AQM scheme may cause the unfairness between the flows,		Introducing an AQM scheme may cause the unfairness between the flows,
	even if the RTTs are identical. This potential unfairness SHOULD be		even if the RTTs are identical. This potential unfairness should be
	investigated as well.		investigated as well.
	6.2. Recommended tests		6.2. Recommended tests
	The RECOMMENDED topology is detailed in Figure 1.		The recommended topology is detailed in Figure 1.
	To evaluate the RTT fairness, for each run, two flows divided into		To evaluate the RTT fairness, for each run, two flows are divided
	two categories. Category I whose RTT between sender A and receiver B		into two categories. Category I whose RTT between sender A and
	SHOULD be 100ms. Category II which RTT between sender A and receiver		receiver B should be 100ms. Category II which RTT between sender A
	B should be in the range [5ms;560ms] inclusive. The maximum value		and receiver B should be in the range [5ms;560ms] inclusive. The
	for the RTT represents the RTT of a satellite link that, according to		maximum value for the RTT represents the RTT of a satellite link
	section 2 of [RFC2488] should be at least 558ms.		[RFC2488].
	A set of evaluated flows MUST use the same congestion control		A set of evaluated flows must use the same congestion control
	algorithm: all the generated flows could be single long-lived non		algorithm: all the generated flows could be single long-lived non
	application-limited TCP NewReno flows.		application-limited TCP NewReno flows.
	6.3. Metrics to evaluate the RTT fairness		6.3. Metrics to evaluate the RTT fairness
	The outputs that MUST be measured are: (1) the cumulative average		The outputs that must be measured are: (1) the cumulative average
	goodput of the flow from Category I, goodput_Cat_I (Section 2.5); (2)		goodput of the flow from Category I, goodput_Cat_I (Section 2.5); (2)
	the cumulative average goodput of the flow from Category II,		the cumulative average goodput of the flow from Category II,
	goodput_Cat_II (Section 2.5); (3) the ratio goodput_Cat_II/		goodput_Cat_II (Section 2.5); (3) the ratio goodput_Cat_II/
	goodput_Cat_I; (4) the average packet drop rate for each category		goodput_Cat_I; (4) the average packet drop rate for each category
	(Section 2.3).		(Section 2.3).
	7. Burst Absorption		7. Burst Absorption
	"AQM mechanisms need to control the overall queue sizes, to ensure		"AQM mechanisms need to control the overall queue sizes, to ensure
	that arriving bursts can be accommodated without dropping packets"		that arriving bursts can be accommodated without dropping packets"
	skipping to change at page 21, line 22 ¶		skipping to change at page 22, line 40 ¶
	in the buffer for bursts of arriving packets. The tolerance to		in the buffer for bursts of arriving packets. The tolerance to
	bursts of packets depends upon the number of packets in the queue,		bursts of packets depends upon the number of packets in the queue,
	which is directly linked to the AQM algorithm. Moreover, an AQM		which is directly linked to the AQM algorithm. Moreover, an AQM
	scheme may implement a feature controlling the maximum size of		scheme may implement a feature controlling the maximum size of
	accepted bursts, that can depend on the buffer occupancy or the		accepted bursts, that can depend on the buffer occupancy or the
	currently estimated queuing delay. The impact of the buffer size on		currently estimated queuing delay. The impact of the buffer size on
	the burst allowance may be evaluated.		the burst allowance may be evaluated.
	7.2. Recommended tests		7.2. Recommended tests
	For this scenario, tester MUST evaluate how the AQM performs with the		For this scenario, tester must evaluate how the AQM performs with a
	following traffic generated from sender A to receiver B:		traffic mixed that could be composed of (from sender A to receiver
			B):
	o Web traffic with IW10;
	o Bursty video frames;		o Burst of packets at the beginning of a transmission, such as web
			traffic with IW10;
	o Constant Bit Rate (CBR) UDP traffic.		o Applications that send large bursts of data, such as bursty video
			frames;
	o A single non application-limited bulk TCP flow as background		o Background traffic, such as Constant Bit Rate (CBR) UDP traffic
	traffic.		and/or A single non application-limited bulk TCP flow as
			background traffic.
	Figure 2 presents the various cases for the traffic that MUST be		Figure 2 presents the various cases for the traffic that must be
	generated between sender A and receiver B.		generated between sender A and receiver B.
	+-------------------------------------------------+		+-------------------------------------------------+
	|Case| Traffic Type |		|Case| Traffic Type |
	| +-----+------------+----+--------------------+		| +-----+------------+----+--------------------+
	| |Video|Web (IW 10)| CBR| Bulk TCP Traffic |		| |Video|Web (IW 10)| CBR| Bulk TCP Traffic |
	+----|-----|------------|----|--------------------|		+----|-----|------------|----|--------------------|
	|I | 0 | 1 | 1 | 0 |		|I | 0 | 1 | 1 | 0 |
	+----|-----|------------|----|--------------------|		+----|-----|------------|----|--------------------|
	|II | 0 | 1 | 1 | 1 |		|II | 0 | 1 | 1 | 1 |
	skipping to change at page 22, line 8 ¶		skipping to change at page 23, line 27 ¶
	|III | 1 | 1 | 1 | 0 |		|III | 1 | 1 | 1 | 0 |
	+----|-----|------------|----|--------------------|		+----|-----|------------|----|--------------------|
	|IV | 1 | 1 | 1 | 1 |		|IV | 1 | 1 | 1 | 1 |
	+----+-----+------------+----+--------------------+		+----+-----+------------+----+--------------------+
	Figure 2: Bursty traffic scenarios		Figure 2: Bursty traffic scenarios
	A new web page download could start after the previous web page		A new web page download could start after the previous web page
	download is finished. Each web page could be composed by at least 50		download is finished. Each web page could be composed by at least 50
	objects and the size of each object should be at least 1kB. 6 TCP		objects and the size of each object should be at least 1kB. 6 TCP
	parallel connections SHOULD be generated to download the objects,		parallel connections should be generated to download the objects,
	each parallel connections having an initial congestion window set to		each parallel connections having an initial congestion window set to
	10 packets.		10 packets.
	For each of these scenarios, the graph described in Section 2.7 could		For each of these scenarios, the graph described in Section 2.7 could
	be generated for each application. Metrics such as end-to-end		be generated for each application. Metrics such as end-to-end
	latency, jitter, flow completion time MAY be generated. For the		latency, jitter, flow completion time may be generated. For the
	cases of frame generation of bursty video traffic as well as the		cases of frame generation of bursty video traffic as well as the
	choice of web traffic pattern, these details and their presentation		choice of web traffic pattern, these details and their presentation
	are left to the testers.		are left to the testers.
	8. Stability		8. Stability
	8.1. Motivation		8.1. Motivation
	The safety of an AQM scheme is directly related to its stability		The safety of an AQM scheme is directly related to its stability
	under varying operating conditions such as varying traffic profiles		under varying operating conditions such as varying traffic profiles
	skipping to change at page 23, line 9 ¶		skipping to change at page 24, line 26 ¶
	Whether the target context is a not stable environment, the ability		Whether the target context is a not stable environment, the ability
	of an AQM scheme to maintain its control over the queuing delay and		of an AQM scheme to maintain its control over the queuing delay and
	buffer occupancy can be challenged. This document proposes		buffer occupancy can be challenged. This document proposes
	guidelines to assess the behavior of AQM schemes under varying		guidelines to assess the behavior of AQM schemes under varying
	congestion levels and varying draining rates.		congestion levels and varying draining rates.
	8.2. Recommended tests		8.2. Recommended tests
	Note that the traffic profiles explained below comprises non		Note that the traffic profiles explained below comprises non
	application-limited TCP flows. For each of the below scenarios, the		application-limited TCP flows. For each of the below scenarios, the
	graphs described in Section 2.7 SHOULD be generated, and the goodput		graphs described in Section 2.7 should be generated, and the goodput
	of the various flows should be cumulated. For Section 8.2.5 and		of the various flows should be cumulated. For Section 8.2.5 and
	Section 8.2.6 they SHOULD incorporate the results in per-phase basis		Section 8.2.6 they should incorporate the results in per-phase basis
	as well.		as well.
	Wherever the notion of time has explicitly mentioned in this		Wherever the notion of time has explicitly mentioned in this
	subsection, time 0 starts from the moment all TCP flows have already		subsection, time 0 starts from the moment all TCP flows have already
	reached their congestion avoidance phase.		reached their congestion avoidance phase.
	8.2.1. Definition of the congestion Level		8.2.1. Definition of the congestion Level
	In these guidelines, the congestion levels are represented by the		In these guidelines, the congestion levels are represented by the
	projected packet drop rate, had a drop-tail queue was chosen instead		projected packet drop rate, had a drop-tail queue was chosen instead
	skipping to change at page 24, line 48 ¶		skipping to change at page 26, line 15 ¶
	o Experiment 1: the capacity varies between two values within a		o Experiment 1: the capacity varies between two values within a
	large time-scale. As an example, the following phases may be		large time-scale. As an example, the following phases may be
	considered: phase I - 100Mbps during 0-20s; phase II - 10Mbps		considered: phase I - 100Mbps during 0-20s; phase II - 10Mbps
	during 20-40s; phase I again, and so on.		during 20-40s; phase I again, and so on.
	o Experiment 2: the capacity varies between two values within a		o Experiment 2: the capacity varies between two values within a
	short time-scale. As an example, the following phases may be		short time-scale. As an example, the following phases may be
	considered: phase I - 100Mbps during 0-100ms; phase II - 10Mbps		considered: phase I - 100Mbps during 0-100ms; phase II - 10Mbps
	during 100-200ms; phase I again, and so on.		during 100-200ms; phase I again, and so on.
	The tester MAY choose a phase time-interval value different than what		The tester may choose a phase time-interval value different than what
	is stated above, if the network's path conditions (such as bandwidth-		is stated above, if the network's path conditions (such as bandwidth-
	delay product) necessitate. In this case the choice of such time-		delay product) necessitate. In this case the choice of such time-
	interval value SHOULD be stated and elaborated.		interval value should be stated and elaborated.
	The tester MAY additionally evaluate the two mentioned scenarios		The tester may additionally evaluate the two mentioned scenarios
	(short-term and long-term capacity variations), during and/or		(short-term and long-term capacity variations), during and/or
	including TCP slow-start phase.		including TCP slow-start phase.
	More realistic fluctuating capacity patterns MAY be considered. The		More realistic fluctuating capacity patterns may be considered. The
	tester MAY choose to incorporate realistic scenarios with regards to		tester may choose to incorporate realistic scenarios with regards to
	common fluctuation of bandwidth in state-of-the-art technologies.		common fluctuation of bandwidth in state-of-the-art technologies.
	The scenario consists of TCP NewReno flows between sender A and		The scenario consists of TCP NewReno flows between sender A and
	receiver B. To better assess the impact of draining rates on the AQM		receiver B. To better assess the impact of draining rates on the AQM
	behavior, the tester MUST compare its performance with those of drop-		behavior, the tester must compare its performance with those of drop-
	tail and SHOULD provide a reference document for their proposal		tail and should provide a reference document for their proposal
	discussing performance and deployment compared to those of drop-tail.		discussing performance and deployment compared to those of drop-tail.
	Burst traffic, such as presented in Section 7.2, could also be		Burst traffic, such as presented in Section 7.2, could also be
	considered to assess the impact of varying available capacity on the		considered to assess the impact of varying available capacity on the
	burst absorption of the AQM.		burst absorption of the AQM.
	8.3. Parameter sensitivity and stability analysis		8.3. Parameter sensitivity and stability analysis
	The control law used by an AQM is the primary means by which the		The control law used by an AQM is the primary means by which the
	queuing delay is controlled. Hence understanding the control law is		queuing delay is controlled. Hence understanding the control law is
	critical to understanding the behavior of the AQM scheme. The		critical to understanding the behavior of the AQM scheme. The
	skipping to change at page 25, line 44 ¶		skipping to change at page 27, line 10 ¶
	Transports operating under the control of AQM experience the effect		Transports operating under the control of AQM experience the effect
	of multiple control loops that react over different timescales. It		of multiple control loops that react over different timescales. It
	is therefore important that proposed AQM schemes are seen to be		is therefore important that proposed AQM schemes are seen to be
	stable when they are deployed at multiple points of potential		stable when they are deployed at multiple points of potential
	congestion along an Internet path. The pattern of congestion signals		congestion along an Internet path. The pattern of congestion signals
	(loss or ECN-marking) arising from AQM methods also need to not		(loss or ECN-marking) arising from AQM methods also need to not
	adversely interact with the dynamics of the transport protocols that		adversely interact with the dynamics of the transport protocols that
	they control.		they control.
	AQM proposals SHOULD provide background material showing control		AQM proposals should provide background material showing control
	theoretic analysis of the AQM control law and the input parameter		theoretic analysis of the AQM control law and the input parameter
	space within which the control law operates as expected; or could use		space within which the control law operates as expected; or could use
	another way to discuss the stability of the control law. For		another way to discuss the stability of the control law. For
	parameters that are auto-tuned, the material SHOULD include stability		parameters that are auto-tuned, the material should include stability
	analysis of the auto-tuning mechanism(s) as well. Such analysis		analysis of the auto-tuning mechanism(s) as well. Such analysis
	helps to understand an AQM control law better and the network		helps to understand an AQM control law better and the network
	conditions/deployments under which the AQM is stable.		conditions/deployments under which the AQM is stable.
	9. Various Traffic Profiles		9. Various Traffic Profiles
	This section provides guidelines to assess the performance of an AQM		This section provides guidelines to assess the performance of an AQM
	proposal for various traffic profiles such as traffic with different		proposal for various traffic profiles such as traffic with different
	applications or bi-directional traffic.		applications or bi-directional traffic.
	skipping to change at page 26, line 24 ¶		skipping to change at page 27, line 38 ¶
	traffic mix consisting of different applications such as:		traffic mix consisting of different applications such as:
	o Bulk TCP transfer		o Bulk TCP transfer
	o Web traffic		o Web traffic
	o VoIP		o VoIP
	o Constant Bit Rate (CBR) UDP traffic		o Constant Bit Rate (CBR) UDP traffic
	o Adaptive video streaming		o Adaptive video streaming (either unidirectional or bidirectional)
	Various traffic mixes can be considered. These guidelines RECOMMEND		Various traffic mixes can be considered. These guidelines recommend
	to examine at least the following example: 1 bi-directional VoIP; 6		to examine at least the following example: 1 bi-directional VoIP; 6
	Web pages download (such as detailed in Section 7.2); 1 CBR; 1		Web pages download (such as detailed in Section 7.2); 1 CBR; 1
	Adaptive Video; 5 bulk TCP. Any other combinations could be		Adaptive Video; 5 bulk TCP. Any other combinations could be
	considered and should be carefully documented.		considered and should be carefully documented.
	For each scenario, the graph described in Section 2.7 could be		For each scenario, the graph described in Section 2.7 could be
	generated for each class of traffic. Metrics such as end-to-end		generated for each class of traffic. Metrics such as end-to-end
	latency, jitter and flow completion time MAY be reported.		latency, jitter and flow completion time may be reported.
	9.2. Bi-directional traffic		9.2. Bi-directional traffic
	Control packets such as DNS requests/responses, TCP SYNs/ACKs are		Control packets such as DNS requests/responses, TCP SYNs/ACKs are
	small, but their loss can severely impact the application		small, but their loss can severely impact the application
	performance. The scenario proposed in this section will help in		performance. The scenario proposed in this section will help in
	assessing whether the introduction of an AQM scheme increases the		assessing whether the introduction of an AQM scheme increases the
	loss probability of these important packets.		loss probability of these important packets.
	For this scenario, traffic MUST be generated in both downlink and		For this scenario, traffic must be generated in both downlink and
	uplink, such as defined in Section 3.1. These guidelines RECOMMEND		uplink, such as defined in Section 3.1. The amount of asymmetry
	to consider a mild congestion level and the traffic presented in		between the uplink and the downlink depends on the context. These
	Section 8.2.2 in both directions. In this case, the metrics reported		guidelines recommend to consider a mild congestion level and the
	MUST be the same as in Section 8.2 for each direction.		traffic presented in Section 8.2.2 in both directions. In this case,
			the metrics reported must be the same as in Section 8.2 for each
			direction.
	The traffic mix presented in Section 9.1 MAY also be generated in		The traffic mix presented in Section 9.1 may also be generated in
	both directions.		both directions.
	10. Multi-AQM Scenario		10. Example of multi-AQM scenario
	10.1. Motivation		10.1. Motivation
	Transports operating under the control of AQM experience the effect		Transports operating under the control of AQM experience the effect
	of multiple control loops that react over different timescales. It		of multiple control loops that react over different timescales. It
	is therefore important that proposed AQM schemes are seen to be		is therefore important that proposed AQM schemes are seen to be
	stable when they are deployed at multiple points of potential		stable when they are deployed at multiple points of potential
	congestion along an Internet path. The pattern of congestion signals		congestion along an Internet path. The pattern of congestion signals
	(loss or ECN-marking) arising from AQM methods also need to not		(loss or ECN-marking) arising from AQM methods also need to not
	adversely interact with the dynamics of the transport protocols that		adversely interact with the dynamics of the transport protocols that
	they control.		they control.
	10.2. Details on the evaluation scenario		10.2. Details on the evaluation scenario
	+---------+ +-----------+		+---------+ +-----------+
	|senders A|---+ +---|receivers A|		|senders A|---+ +---|receivers A|
	+---------+ | | +-----------+		+---------+ | | +-----------+
	+-----+---+ +---------+ +--+-----+		+-----+---+ +---------+ +--+-----+
	|Router L |--|Router M |--|Router R|		|Router L |--|Router M |--|Router R|
	|AQM | |AQM | |No AQM |		|AQM A | |AQM M | |No AQM |
	+---------+ +--+------+ +--+-----+		+---------+ +--+------+ +--+-----+
	+---------+ | | +-----------+		+---------+ | | +-----------+
	|senders B|-------------+ +---|receivers B|		|senders B|-------------+ +---|receivers B|
	+---------+ +-----------+		+---------+ +-----------+
	Figure 3: Topology for the Multi-AQM scenario		Figure 3: Topology for the Multi-AQM scenario
	This scenario can be used to evaluate how having AQM schemes in		Figure Figure 3 describes topology options for evaluating multi-AQM
	sequence impact the induced latency reduction, the induced goodput		scenarios. The AQM schemes are applied in sequence and impact the
	maximization and the trade-off between these two. The topology		induced latency reduction, the induced goodput maximization and the
	presented in Figure 3 could be used. AQM schemes introduced in		trade-off between these two. Note that AQM schemes A and B
	Router L and Router M should be the same; any other configurations		introduced in Routers L and M could be (I) same scheme with identical
	could be considered. For this scenario, it is recommended to		parameter values, (ii) same scheme with different parameter values,
	consider a mild congestion level, the number of flows specified in		or (iii) two different schemed. To best understand the interactions
	Section 8.2.2 being equally shared among senders A and B. Any other		and implications, the mild congestion scenario as described in
	relevant combination of congestion levels could be considered. We		Section 8.2.2 is recommended such that the number of flows is equally
	recommend to measure the metrics presented in Section 8.2.		shared among senders A and B. Other relevant combination of
			congestion levels could also be considered. We recommend to measure
			the metrics presented in Section 8.2.
	11. Implementation cost		11. Implementation cost
	11.1. Motivation		11.1. Motivation
	Successful deployment of AQM is directly related to its cost of		Successful deployment of AQM is directly related to its cost of
	implementation. Network devices can need hardware or software		implementation. Network devices can need hardware or software
	implementations of the AQM mechanism. Depending on a device's		implementations of the AQM mechanism. Depending on a device's
	capabilities and limitations, the device may or may not be able to		capabilities and limitations, the device may or may not be able to
	implement some or all parts of their AQM logic.		implement some or all parts of their AQM logic.
	AQM proposals SHOULD provide pseudo-code for the complete AQM scheme,		AQM proposals should provide pseudo-code for the complete AQM scheme,
	highlighting generic implementation-specific aspects of the scheme		highlighting generic implementation-specific aspects of the scheme
	such as "drop-tail" vs. "drop-head", inputs (e.g., current queuing		such as "drop-tail" vs. "drop-head", inputs (e.g., current queuing
	delay, queue length), computations involved, need for timers, etc.		delay, queue length), computations involved, need for timers, etc.
	This helps to identify costs associated with implementing the AQM		This helps to identify costs associated with implementing the AQM
	scheme on a particular hardware or software device. This also		scheme on a particular hardware or software device. This also
	facilitates discsusions around which kind of devices can easily		facilitates discsusions around which kind of devices can easily
	support the AQM and which cannot.		support the AQM and which cannot.
	11.2. Recommended discussion		11.2. Recommended discussion
	AQM proposals SHOULD highlight parts of their AQM logic that are		AQM proposals should highlight parts of their AQM logic that are
	device dependent and discuss if and how AQM behavior could be		device dependent and discuss if and how AQM behavior could be
	impacted by the device. For example, a queueing-delay based AQM		impacted by the device. For example, a queueing-delay based AQM
	scheme requires current queuing delay as input from the device. If		scheme requires current queuing delay as input from the device. If
	the device already maintains this value, then it can be trivial to		the device already maintains this value, then it can be trivial to
	implement the their AQM logic on the device. If the device provides		implement the their AQM logic on the device. If the device provides
	indirect means to estimate the queuing delay (for example:		indirect means to estimate the queuing delay (for example:
	timestamps, dequeuing rate), then the AQM behavior is sensitive to		timestamps, dequeuing rate), then the AQM behavior is sensitive to
	the precision of the queuing delay estimations are for that device.		the precision of the queuing delay estimations are for that device.
	Highlighting the sensitivity of an AQM scheme to queuing delay		Highlighting the sensitivity of an AQM scheme to queuing delay
	estimations helps implementers to identify appropriate means of		estimations helps implementers to identify appropriate means of
	skipping to change at page 28, line 50 ¶		skipping to change at page 30, line 26 ¶
	Any AQM scheme is likely to have parameters whose values affect the		Any AQM scheme is likely to have parameters whose values affect the
	control law and behaviour of an AQM. Exposing all these parameters		control law and behaviour of an AQM. Exposing all these parameters
	as control parameters to a network operator (or user) can easily		as control parameters to a network operator (or user) can easily
	result in a unsafe AQM deployment. Unexpected AQM behavior ensues		result in a unsafe AQM deployment. Unexpected AQM behavior ensues
	when parameter values are set improperly. A minimal number of		when parameter values are set improperly. A minimal number of
	control parameters minimizes the number of ways a user can break a		control parameters minimizes the number of ways a user can break a
	system where an AQM scheme is deployed at. Fewer control parameters		system where an AQM scheme is deployed at. Fewer control parameters
	make the AQM scheme more user-friendly and easier to deploy and		make the AQM scheme more user-friendly and easier to deploy and
	debug.		debug.
	[RFC7567] states "AQM algorithms SHOULD NOT require tuning of initial		"AQM algorithms should not require tuning of initial or configuration
	or configuration parameters in common use cases." A scheme ought to		parameters in common use cases." such as stated in the section 4.3 of
	expose only those parameters that control the macroscopic AQM		the AQM recommendation document [RFC7567]. A scheme ought to expose
	behavior such as queue delay threshold, queue length threshold, etc.		only those parameters that control the macroscopic AQM behavior such
			as queue delay threshold, queue length threshold, etc.
	Additionally, the safety of an AQM scheme is directly related to its		Additionally, the safety of an AQM scheme is directly related to its
	stability under varying operating conditions such as varying traffic		stability under varying operating conditions such as varying traffic
	profiles and fluctuating network conditions, as described in		profiles and fluctuating network conditions, as described in
	Section 8. Operating conditions vary often and hence the AQM needs		Section 8. Operating conditions vary often and hence the AQM needs
	to remain stable under these conditions without the need for		to remain stable under these conditions without the need for
	additional external tuning. If AQM parameters require tuning under		additional external tuning. If AQM parameters require tuning under
	these conditions, then the AQM must self-adapt necessary parameter		these conditions, then the AQM must self-adapt necessary parameter
	values by employing auto-tuning techniques.		values by employing auto-tuning techniques.
	12.2. Recommended discussion		12.2. Recommended discussion
	In order to understand an AQM's deployment considerations and		In order to understand an AQM's deployment considerations and
	performance under a specific environment, AQM proposals SHOULD		performance under a specific environment, AQM proposals should
	describe the parameters that control the macroscopic AQM behavior,		describe the parameters that control the macroscopic AQM behavior,
	and identify any parameters that require tuning to operational		and identify any parameters that require tuning to operational
	conditions. It could be interesting to also discuss that even if an		conditions. It could be interesting to also discuss that even if an
	AQM scheme may not adequately auto-tune its parameters, the resulting		AQM scheme may not adequately auto-tune its parameters, the resulting
	performance may not be optimal, but close to something reasonable.		performance may not be optimal, but close to something reasonable.
	If there are any fixed parameters within the AQM, their setting		If there are any fixed parameters within the AQM, their setting
	SHOULD be discussed and justified, to help understand whether a fixed		should be discussed and justified, to help understand whether a fixed
	parameter value is applicable for a particular environment.		parameter value is applicable for a particular environment.
	If an AQM scheme is evaluated with parameter(s) that were externally		If an AQM scheme is evaluated with parameter(s) that were externally
	tuned for optimization or other purposes, these values MUST be		tuned for optimization or other purposes, these values must be
	disclosed.		disclosed.
	13. Conclusion		13. Summary
	Figure 4 lists the scenarios and their requirements.		Figure 4 lists the scenarios and their requirements for an extended
			characterization of an AQM scheme.
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Scenario |Sec. |Requirement |		|Scenario |Sec. |Requirement |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Interaction with ECN | 4.5 |MUST be discussed if supported |		|Interaction with ECN | 4.5 |MUST be discussed if supported |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Interaction with Scheduling| 4.6 |Feasibility MUST be discussed |		|Interaction with Scheduling| 4.6 |Feasibility MUST be discussed |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Transport Protocols |5. | |		|Transport Protocols |5. | |
	| TCP-friendly sender | 5.1 |Scenario MUST be considered |		| TCP-friendly sender | 5.1 |Scenario MUST be considered |
	| Aggressive sender | 5.2 |Scenario MUST be considered |		| Aggressive sender | 5.2 |Scenario MUST be considered |
	| Unresponsive sender | 5.3 |Scenario MUST be considered |		| Unresponsive sender | 5.3 |Scenario MUST be considered |
	| LBE sender | 5.4 |Scenario MAY be considered |		| LBE sender | 5.4 |Scenario MAY be considered |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Round Trip Time Fairness | 6.2 |Scenario MUST be considered |		|Round Trip Time Fairness | 6.2 |Scenario MUST be considered |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Burst Absorption | 7.2 |Scenario MUST be considered |		|Burst Absorption | 7.2 |Scenario MUST be considered |
	skipping to change at page 30, line 34 ¶		skipping to change at page 31, line 47 ¶
	| Varying congestion levels | 8.2.5|Scenario MUST be considered |		| Varying congestion levels | 8.2.5|Scenario MUST be considered |
	| Varying available capacity| 8.2.6|Scenario MUST be considered |		| Varying available capacity| 8.2.6|Scenario MUST be considered |
	| Parameters and stability | 8.3 |This SHOULD be discussed |		| Parameters and stability | 8.3 |This SHOULD be discussed |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Various Traffic Profiles |9. | |		|Various Traffic Profiles |9. | |
	| Traffic mix | 9.1 |Scenario is RECOMMENDED |		| Traffic mix | 9.1 |Scenario is RECOMMENDED |
	| Bi-directional traffic | 9.2 |Scenario MAY be considered |		| Bi-directional traffic | 9.2 |Scenario MAY be considered |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Multi-AQM | 10.2 |Scenario MAY be considered |		|Multi-AQM | 10.2 |Scenario MAY be considered |
	+------------------------------------------------------------------+		+------------------------------------------------------------------+
	|Implementation Cost | 11.2 |Pseudo-code SHOULD be provided |
	+------------------------------------------------------------------+
	|Operator Control | 12.2 |Tuning SHOULD NOT be required |
	+------------------------------------------------------------------+
	Figure 4: Summary of the scenarios and their requirements		Figure 4: Summary of the scenarios and their requirements
	14. Acknowledgements		14. Acknowledgements
	This work has been partially supported by the European Community		This work has been partially supported by the European Community
	under its Seventh Framework Programme through the Reducing Internet		under its Seventh Framework Programme through the Reducing Internet
	Transport Latency (RITE) project (ICT-317700).		Transport Latency (RITE) project (ICT-317700).
	15. Contributors		15. Contributors
	skipping to change at page 31, line 21 ¶		skipping to change at page 32, line 32 ¶
	17. Security Considerations		17. Security Considerations
	Some security considerations for AQM are identified in [RFC7567].This		Some security considerations for AQM are identified in [RFC7567].This
	document, by itself, presents no new privacy nor security issues.		document, by itself, presents no new privacy nor security issues.
	18. References		18. References
	18.1. Normative References		18.1. Normative References
	[I-D.ietf-tcpm-cubic]
	Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and
	R. Scheffenegger, "CUBIC for Fast Long-Distance Networks",
	draft-ietf-tcpm-cubic-01 (work in progress), January 2016.
	[I-D.irtf-iccrg-tcpeval]
	Hayes, D., Ros, D., Andrew, L., and S. Floyd, "Common TCP
	Evaluation Suite", draft-irtf-iccrg-tcpeval-01 (work in
	progress), July 2014.
	[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
	RFC 793, DOI 10.17487/RFC0793, September 1981,
	<http://www.rfc-editor.org/info/rfc793>.
	[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate		[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
	Requirement Levels", RFC 2119, 1997.		Requirement Levels", RFC 2119, 1997.
	[RFC2488] Allman, M., Glover, D., and L. Sanchez, "Enhancing TCP
	Over Satellite Channels using Standard Mechanisms",
	BCP 28, RFC 2488, DOI 10.17487/RFC2488, January 1999,
	<http://www.rfc-editor.org/info/rfc2488>.
	[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for		[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
	Network Interconnect Devices", RFC 2544,		Network Interconnect Devices", RFC 2544,
	DOI 10.17487/RFC2544, March 1999,		DOI 10.17487/RFC2544, March 1999,
	<http://www.rfc-editor.org/info/rfc2544>.		<http://www.rfc-editor.org/info/rfc2544>.
	[RFC2647] Newman, D., "Benchmarking Terminology for Firewall		[RFC2647] Newman, D., "Benchmarking Terminology for Firewall
	Performance", RFC 2647, DOI 10.17487/RFC2647, August 1999,		Performance", RFC 2647, DOI 10.17487/RFC2647, August 1999,
	<http://www.rfc-editor.org/info/rfc2647>.		<http://www.rfc-editor.org/info/rfc2647>.
	[RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way		[RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
	Delay Metric for IPPM", RFC 2679, DOI 10.17487/RFC2679,		Delay Metric for IPPM", RFC 2679, DOI 10.17487/RFC2679,
	September 1999, <http://www.rfc-editor.org/info/rfc2679>.		September 1999, <http://www.rfc-editor.org/info/rfc2679>.
	[RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way		[RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
	Packet Loss Metric for IPPM", RFC 2680,		Packet Loss Metric for IPPM", RFC 2680,
	DOI 10.17487/RFC2680, September 1999,		DOI 10.17487/RFC2680, September 1999,
	<http://www.rfc-editor.org/info/rfc2680>.		<http://www.rfc-editor.org/info/rfc2680>.
	[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
	of Explicit Congestion Notification (ECN) to IP",
	RFC 3168, DOI 10.17487/RFC3168, September 2001,
	<http://www.rfc-editor.org/info/rfc3168>.
	[RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed.,
	"RTP Control Protocol Extended Reports (RTCP XR)",
	RFC 3611, DOI 10.17487/RFC3611, November 2003,
	<http://www.rfc-editor.org/info/rfc3611>.
	[RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP
	Friendly Rate Control (TFRC): Protocol Specification",
	RFC 5348, DOI 10.17487/RFC5348, September 2008,
	<http://www.rfc-editor.org/info/rfc5348>.
	[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation		[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation
	Applicability Statement", RFC 5481, DOI 10.17487/RFC5481,		Applicability Statement", RFC 5481, DOI 10.17487/RFC5481,
	March 2009, <http://www.rfc-editor.org/info/rfc5481>.		March 2009, <http://www.rfc-editor.org/info/rfc5481>.
	[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
	Control", RFC 5681, DOI 10.17487/RFC5681, September 2009,
	<http://www.rfc-editor.org/info/rfc5681>.
	[RFC6297] Welzl, M. and D. Ros, "A Survey of Lower-than-Best-Effort
	Transport Protocols", RFC 6297, DOI 10.17487/RFC6297, June
	2011, <http://www.rfc-editor.org/info/rfc6297>.
	[RFC6817] Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind,
	"Low Extra Delay Background Transport (LEDBAT)", RFC 6817,
	DOI 10.17487/RFC6817, December 2012,
	<http://www.rfc-editor.org/info/rfc6817>.
	[RFC7141] Briscoe, B. and J. Manner, "Byte and Packet Congestion
	Notification", RFC 7141, 2014.
	[RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF		[RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF
	Recommendations Regarding Active Queue Management",		Recommendations Regarding Active Queue Management",
	BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015,		BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015,
	<http://www.rfc-editor.org/info/rfc7567>.		<http://www.rfc-editor.org/info/rfc7567>.
	18.2. Informative References		18.2. Informative References
	[ANEL2014]		[ANEL2014]
	Anelli, P., Diana, R., and E. Lochin, "FavorQueue: a		Anelli, P., Diana, R., and E. Lochin, "FavorQueue: a
	Parameterless Active Queue Management to Improve TCP		Parameterless Active Queue Management to Improve TCP
	skipping to change at page 33, line 31 ¶		skipping to change at page 33, line 40 ¶
	[HASS2008]		[HASS2008]
	Hassayoun, S. and D. Ros, "Loss Synchronization and Router		Hassayoun, S. and D. Ros, "Loss Synchronization and Router
	Buffer Sizing with High-Speed Versions of TCP", IEEE		Buffer Sizing with High-Speed Versions of TCP", IEEE
	INFOCOM Workshops , 2008.		INFOCOM Workshops , 2008.
	[HOEI2015]		[HOEI2015]
	Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys,		Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys,
	J., and E. Dumazet, "FlowQueue-Codel", IETF (Work-in-		J., and E. Dumazet, "FlowQueue-Codel", IETF (Work-in-
	Progress) , January 2015.		Progress) , January 2015.
			[I-D.ietf-aqm-codel]
			Nichols, K., Jacobson, V., McGregor, A., and J. Iyengar,
			"Controlled Delay Active Queue Management", draft-ietf-
			aqm-codel-04 (work in progress), June 2016.
			[I-D.ietf-aqm-pie]
			Pan, R., Natarajan, P., Baker, F., and G. White, "PIE: A
			Lightweight Control Scheme To Address the Bufferbloat
			Problem", draft-ietf-aqm-pie-08 (work in progress), June
			2016.
			[I-D.ietf-tcpm-cubic]
			Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and
			R. Scheffenegger, "CUBIC for Fast Long-Distance Networks",
			draft-ietf-tcpm-cubic-01 (work in progress), January 2016.
			[I-D.irtf-iccrg-tcpeval]
			Hayes, D., Ros, D., Andrew, L., and S. Floyd, "Common TCP
			Evaluation Suite", draft-irtf-iccrg-tcpeval-01 (work in
			progress), July 2014.
	[JAY2006] Jay, P., Fu, Q., and G. Armitage, "A preliminary analysis		[JAY2006] Jay, P., Fu, Q., and G. Armitage, "A preliminary analysis
	of loss synchronisation between concurrent TCP flows",		of loss synchronisation between concurrent TCP flows",
	Australian Telecommunication Networks and Application		Australian Telecommunication Networks and Application
	Conference (ATNAC) , 2006.		Conference (ATNAC) , 2006.
	[MORR2000]		[MORR2000]
	Morris, R., "Scalable TCP congestion control", IEEE		Morris, R., "Scalable TCP congestion control", IEEE
	INFOCOM , 2000.		INFOCOM , 2000.
	[NICH2012]		[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
	Nichols, K. and V. Jacobson, "Controlling Queue Delay",		RFC 793, DOI 10.17487/RFC0793, September 1981,
	ACM Queue , 2012.		<http://www.rfc-editor.org/info/rfc793>.
	[PAN2013] Pan, R., Natarajan, P., Piglione, C., Prabhu, MS.,
	Subramanian, V., Baker, F., and B. VerSteeg, "PIE: A
	lightweight control scheme to address the bufferbloat
	problem", IEEE HPSR , 2013.
	[RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering,		[RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering,
	S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,		S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,
	Partridge, C., Peterson, L., Ramakrishnan, K., Shenker,		Partridge, C., Peterson, L., Ramakrishnan, K., Shenker,
	S., Wroclawski, J., and L. Zhang, "Recommendations on		S., Wroclawski, J., and L. Zhang, "Recommendations on
	Queue Management and Congestion Avoidance in the		Queue Management and Congestion Avoidance in the
	Internet", RFC 2309, April 1998.		Internet", RFC 2309, April 1998.
			[RFC2488] Allman, M., Glover, D., and L. Sanchez, "Enhancing TCP
			Over Satellite Channels using Standard Mechanisms",
			BCP 28, RFC 2488, DOI 10.17487/RFC2488, January 1999,
			<http://www.rfc-editor.org/info/rfc2488>.
			[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
			of Explicit Congestion Notification (ECN) to IP",
			RFC 3168, DOI 10.17487/RFC3168, September 2001,
			<http://www.rfc-editor.org/info/rfc3168>.
			[RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed.,
			"RTP Control Protocol Extended Reports (RTCP XR)",
			RFC 3611, DOI 10.17487/RFC3611, November 2003,
			<http://www.rfc-editor.org/info/rfc3611>.
			[RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP
			Friendly Rate Control (TFRC): Protocol Specification",
			RFC 5348, DOI 10.17487/RFC5348, September 2008,
			<http://www.rfc-editor.org/info/rfc5348>.
			[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
			Control", RFC 5681, DOI 10.17487/RFC5681, September 2009,
			<http://www.rfc-editor.org/info/rfc5681>.
			[RFC6297] Welzl, M. and D. Ros, "A Survey of Lower-than-Best-Effort
			Transport Protocols", RFC 6297, DOI 10.17487/RFC6297, June
			2011, <http://www.rfc-editor.org/info/rfc6297>.
			[RFC6817] Shalunov, S., Hazel, G., Iyengar, J., and M. Kuehlewind,
			"Low Extra Delay Background Transport (LEDBAT)", RFC 6817,
			DOI 10.17487/RFC6817, December 2012,
			<http://www.rfc-editor.org/info/rfc6817>.
			[RFC7141] Briscoe, B. and J. Manner, "Byte and Packet Congestion
			Notification", RFC 7141, 2014.
	[TRAN2014]		[TRAN2014]
	Trang, S., Kuhn, N., Lochin, E., Baudoin, C., Dubois, E.,		Trang, S., Kuhn, N., Lochin, E., Baudoin, C., Dubois, E.,
	and P. Gelard, "On The Existence Of Optimal LEDBAT		and P. Gelard, "On The Existence Of Optimal LEDBAT
	Parameters", IEEE ICC 2014 - Communication QoS,		Parameters", IEEE ICC 2014 - Communication QoS,
	Reliability and Modeling Symposium , 2014.		Reliability and Modeling Symposium , 2014.
	[WELZ2015]		[WELZ2015]
	Welzl, M. and G. Fairhurst, "The Benefits to Applications		Welzl, M. and G. Fairhurst, "The Benefits to Applications
	of using Explicit Congestion Notification (ECN)", IETF		of using Explicit Congestion Notification (ECN)", IETF
	(Work-in-Progress) , June 2015.		(Work-in-Progress) , June 2015.
End of changes. 139 change blocks.
	413 lines changed or deleted		459 lines changed or added
This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/