[Gen-art] Gen-ART review of draft-ietf-aqm-eval-guidelines-11

["Ralph Droms (rdroms)" <[e-mail]>]

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Document: draft-ietf-aqm-eval-guidelines-11
Reviewer: Ralph Droms
Review Date: 2016-04-28
IETF LC End Date: 2016-05-04
IESG Telechat date: 2016-05-19

Summary: This draft is on the right track but has open issues, described in the review.

In general, I think the document could be read, implemented and used to generate useful characterizations of AQM schemes.  However, the motivations for some of the measurements and scenarios seems weak to me, which might compromise the weight given to the conclusions drawn from the guidelines.

Major issues:

None.  However, the list of minor issues and nits, taken together, could be considered a major issue to be resolved before publication.

Minor issues:

I often react to the use of RFC 2119 language in an Informational document by asking is that language really necessary?  I'll ask the question here: in the context of this Informational document, which appears to be entirely advisory in providing guidelines, what does the use of RFC 2119 "requirements language" add to the meaning of the document.

Figure 1 is not clear to me.  Where are the physical links and interfaces?  Are there multiple physical senders and receivers or are "senders A" instantiated on a single host (does it make a difference)?  Are there static-sized buffers for each interface or do all the buffers share one memory space?

In section 3.1, is there a need to say something about the relative capacities of the various links and the rates at which the various flows generate traffic?

I would have trouble following the guidelines set out in section 4.3.1.  I can understand the need for consideration of the tunable control parameters when comparing different AQM schemes.  However, I don't know what "comparable" means for control parameters that are likely quite different between AQM schemes.  I also think one would want to compare optimal control settings for the different schemes, to compare best-case performance.  Or, for AQM schemes whose performance is highly dependent on operational conditions, one might want to compare settings that are sub-optimal for any particular test condition but that give better performance over a wide range of conditions.

Section 4.4 seems to give advice to the AQM designer rather than describe guidelines for characterization.  Section 4.4 should either be rewritten to give guidelines for structuring measurements to account for varying packet sizes or the section should be elided.

In section 4.5, what is the motivation for giving the advice about ECN to AQM designers?  I can understand that ECN will have affect the impact of AQM, but for this document I think the section should focus on measurement guidlines that account for that impact.

The specific topology in section 10 does not seem well-motivated to me.  Why is router R with no AQM included in the topology?  The choice of measurements is similarly not well-motivated.  Why would it not be of interest to run all the tests described earlier in the document?

Nits/editorial comments:

There are several instances of the word "advice" which should be replaced with "advise"; e.g., in section 2.3.

Last sentence of the abstract: I don't get the meaning of "precautionary characterizations of AQM schemes".  I recommend that the phrase be reworded

Section 1, first paragraph: The last sentence doesn't follow the rest of the paragraph and I recommend that it be elided.

Section 1, third paragraph: This text is redundant with the text in the Glossary section:

   When speaking of a specific queue in this
   document, "buffer occupancy" refers to the amount of data (measured
   in bytes or packets) that are in the queue, and the "maximum buffer
   size" refers to the maximum buffer occupancy.

Section 1, third paragraph:

OLD:

   In real
   implementations of switches, a global memory is often shared between
   the available devices, and thus, the maximum buffer size may vary
   over the time.

NEW:

   In switches and routers, a global memory space is often shared
   between the available interfaces, and thus, the maximum buffer size
   for any given interface may vary over the time.

Section 1, fifth paragraph, last sentence: Is this document just concerned with "deployability" or more generally with "applicability, performance and deployability"?

Section 1.1, first paragraph: Would it be helpful to qualify "goodput" as "goodput in individual flows", to contrast with "goodput at a router"?  If "goodput" is well-known in this community to be "flow goodput", no change is needed.

Section 1.1, second paragraph: What is "BDP", as in "BDP-sized buffer"?

Section 1.1, third paragraph, first sentence: "environment" is unclear; perhaps "deployment scenario" or "use case"?  I'm not sure what was meant.

Section 1.3: for completeness, a definition of "latency" would be useful.

Section 2, first paragraph: I'm going to sound pedantic here, but it seems to me this section is not just intended "to better quantify (1) the reduction of latency, (2) maximization of goodput and (3) the trade-off between these two" but also to define other performance metrics that get to the goal from the abstract of describing "various criteria for performing precautionary characterizations of AQM schemes"

Section 2, last paragraph: What is "application-limited traffic"?  Later, what is "non application-limited" traffic?

Section 2.2: s/of/and/

Section 2.5, first paragraph: s/induces/requires/  ???

Section 2.6, second paragraph is just not clear to me.  In particular, what is the antecedent of "they":

   The introduction of an AQM scheme would impact these
   metrics (end-to-end latency and jitter) and therefore they should be
   considered in the end-to-end evaluation of performance."

Section 5.1: is this new text correct?

OLD:

   The transmission of the non application-limited flow must start
   before the transmission of the application-limited flow and only
   after the steady state has been reached by non application-limited
   flow.

NEW:

   The transmission of the non application-limited flow must start
   first and the transmission of the application-limited flow starts
   after the non application-limited flow has reached steady state.

Throughout section 5, the text like "the graph described in Section 2.7 could be generated" seems redundant.

Section 6.1, second paragraph: I don't understand the last sentence.  What is the "type of fairness" that the AQM scheme might affect?

Section 6.2, second paragraph: The words up to the first period don't constitute a sentence.

In section 7.2, what is "IW10"?  Why are the particular traffic flow chosen in figure 2?  s/could/should/ in the last paragraph?