[aqm] Alissa Cooper's No Objection on draft-ietf-aqm-eval-guidelines-11: (with COMMENT)

["Alissa Cooper" <[e-mail]>]

Alissa Cooper has entered the following ballot position for
draft-ietf-aqm-eval-guidelines-11: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-aqm-eval-guidelines/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

(1) In the abstract, I'm not sure "precautionary" is really the right
adjective.

(2) Sec. 4.2 says: "However, the evaluation MUST be primarily based on
externally observed end-to-end metrics." This seems like a bit of a weird
use of normative language. It makes sense to me that you would
normatively recommend how to setup an evaluation environment to be able
to compare AQMs, but if people want to weigh their evaluations in the
wrong the direction, there is not much to be done about it. It seems like
this would be more useful if it focused on explaining why primarily
focusing on e2e measurements is preferable.

(3) Overall I would recommend a pass through the whole document to check
for the consistency with which normative language is used -- it seems
like in some cases it isn't really necessary but it's there anyway, or
its use is uneven (e.g., 4.3.2, 5.2, 6.1, 6.2, 6.3, 7.2 to name a few).
There also seems to be a mix of normative statements about what proposals
should say and how evaluation environments should be setup. It would help
if each time a normative statement was made it clearly distinguished who
the recommendation is directed towards. This became especially confusing
in Sec. 13, where it is unclear whether "Scenario MUST be considered"
means that an AQM spec needs to discuss this, or an AQM evaluation
environment needs to test for this.

(4) Since this document provides guidelines for how to characterize AQM
proposals, it seems inappropriate for it to be making normative
recommendations about how AQM proposals should work, even if they are
just re-statements of things from RFC 7567. In particular, these
statements seem out of place:

	- Sec. 4.4: "An AQM scheme SHOULD adhere to the recommendations outlined
in
   [RFC7141], and SHOULD NOT provide undue advantage to flows with
   smaller packets [RFC7567].
   
    - Sec. 4.5: "Deployed AQM algorithms SHOULD implement Explicit
Congestion
   Notification (ECN) as well as loss to signal congestion to endpoints
   [RFC7567]."
   
   - Sec. 13: "Tuning SHOULD NOT be required"

(5) Sec. 4.6 says: "This discussion as an instance, MAY explain whether
the dropping policy is applied when packets are being enqueued or
dequeued." I don't understand what "this discussion as an instance"
means.

(6) In Sec. 7.2, is there some justification that could be provided for
choosing the particular parameters listed to simulate web traffic? E.g.,
are these parameters considered typical or is there a reference to some
standard way of simulating web traffic that could be referenced?

(7) Also in Sec. 7.2, why is the detail for the web flow given but the
detail for bursty video is not? Surely there is more than one way to
generate or simulate bursty video -- does it not matter what the bursts
look like or how they are timed for this measurement purpose?

(8) In 9.1, I would suggest that bi-directional video is prevalent and
important enough to be listed in the traffic mix analysis as well.

(9) I think the statement in Sec. 17 assumes that the tests envisioned by
this document will be based on active measurements of simulated traffic.
If that is a real scope limitation, and evaluation of AQMs based on
passive measurement of traffic is out of scope, I think that should be
stated somewhere in the document.

(10) Just curious: has anyone thought about using LMAP, or the LMAP
information model at least, to configure the kinds of tests envisioned in
this document?