Re: [aqm] Alissa Cooper's No Objection on draft-ietf-aqm-eval-guidelines-11: (with COMMENT)

[Kuhn Nicolas <[e-mail]>]

Dear Alissa Cooper, 

Thank you for your review and sorry for the delay of our answer. We will consider your comments as you can see inline.
We will push an updated version including the changes as soon as possible. 
Please find attached to this email the diff between v11 and expected v12.

Kind regards, 

The authors 

-----Message d'origine-----
De : Alissa Cooper [mailto:[e-mail]] 
Envoyé : mercredi 18 mai 2016 21:15
À : The IESG
Cc : [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Objet : Alissa Cooper's No Objection on draft-ietf-aqm-eval-guidelines-11: (with COMMENT)

Alissa Cooper has entered the following ballot position for
draft-ietf-aqm-eval-guidelines-11: No Objection

When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-aqm-eval-guidelines/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

(1) In the abstract, I'm not sure "precautionary" is really the right adjective.

[NK] (1) We have removed the word "precautionary" (we had other comments on it being the wrong word)

(2) Sec. 4.2 says: "However, the evaluation MUST be primarily based on externally observed end-to-end metrics." This seems like a bit of a weird use of normative language. It makes sense to me that you would normatively recommend how to setup an evaluation environment to be able to compare AQMs, but if people want to weigh their evaluations in the wrong the direction, there is not much to be done about it. It seems like this would be more useful if it focused on explaining why primarily focusing on e2e measurements is preferable.

[NK] (2) You are right on the normative language. We had this discussion before the IESG meeting and will update the normative language in the next release

(3) Overall I would recommend a pass through the whole document to check for the consistency with which normative language is used -- it seems like in some cases it isn't really necessary but it's there anyway, or its use is uneven (e.g., 4.3.2, 5.2, 6.1, 6.2, 6.3, 7.2 to name a few).
There also seems to be a mix of normative statements about what proposals should say and how evaluation environments should be setup. It would help if each time a normative statement was made it clearly distinguished who the recommendation is directed towards. This became especially confusing in Sec. 13, where it is unclear whether "Scenario MUST be considered"
means that an AQM spec needs to discuss this, or an AQM evaluation environment needs to test for this.

[NK] (3) Same as (2)

(4) Since this document provides guidelines for how to characterize AQM proposals, it seems inappropriate for it to be making normative recommendations about how AQM proposals should work, even if they are just re-statements of things from RFC 7567. In particular, these statements seem out of place:

	- Sec. 4.4: "An AQM scheme SHOULD adhere to the recommendations outlined in
   [RFC7141], and SHOULD NOT provide undue advantage to flows with
   smaller packets [RFC7567].
   
    - Sec. 4.5: "Deployed AQM algorithms SHOULD implement Explicit Congestion
   Notification (ECN) as well as loss to signal congestion to endpoints
   [RFC7567]."
   
   - Sec. 13: "Tuning SHOULD NOT be required"

[NK] (4) Indeed. We have tried to clarify the overlaps with RFC7567 and removed text when this had occurred. We have removed the normative language in our draft, to avoid confusion.

(5) Sec. 4.6 says: "This discussion as an instance, MAY explain whether the dropping policy is applied when packets are being enqueued or dequeued." I don't understand what "this discussion as an instance"
means.

[NK] (5) Text updated as follows: " It can be explained whether the dropping policy is applied when packets are being enqueued or dequeued"

(6) In Sec. 7.2, is there some justification that could be provided for choosing the particular parameters listed to simulate web traffic? E.g., are these parameters considered typical or is there a reference to some standard way of simulating web traffic that could be referenced?

[NK] (6) We believe that specifying that would hardly difficult - and would result in non-ending discussions, on whether it is better to consider HTTP2, HTTP1.1, etc. and this is out of the main scope of the draft, even if some directions could have been interesting.

(7) Also in Sec. 7.2, why is the detail for the web flow given but the detail for bursty video is not? Surely there is more than one way to generate or simulate bursty video -- does it not matter what the bursts look like or how they are timed for this measurement purpose?

[NK] (7) Same as for the web traffic, we believe that this is up to the tester, otherwise, it will be hard to get a consensus.

(8) In 9.1, I would suggest that bi-directional video is prevalent and important enough to be listed in the traffic mix analysis as well.

[NK] (8) Indeed, we have added some text to say that the adaptive video can be either unidirectional or bidirectional. 

(9) I think the statement in Sec. 17 assumes that the tests envisioned by this document will be based on active measurements of simulated traffic.
If that is a real scope limitation, and evaluation of AQMs based on passive measurement of traffic is out of scope, I think that should be stated somewhere in the document.

[NK] (9) We believe that there is no limitation on simulated traffic and that some aspects in the guidelines could be valide for passive measurements well.

(10) Just curious: has anyone thought about using LMAP, or the LMAP information model at least, to configure the kinds of tests envisioned in this document?

[NK] (10) I am afraid that we have not.