Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

["Mirja Kuehlewind (IETF)" <[e-mail]>]

Hi Al,

what kind of detail are you looking for? Because I thought with the given equation this one was pretty clear. 

Do you have a reference to the benchmarking work?

Mirja


> Am 08.06.2016 um 11:18 schrieb MORTON, ALFRED C (AL) <[e-mail]>:
> 
> Hi Mirja,
> 
> That sounds fairly reasonable to me.
> Would it be possible to ask the authors provide a bit more 
> detail on Flow Completion Time?
> 
>>>> Flow Completion Time is close to a definition for a new metric,
>>>> and could benefit from more attention, perhaps a few more details.
>>>> RFC6390 will provide some areas for improvement.
> 
> I imagine that related benchmarking efforts may wish to measure this 
> metric, and there would be independent implementations based on
> the description provided here.
> 
> regards from Geneve'
> Al
> 
>> -----Original Message-----
>> From: Mirja Kuehlewind (IETF) [mailto:[e-mail]]
>> Sent: Wednesday, June 08, 2016 4:46 AM
>> To: MORTON, ALFRED C (AL); Benoit Claise
>> Cc: [e-mail]; [e-mail]; The IESG; draft-ietf-aqm-eval-
>> [e-mail]; [e-mail]
>> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
>> guidelines-11: (with DISCUSS and COMMENT)
>> 
>> Hi Benoit,
>> 
>> I finally had another look at the document as well as at RFC6390. I
>> guess the metrics in question are Flow completion time (sec 2.1.), Flow
>> start up time (2.2.) and Packet loss synchronization (2.4.). And you are
>> right that these metric could be see as Application-Specific Performance
>> Metric as defined in RFC6390. However, I agree with Al that given the
>> scope of this document is providing
>> "a generic list of scenarios against which an
>>   AQM proposal should be evaluated, considering both potential
>>   performance gain and safety of deployment.“,
>> I don’t think these metric need to be defined and registered this way.
>> 
>> I guess as soon as we have a registry, maybe there is someone interest
>> in IPPM to catch up these metrics again and provide a RFC6390 definition
>> but I would rather not like this document doing it.
>> 
>> Is that acceptable for you?
>> 
>> We could add one more sentence in the abstract to make the scope on lab
>> testing during development and before deployment clear. Would that help?
>> 
>> Mirja
>> 
>> 
>> 
>>> Am 25.05.2016 um 14:22 schrieb Mirja Kuehlewind (IETF)
>> <[e-mail]>:
>>> 
>>> Hi Al, Benoit, hi all,
>>> 
>>> thanks for the feedback. Sorry for me delaying this maybe a little but
>> I need t have another look at the document which will be next week at
>> this point. In general I agree that this does not need to only rely on
>> registered metrics because is mostly for lab tests; further this might
>> probably not the right doc to register new metrics. However, I would
>> still like to have another look at the doc and see if we can improve
>> anything or figure out if any of the ’new’/non-registed metrics
>> should/could be taken up by e.g. ippm.
>>> 
>>> Mirja
>>> 
>>> 
>>> 
>>>> Am 20.05.2016 um 14:53 schrieb MORTON, ALFRED C (AL)
>> <[e-mail]>:
>>>> 
>>>> All,
>>>> a few replies in-line below,
>>>> Al
>>>> 
>>>>> -----Original Message-----
>>>>> From: Benoit Claise [mailto:[e-mail]]
>>>>> Sent: Thursday, May 19, 2016 5:38 AM
>>>>> To: The IESG
>>>>> Cc: [e-mail]; [e-mail];
>> aqm-
>>>>> [e-mail]; [e-mail]; [e-mail]; linda Dunbar;
>>>>> MORTON, ALFRED C (AL)
>>>>> Subject: Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-
>> 11:
>>>>> (with DISCUSS and COMMENT)
>>>>> 
>>>>> Benoit Claise has entered the following ballot position for
>>>>> draft-ietf-aqm-eval-guidelines-11: Discuss
>>>>> 
>>>> ...
>>>>> --------------------------------------------------------------------
>> --
>>>>> DISCUSS:
>>>>> --------------------------------------------------------------------
>> --
>>>>> 
>>>>> Has a RFC6390 performance directorate review done for the 2.X
>> metrics?
>>>>> It
>>>>> should.
>>>> [ACM]
>>>> I reviewed this draft about 18 months ago.
>>>> Mostly, it points to existing RFCs for fundamental metrics,
>>>> and discusses others.  I read this:
>>>> ...This document provides characterization guidelines that
>>>> can be used to assess the deployability of an AQM, whether it is
>>>> candidate for standardization at IETF or not.
>>>> as restricted to lab testing.
>>>> 
>>>>> See http://www.ietf.org/iesg/directorate/performance-metrics.html
>>>>> I guess that the metrics will be recorded in the future (See
>>>>> draft-ietf-ippm-metric-registry-06
>>>>> ), right?
>>>> [ACM]
>>>> That's up to the authors, they might simply point to
>>>> metrics in the registry contributed by others
>>>> (when following these guidelines at a future time).
>>>> 
>>>>> For example, Flow Completion Time and Packet Loss Synchronization
>> are
>>>>> new, I believe.
>>>> [ACM]
>>>> Flow Completion Time is close to a definition for a new metric,
>>>> and could benefit from more attention, perhaps a few more details.
>>>> RFC6390 will provide some areas for improvement.
>>>> 
>>>> Packet loss sync full methodology is described in [JAY006],
>>>> according to the text.
>>>> 
>>>>> And some other metrics are already documented in RFC6390 compliant
>>>>> documents. Pointers should be provided.
>>>> [ACM]
>>>> Most others are discussion sections and provide references.
>>>> 
>>>>> See
>>>>> https://tools.ietf.org/html/draft-ietf-xrblock-independent-burst-
>> gap-
>>>>> discard-01#appendix-A
>>>>> for an example
>>>>> 
>>>>> 
>>>>> --------------------------------------------------------------------
>> --
>>>>> COMMENT:
>>>>> --------------------------------------------------------------------
>> --
>>>>> 
>>>>> - Random Early Detection (RED), BLUE, and Proportional Integral
>>>>> controller (PI)
>>>>> Would you have references?
>>>>> 
>>>>> - BDP is mentioned a few times. Please expand.
>>>>> 
>>>>> - Glossary section = terminology section, right? If we want to be
>>>>> consistent across documents
>>>>> 
>>>>> - section 12.2. Why not a MUST below?
>>>>> In order to understand an AQM's deployment considerations and
>>>>> performance under a specific environment, AQM proposals SHOULD
>>>>> describe the parameters that control the macroscopic AQM behavior,
>>>>> and identify any parameters that require tuning to operational
>>>>> conditions.
>>>>> 
>>>> 
>>> 
>>> _______________________________________________
>>> aqm mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/aqm
>