Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

Hi Mirja,

That sounds fairly reasonable to me.
Would it be possible to ask the authors provide a bit more 
detail on Flow Completion Time?

> >> Flow Completion Time is close to a definition for a new metric,
> >> and could benefit from more attention, perhaps a few more details.
> >> RFC6390 will provide some areas for improvement.

I imagine that related benchmarking efforts may wish to measure this 
metric, and there would be independent implementations based on
the description provided here.

regards from Geneve'
Al

> -----Original Message-----
> From: Mirja Kuehlewind (IETF) [mailto:ietf@kuehlewind.net]
> Sent: Wednesday, June 08, 2016 4:46 AM
> To: MORTON, ALFRED C (AL); Benoit Claise
> Cc: wes@mti-systems.com; aqm@ietf.org; The IESG; draft-ietf-aqm-eval-
> guidelines@ietf.org; aqm-chairs@ietf.org
> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
> guidelines-11: (with DISCUSS and COMMENT)
> 
> Hi Benoit,
> 
> I finally had another look at the document as well as at RFC6390. I
> guess the metrics in question are Flow completion time (sec 2.1.), Flow
> start up time (2.2.) and Packet loss synchronization (2.4.). And you are
> right that these metric could be see as Application-Specific Performance
> Metric as defined in RFC6390. However, I agree with Al that given the
> scope of this document is providing
> "a generic list of scenarios against which an
>    AQM proposal should be evaluated, considering both potential
>    performance gain and safety of deployment.“,
> I don’t think these metric need to be defined and registered this way.
> 
> I guess as soon as we have a registry, maybe there is someone interest
> in IPPM to catch up these metrics again and provide a RFC6390 definition
> but I would rather not like this document doing it.
> 
> Is that acceptable for you?
> 
> We could add one more sentence in the abstract to make the scope on lab
> testing during development and before deployment clear. Would that help?
> 
> Mirja
> 
> 
> 
> > Am 25.05.2016 um 14:22 schrieb Mirja Kuehlewind (IETF)
> <ietf@kuehlewind.net>:
> >
> > Hi Al, Benoit, hi all,
> >
> > thanks for the feedback. Sorry for me delaying this maybe a little but
> I need t have another look at the document which will be next week at
> this point. In general I agree that this does not need to only rely on
> registered metrics because is mostly for lab tests; further this might
> probably not the right doc to register new metrics. However, I would
> still like to have another look at the doc and see if we can improve
> anything or figure out if any of the ’new’/non-registed metrics
> should/could be taken up by e.g. ippm.
> >
> > Mirja
> >
> >
> >
> >> Am 20.05.2016 um 14:53 schrieb MORTON, ALFRED C (AL)
> <acmorton@att.com>:
> >>
> >> All,
> >> a few replies in-line below,
> >> Al
> >>
> >>> -----Original Message-----
> >>> From: Benoit Claise [mailto:bclaise@cisco.com]
> >>> Sent: Thursday, May 19, 2016 5:38 AM
> >>> To: The IESG
> >>> Cc: draft-ietf-aqm-eval-guidelines@ietf.org; wes@mti-systems.com;
> aqm-
> >>> chairs@ietf.org; wes@mti-systems.com; aqm@ietf.org; linda Dunbar;
> >>> MORTON, ALFRED C (AL)
> >>> Subject: Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-
> 11:
> >>> (with DISCUSS and COMMENT)
> >>>
> >>> Benoit Claise has entered the following ballot position for
> >>> draft-ietf-aqm-eval-guidelines-11: Discuss
> >>>
> >> ...
> >>> --------------------------------------------------------------------
> --
> >>> DISCUSS:
> >>> --------------------------------------------------------------------
> --
> >>>
> >>> Has a RFC6390 performance directorate review done for the 2.X
> metrics?
> >>> It
> >>> should.
> >> [ACM]
> >> I reviewed this draft about 18 months ago.
> >> Mostly, it points to existing RFCs for fundamental metrics,
> >> and discusses others.  I read this:
> >>  ...This document provides characterization guidelines that
> >>  can be used to assess the deployability of an AQM, whether it is
> >>  candidate for standardization at IETF or not.
> >> as restricted to lab testing.
> >>
> >>> See http://www.ietf.org/iesg/directorate/performance-metrics.html
> >>> I guess that the metrics will be recorded in the future (See
> >>> draft-ietf-ippm-metric-registry-06
> >>> ), right?
> >> [ACM]
> >> That's up to the authors, they might simply point to
> >> metrics in the registry contributed by others
> >> (when following these guidelines at a future time).
> >>
> >>> For example, Flow Completion Time and Packet Loss Synchronization
> are
> >>> new, I believe.
> >> [ACM]
> >> Flow Completion Time is close to a definition for a new metric,
> >> and could benefit from more attention, perhaps a few more details.
> >> RFC6390 will provide some areas for improvement.
> >>
> >> Packet loss sync full methodology is described in [JAY006],
> >> according to the text.
> >>
> >>> And some other metrics are already documented in RFC6390 compliant
> >>> documents. Pointers should be provided.
> >> [ACM]
> >> Most others are discussion sections and provide references.
> >>
> >>> See
> >>> https://tools.ietf.org/html/draft-ietf-xrblock-independent-burst-
> gap-
> >>> discard-01#appendix-A
> >>> for an example
> >>>
> >>>
> >>> --------------------------------------------------------------------
> --
> >>> COMMENT:
> >>> --------------------------------------------------------------------
> --
> >>>
> >>> - Random Early Detection (RED), BLUE, and Proportional Integral
> >>> controller (PI)
> >>> Would you have references?
> >>>
> >>> - BDP is mentioned a few times. Please expand.
> >>>
> >>> - Glossary section = terminology section, right? If we want to be
> >>> consistent across documents
> >>>
> >>> - section 12.2. Why not a MUST below?
> >>>  In order to understand an AQM's deployment considerations and
> >>>  performance under a specific environment, AQM proposals SHOULD
> >>>  describe the parameters that control the macroscopic AQM behavior,
> >>>  and identify any parameters that require tuning to operational
> >>>  conditions.
> >>>
> >>
> >
> > _______________________________________________
> > aqm mailing list
> > aqm@ietf.org
> > https://www.ietf.org/mailman/listinfo/aqm