Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

["MORTON, ALFRED C (AL)" <[e-mail]>]

Because we are using Goodput, G, I take as given that there
must be a protocol with retransmission capability.
Otherwise, further simplification is possible (with dummy traffic).

But yes, Fs and G need to be reported on payload
at the same layer, so the protocol layer chosen is
an input parameter for this metric.

Al

> -----Original Message-----
> From: Mirja Kuehlewind (IETF) [mailto:[e-mail]]
> Sent: Wednesday, June 08, 2016 7:21 AM
> To: MORTON, ALFRED C (AL)
> Cc: [e-mail]; [e-mail]; The IESG; draft-ietf-aqm-
> [e-mail]; Benoit Claise; [e-mail]
> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
> guidelines-11: (with DISCUSS and COMMENT)
> 
> Actually, it really doesn't matter that much in this case, I’d say. As
> we are talking about a lab environment, you might use dummy traffic that
> has some headers or not, that you might take into account of not, mostly
> depending on which information can be more easily accessed. What is
> important is that you do the same thing for all schemes that you
> compare.
> 
> I guess one could add a note that there are different ways to measure
> this and that it is important to measure G at the same layer. Does that
> make sense?
> 
> Mirja
> 
> 
> > Am 08.06.2016 um 13:03 schrieb MORTON, ALFRED C (AL)
> <[e-mail]>:
> >
> > Here's one area which could use more detail:
> >
> >   ...The Flow Completion Time (FCT) is
> >   related to the flow size (Fs) and the goodput for the flow (G) as
> >   follows:
> >
> >   FCT [s] = Fs [Byte] / ( G [Bit/s] / 8 [Bit/Byte] )
> >
> > What protocol layers are included and excluded from Fs?
> >
> > Also, G needs to be measured at the same layer, and the
> > definition in RFC 2647 is a bit vague about layers, too.
> > It would be good to clarify which bytes to count here.
> >
> > Al
> >
> >> -----Original Message-----
> >> From: Mirja Kuehlewind (IETF) [mailto:[e-mail]]
> >> Sent: Wednesday, June 08, 2016 5:40 AM
> >> To: MORTON, ALFRED C (AL)
> >> Cc: Benoit Claise; [e-mail]; [e-mail];
> >> [e-mail]; [e-mail]; The IESG
> >> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
> >> guidelines-11: (with DISCUSS and COMMENT)
> >>
> >> Hi Al,
> >>
> >> what kind of detail are you looking for? Because I thought with the
> >> given equation this one was pretty clear.
> >>
> >> Do you have a reference to the benchmarking work?
> >>
> >> Mirja
> >>
> >>
> >>> Am 08.06.2016 um 11:18 schrieb MORTON, ALFRED C (AL)
> >> <[e-mail]>:
> >>>
> >>> Hi Mirja,
> >>>
> >>> That sounds fairly reasonable to me.
> >>> Would it be possible to ask the authors provide a bit more
> >>> detail on Flow Completion Time?
> >>>
> >>>>>> Flow Completion Time is close to a definition for a new metric,
> >>>>>> and could benefit from more attention, perhaps a few more
> details.
> >>>>>> RFC6390 will provide some areas for improvement.
> >>>
> >>> I imagine that related benchmarking efforts may wish to measure this
> >>> metric, and there would be independent implementations based on
> >>> the description provided here.
> >>>
> >>> regards from Geneve'
> >>> Al
> >>>
> >>>> -----Original Message-----
> >>>> From: Mirja Kuehlewind (IETF) [mailto:[e-mail]]
> >>>> Sent: Wednesday, June 08, 2016 4:46 AM
> >>>> To: MORTON, ALFRED C (AL); Benoit Claise
> >>>> Cc: [e-mail]; [e-mail]; The IESG; draft-ietf-aqm-
> eval-
> >>>> [e-mail]; [e-mail]
> >>>> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
> >>>> guidelines-11: (with DISCUSS and COMMENT)
> >>>>
> >>>> Hi Benoit,
> >>>>
> >>>> I finally had another look at the document as well as at RFC6390. I
> >>>> guess the metrics in question are Flow completion time (sec 2.1.),
> >> Flow
> >>>> start up time (2.2.) and Packet loss synchronization (2.4.). And
> you
> >> are
> >>>> right that these metric could be see as Application-Specific
> >> Performance
> >>>> Metric as defined in RFC6390. However, I agree with Al that given
> the
> >>>> scope of this document is providing
> >>>> "a generic list of scenarios against which an
> >>>>  AQM proposal should be evaluated, considering both potential
> >>>>  performance gain and safety of deployment.“,
> >>>> I don’t think these metric need to be defined and registered this
> >> way.
> >>>>
> >>>> I guess as soon as we have a registry, maybe there is someone
> >> interest
> >>>> in IPPM to catch up these metrics again and provide a RFC6390
> >> definition
> >>>> but I would rather not like this document doing it.
> >>>>
> >>>> Is that acceptable for you?
> >>>>
> >>>> We could add one more sentence in the abstract to make the scope on
> >> lab
> >>>> testing during development and before deployment clear. Would that
> >> help?
> >>>>
> >>>> Mirja
> >>>>
> >>>>
> >>>>
> >>>>> Am 25.05.2016 um 14:22 schrieb Mirja Kuehlewind (IETF)
> >>>> <[e-mail]>:
> >>>>>
> >>>>> Hi Al, Benoit, hi all,
> >>>>>
> >>>>> thanks for the feedback. Sorry for me delaying this maybe a little
> >> but
> >>>> I need t have another look at the document which will be next week
> at
> >>>> this point. In general I agree that this does not need to only rely
> >> on
> >>>> registered metrics because is mostly for lab tests; further this
> >> might
> >>>> probably not the right doc to register new metrics. However, I
> would
> >>>> still like to have another look at the doc and see if we can
> improve
> >>>> anything or figure out if any of the ’new’/non-registed metrics
> >>>> should/could be taken up by e.g. ippm.
> >>>>>
> >>>>> Mirja
> >>>>>
> >>>>>
> >>>>>
> >>>>>> Am 20.05.2016 um 14:53 schrieb MORTON, ALFRED C (AL)
> >>>> <[e-mail]>:
> >>>>>>
> >>>>>> All,
> >>>>>> a few replies in-line below,
> >>>>>> Al
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Benoit Claise [mailto:[e-mail]]
> >>>>>>> Sent: Thursday, May 19, 2016 5:38 AM
> >>>>>>> To: The IESG
> >>>>>>> Cc: [e-mail]; wes@mti-
> systems.com;
> >>>> aqm-
> >>>>>>> [e-mail]; [e-mail]; [e-mail]; linda
> Dunbar;
> >>>>>>> MORTON, ALFRED C (AL)
> >>>>>>> Subject: Benoit Claise's Discuss on draft-ietf-aqm-eval-
> >> guidelines-
> >>>> 11:
> >>>>>>> (with DISCUSS and COMMENT)
> >>>>>>>
> >>>>>>> Benoit Claise has entered the following ballot position for
> >>>>>>> draft-ietf-aqm-eval-guidelines-11: Discuss
> >>>>>>>
> >>>>>> ...
> >>>>>>> ----------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>>> DISCUSS:
> >>>>>>> ----------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>>>
> >>>>>>> Has a RFC6390 performance directorate review done for the 2.X
> >>>> metrics?
> >>>>>>> It
> >>>>>>> should.
> >>>>>> [ACM]
> >>>>>> I reviewed this draft about 18 months ago.
> >>>>>> Mostly, it points to existing RFCs for fundamental metrics,
> >>>>>> and discusses others.  I read this:
> >>>>>> ...This document provides characterization guidelines that
> >>>>>> can be used to assess the deployability of an AQM, whether it is
> >>>>>> candidate for standardization at IETF or not.
> >>>>>> as restricted to lab testing.
> >>>>>>
> >>>>>>> See http://www.ietf.org/iesg/directorate/performance-
> metrics.html
> >>>>>>> I guess that the metrics will be recorded in the future (See
> >>>>>>> draft-ietf-ippm-metric-registry-06
> >>>>>>> ), right?
> >>>>>> [ACM]
> >>>>>> That's up to the authors, they might simply point to
> >>>>>> metrics in the registry contributed by others
> >>>>>> (when following these guidelines at a future time).
> >>>>>>
> >>>>>>> For example, Flow Completion Time and Packet Loss
> Synchronization
> >>>> are
> >>>>>>> new, I believe.
> >>>>>> [ACM]
> >>>>>> Flow Completion Time is close to a definition for a new metric,
> >>>>>> and could benefit from more attention, perhaps a few more
> details.
> >>>>>> RFC6390 will provide some areas for improvement.
> >>>>>>
> >>>>>> Packet loss sync full methodology is described in [JAY006],
> >>>>>> according to the text.
> >>>>>>
> >>>>>>> And some other metrics are already documented in RFC6390
> compliant
> >>>>>>> documents. Pointers should be provided.
> >>>>>> [ACM]
> >>>>>> Most others are discussion sections and provide references.
> >>>>>>
> >>>>>>> See
> >>>>>>> https://tools.ietf.org/html/draft-ietf-xrblock-independent-
> burst-
> >>>> gap-
> >>>>>>> discard-01#appendix-A
> >>>>>>> for an example
> >>>>>>>
> >>>>>>>
> >>>>>>> ----------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>>> COMMENT:
> >>>>>>> ----------------------------------------------------------------
> --
> >> --
> >>>> --
> >>>>>>>
> >>>>>>> - Random Early Detection (RED), BLUE, and Proportional Integral
> >>>>>>> controller (PI)
> >>>>>>> Would you have references?
> >>>>>>>
> >>>>>>> - BDP is mentioned a few times. Please expand.
> >>>>>>>
> >>>>>>> - Glossary section = terminology section, right? If we want to
> be
> >>>>>>> consistent across documents
> >>>>>>>
> >>>>>>> - section 12.2. Why not a MUST below?
> >>>>>>> In order to understand an AQM's deployment considerations and
> >>>>>>> performance under a specific environment, AQM proposals SHOULD
> >>>>>>> describe the parameters that control the macroscopic AQM
> behavior,
> >>>>>>> and identify any parameters that require tuning to operational
> >>>>>>> conditions.
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> aqm mailing list
> >>>>> [e-mail]
> >>>>> https://www.ietf.org/mailman/listinfo/aqm
> >>>
> >