Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

[Kuhn Nicolas <[e-mail]>]

Hi all,



Sorry for the delay in our answer - there have been couple of busy weeks.

Please find attached to this email the diff between v11 and expected v12.



For this specific issue, we propose then to :

- add one more sentence in the abstract to make the scope on lab testing during development and before deployment

- update the text on FCT as follows:

"The Flow Completion Time (FCT) is related to the flow size (Fs) and the goodput for the flow (G) as follows:



FCT [s] = Fs [Bit] /  G [Bit/s] ,



Where flow size is the size of the application-level flow in bits and goodput is the application-level transfer time (described in Section 2.5). "



We hope that this is fine.

We will push an updated version including the changes as soon as possible.



Kind regards,



The authors,


De : aqm [mailto:[e-mail]] De la part de Benoit Claise
Envoyé : vendredi 10 juin 2016 09:36
À : MORTON, ALFRED C (AL); Mirja Kuehlewind (IETF)
Cc : [e-mail]; [e-mail]; [e-mail]; [e-mail]; The IESG
Objet : Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

Thanks Al,
This is exactly the type of feedback requested from the Performance Metric Directorate.

Mirja,
I buy into your argument:

I guess as soon as we have a registry, maybe there is someone interest in IPPM to catch up these metrics again and provide a RFC6390 definition but I would rather not like this document doing it.
As such, we probably don't need the RFC6390 template in this document. Fair enough.
However, what would be a pity is that if/when someone would like to register this metric, we end with questions such Al's one below, because the metric definitions in the published RFC are not precise. That could result in a RFCbis.

Therefore, going through the Performance Metric Directorate feedback now is important IMO.

Regards, Benoit


Because we are using Goodput, G, I take as given that there

must be a protocol with retransmission capability.

Otherwise, further simplification is possible (with dummy traffic).



But yes, Fs and G need to be reported on payload

at the same layer, so the protocol layer chosen is

an input parameter for this metric.



Al



-----Original Message-----

From: Mirja Kuehlewind (IETF) [mailto:[e-mail]]

Sent: Wednesday, June 08, 2016 7:21 AM

To: MORTON, ALFRED C (AL)

Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; The IESG; draft-ietf-aqm-

[e-mail]<mailto:[e-mail]>; Benoit Claise; [e-mail]<mailto:[e-mail]>

Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-

guidelines-11: (with DISCUSS and COMMENT)



Actually, it really doesn't matter that much in this case, I’d say. As

we are talking about a lab environment, you might use dummy traffic that

has some headers or not, that you might take into account of not, mostly

depending on which information can be more easily accessed. What is

important is that you do the same thing for all schemes that you

compare.



I guess one could add a note that there are different ways to measure

this and that it is important to measure G at the same layer. Does that

make sense?



Mirja





Am 08.06.2016 um 13:03 schrieb MORTON, ALFRED C (AL)

<[e-mail]><mailto:[e-mail]>:



Here's one area which could use more detail:



  ...The Flow Completion Time (FCT) is

  related to the flow size (Fs) and the goodput for the flow (G) as

  follows:



  FCT [s] = Fs [Byte] / ( G [Bit/s] / 8 [Bit/Byte] )



What protocol layers are included and excluded from Fs?



Also, G needs to be measured at the same layer, and the

definition in RFC 2647 is a bit vague about layers, too.

It would be good to clarify which bytes to count here.



Al



-----Original Message-----

From: Mirja Kuehlewind (IETF) [mailto:[e-mail]]

Sent: Wednesday, June 08, 2016 5:40 AM

To: MORTON, ALFRED C (AL)

Cc: Benoit Claise; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>;

[e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; The IESG

Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-

guidelines-11: (with DISCUSS and COMMENT)



Hi Al,



what kind of detail are you looking for? Because I thought with the

given equation this one was pretty clear.



Do you have a reference to the benchmarking work?



Mirja





Am 08.06.2016 um 11:18 schrieb MORTON, ALFRED C (AL)

<[e-mail]><mailto:[e-mail]>:



Hi Mirja,



That sounds fairly reasonable to me.

Would it be possible to ask the authors provide a bit more

detail on Flow Completion Time?



Flow Completion Time is close to a definition for a new metric,

and could benefit from more attention, perhaps a few more

details.

RFC6390 will provide some areas for improvement.



I imagine that related benchmarking efforts may wish to measure this

metric, and there would be independent implementations based on

the description provided here.



regards from Geneve'

Al



-----Original Message-----

From: Mirja Kuehlewind (IETF) [mailto:[e-mail]]

Sent: Wednesday, June 08, 2016 4:46 AM

To: MORTON, ALFRED C (AL); Benoit Claise

Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; The IESG; draft-ietf-aqm-

eval-

[e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>

Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-

guidelines-11: (with DISCUSS and COMMENT)



Hi Benoit,



I finally had another look at the document as well as at RFC6390. I

guess the metrics in question are Flow completion time (sec 2.1.),

Flow

start up time (2.2.) and Packet loss synchronization (2.4.). And

you

are

right that these metric could be see as Application-Specific

Performance

Metric as defined in RFC6390. However, I agree with Al that given

the

scope of this document is providing

"a generic list of scenarios against which an

 AQM proposal should be evaluated, considering both potential

 performance gain and safety of deployment.“,

I don’t think these metric need to be defined and registered this

way.



I guess as soon as we have a registry, maybe there is someone

interest

in IPPM to catch up these metrics again and provide a RFC6390

definition

but I would rather not like this document doing it.



Is that acceptable for you?



We could add one more sentence in the abstract to make the scope on

lab

testing during development and before deployment clear. Would that

help?



Mirja







Am 25.05.2016 um 14:22 schrieb Mirja Kuehlewind (IETF)

<[e-mail]><mailto:[e-mail]>:



Hi Al, Benoit, hi all,



thanks for the feedback. Sorry for me delaying this maybe a little

but

I need t have another look at the document which will be next week

at

this point. In general I agree that this does not need to only rely

on

registered metrics because is mostly for lab tests; further this

might

probably not the right doc to register new metrics. However, I

would

still like to have another look at the doc and see if we can

improve

anything or figure out if any of the ’new’/non-registed metrics

should/could be taken up by e.g. ippm.



Mirja







Am 20.05.2016 um 14:53 schrieb MORTON, ALFRED C (AL)

<[e-mail]><mailto:[e-mail]>:



All,

a few replies in-line below,

Al



-----Original Message-----

From: Benoit Claise [mailto:[e-mail]]

Sent: Thursday, May 19, 2016 5:38 AM

To: The IESG

Cc: [e-mail]<mailto:[e-mail]>; wes@mti-

systems.com;

aqm-

[e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; linda

Dunbar;

MORTON, ALFRED C (AL)

Subject: Benoit Claise's Discuss on draft-ietf-aqm-eval-

guidelines-

11:

(with DISCUSS and COMMENT)



Benoit Claise has entered the following ballot position for

draft-ietf-aqm-eval-guidelines-11: Discuss



...

----------------------------------------------------------------

--

--

--

DISCUSS:

----------------------------------------------------------------

--

--

--



Has a RFC6390 performance directorate review done for the 2.X

metrics?

It

should.

[ACM]

I reviewed this draft about 18 months ago.

Mostly, it points to existing RFCs for fundamental metrics,

and discusses others.  I read this:

...This document provides characterization guidelines that

can be used to assess the deployability of an AQM, whether it is

candidate for standardization at IETF or not.

as restricted to lab testing.



See http://www.ietf.org/iesg/directorate/performance-

metrics.html

I guess that the metrics will be recorded in the future (See

draft-ietf-ippm-metric-registry-06

), right?

[ACM]

That's up to the authors, they might simply point to

metrics in the registry contributed by others

(when following these guidelines at a future time).



For example, Flow Completion Time and Packet Loss

Synchronization

are

new, I believe.

[ACM]

Flow Completion Time is close to a definition for a new metric,

and could benefit from more attention, perhaps a few more

details.

RFC6390 will provide some areas for improvement.



Packet loss sync full methodology is described in [JAY006],

according to the text.



And some other metrics are already documented in RFC6390

compliant

documents. Pointers should be provided.

[ACM]

Most others are discussion sections and provide references.



See

https://tools.ietf.org/html/draft-ietf-xrblock-independent-

burst-

gap-

discard-01#appendix-A

for an example





----------------------------------------------------------------

--

--

--

COMMENT:

----------------------------------------------------------------

--

--

--



- Random Early Detection (RED), BLUE, and Proportional Integral

controller (PI)

Would you have references?



- BDP is mentioned a few times. Please expand.



- Glossary section = terminology section, right? If we want to

be

consistent across documents



- section 12.2. Why not a MUST below?

In order to understand an AQM's deployment considerations and

performance under a specific environment, AQM proposals SHOULD

describe the parameters that control the macroscopic AQM

behavior,

and identify any parameters that require tuning to operational

conditions.







_______________________________________________

aqm mailing list

[e-mail]<mailto:[e-mail]>

https://www.ietf.org/mailman/listinfo/aqm