Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

more below, thanks for the clarifications, Mirja!
Al

> -----Original Message-----
> From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]
> Sent: Friday, June 10, 2016 12:55 PM
> To: MORTON, ALFRED C (AL); Mirja Kühlewind; Benoit Claise
> Cc: wes@mti-systems.com; aqm-chairs@ietf.org; The IESG; draft-ietf-aqm-
> eval-guidelines@ietf.org; Schulthess Nicolas (F&W); aqm@ietf.org
> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
> guidelines-11: (with DISCUSS and COMMENT)
> 
> Hi Al,
> 
> see below.
> 
> On 10.06.2016 18:41, MORTON, ALFRED C (AL) wrote:
> > Hi, see below,
> >
> >> -----Original Message-----
> >> From: Mirja Kühlewind [mailto:ietf@kuehlewind.net]
> >> Sent: Friday, June 10, 2016 9:15 AM
> >> To: Benoit Claise; MORTON, ALFRED C (AL)
> >> Cc: wes@mti-systems.com; aqm-chairs@ietf.org; The IESG; draft-ietf-
> aqm-
> >> eval-guidelines@ietf.org; Schulthess Nicolas (F&W); aqm@ietf.org
> >> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
> >> guidelines-11: (with DISCUSS and COMMENT)
> >>
> >> Benoit,
> >>
> >> waiting for Al. But in the mean time see below.
> >>
> >> On 10.06.2016 11:57, Benoit Claise wrote:
> >>> Al, assuming that someone would like to register this metric in a
> >> registry
> >>> (RFC6390), are they any grey areas in the performance metric
> >> definitions in
> >>> the draft?
> >>>   From what I understand, a point such this one (from Al) is:
> >>>
> >>>      Because we are using Goodput, G, I take as given that there
> >>>      must be a protocol with retransmission capability.
> >>>      Otherwise, further simplification is possible (with dummy
> >> traffic).
> >>
> >> Not really if you have not retransmission, simply your
> >> goodout=throughput.
> >> Don't see a problem here.
> > [ACM]
> > Although Goodput == Throughput for UDP, you can make a
> > simpler measurement, you don't have to check for uniqueness.
> 
> 
> That's the view from someone measuring in the network. But if you do
> simulations or have a controlled testbed, the easiest things is to
> measure in
> the application (and you automatically get the right thing). As we don't
> know
> what exactly people do in the end, I think it is right to leave this
> open
> (and leave it as simple as possible in the description text).
[ACM] 
Ok, but what layer of the application?  The raw media stream(s)?
Or everything in the TCP/UDP payload?

In lab benchmarking, it's sometimes about measuring at 
link speed x number of ports, so every operation makes a difference!

> 
> 
> >
> >>
> >>>
> >>>      But yes, Fs and G need to be reported on payload
> >>>      at the same layer, so the protocol layer chosen is
> >>>      an input parameter for this metric.
> >>
> >> Yes, it need to be the same layer for all your tests; but the goal is
> >> not be
> >> compatible with other tests. So it's your decision. It's guidance how
> >> you
> >> would test AQMs to decide if you want to deploy them in the future
> (or
> >> to
> >> show that your AQM has benefits compared to other AQMs such that
> another
> >> guy
> >> might deploy this in future).
> > [ACM]
> >
> > The current text mentions the "application layer" but needs to add the
> note
> > that the layer chosen needs to be specified/included in with the
> results, so that
> > someone reading results later will know what was tested.
> 
> There actually is now a sentence saying:
> 
> "Where flow size is the size of the application-level flow in bits and
> goodput is the application-level transfer time (described in
> Section 2.5)."
> 
> Is this sufficient?
[ACM] 

I don't mean to prolong this, but I haven't been clear:
The term "application-level" is ambiguous, it could be
RTP, or some other container layer, or one of the MPEG layers,
or the raw media/program stream (with our without meta data).

If by saying "application-level", the transport-layer payload 
is meant, I suggest to say that.

are we there yet? I know I am :-), it's 19:15 down the road in Geneva!
Al

> 
> Mirja
> 
> 
> >
> > Al
> >
> >
> > _______________________________________________
> > aqm mailing list
> > aqm@ietf.org
> > https://www.ietf.org/mailman/listinfo/aqm
> >