Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

Hi Al,

I believe, we agree here. However, I’m not really sure what needs to be changed/added in the draft now. The only concrete item I have is replacing "application-level“ by "transport-layer payload“. Anything else?

Mirja

> 
> Am 10.06.2016 um 19:16 schrieb MORTON, ALFRED C (AL) <acmorton@att.com>:
> 
> more below, thanks for the clarifications, Mirja!
> Al
> 
>> -----Original Message-----
>> From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]
>> Sent: Friday, June 10, 2016 12:55 PM
>> To: MORTON, ALFRED C (AL); Mirja Kühlewind; Benoit Claise
>> Cc: wes@mti-systems.com; aqm-chairs@ietf.org; The IESG; draft-ietf-aqm-
>> eval-guidelines@ietf.org; Schulthess Nicolas (F&W); aqm@ietf.org
>> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
>> guidelines-11: (with DISCUSS and COMMENT)
>> 
>> Hi Al,
>> 
>> see below.
>> 
>> On 10.06.2016 18:41, MORTON, ALFRED C (AL) wrote:
>>> Hi, see below,
>>> 
>>>> -----Original Message-----
>>>> From: Mirja Kühlewind [mailto:ietf@kuehlewind.net]
>>>> Sent: Friday, June 10, 2016 9:15 AM
>>>> To: Benoit Claise; MORTON, ALFRED C (AL)
>>>> Cc: wes@mti-systems.com; aqm-chairs@ietf.org; The IESG; draft-ietf-
>> aqm-
>>>> eval-guidelines@ietf.org; Schulthess Nicolas (F&W); aqm@ietf.org
>>>> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
>>>> guidelines-11: (with DISCUSS and COMMENT)
>>>> 
>>>> Benoit,
>>>> 
>>>> waiting for Al. But in the mean time see below.
>>>> 
>>>> On 10.06.2016 11:57, Benoit Claise wrote:
>>>>> Al, assuming that someone would like to register this metric in a
>>>> registry
>>>>> (RFC6390), are they any grey areas in the performance metric
>>>> definitions in
>>>>> the draft?
>>>>>  From what I understand, a point such this one (from Al) is:
>>>>> 
>>>>>     Because we are using Goodput, G, I take as given that there
>>>>>     must be a protocol with retransmission capability.
>>>>>     Otherwise, further simplification is possible (with dummy
>>>> traffic).
>>>> 
>>>> Not really if you have not retransmission, simply your
>>>> goodout=throughput.
>>>> Don't see a problem here.
>>> [ACM]
>>> Although Goodput == Throughput for UDP, you can make a
>>> simpler measurement, you don't have to check for uniqueness.
>> 
>> 
>> That's the view from someone measuring in the network. But if you do
>> simulations or have a controlled testbed, the easiest things is to
>> measure in
>> the application (and you automatically get the right thing). As we don't
>> know
>> what exactly people do in the end, I think it is right to leave this
>> open
>> (and leave it as simple as possible in the description text).
> [ACM] 
> Ok, but what layer of the application?  The raw media stream(s)?
> Or everything in the TCP/UDP payload?
> 
> In lab benchmarking, it's sometimes about measuring at 
> link speed x number of ports, so every operation makes a difference!
> 
>> 
>> 
>>> 
>>>> 
>>>>> 
>>>>>     But yes, Fs and G need to be reported on payload
>>>>>     at the same layer, so the protocol layer chosen is
>>>>>     an input parameter for this metric.
>>>> 
>>>> Yes, it need to be the same layer for all your tests; but the goal is
>>>> not be
>>>> compatible with other tests. So it's your decision. It's guidance how
>>>> you
>>>> would test AQMs to decide if you want to deploy them in the future
>> (or
>>>> to
>>>> show that your AQM has benefits compared to other AQMs such that
>> another
>>>> guy
>>>> might deploy this in future).
>>> [ACM]
>>> 
>>> The current text mentions the "application layer" but needs to add the
>> note
>>> that the layer chosen needs to be specified/included in with the
>> results, so that
>>> someone reading results later will know what was tested.
>> 
>> There actually is now a sentence saying:
>> 
>> "Where flow size is the size of the application-level flow in bits and
>> goodput is the application-level transfer time (described in
>> Section 2.5)."
>> 
>> Is this sufficient?
> [ACM] 
> 
> I don't mean to prolong this, but I haven't been clear:
> The term "application-level" is ambiguous, it could be
> RTP, or some other container layer, or one of the MPEG layers,
> or the raw media/program stream (with our without meta data).
> 
> If by saying "application-level", the transport-layer payload 
> is meant, I suggest to say that.
> 
> are we there yet? I know I am :-), it's 19:15 down the road in Geneva!
> Al
> 
>> 
>> Mirja
>> 
>> 
>>> 
>>> Al
>>> 
>>> 
>>> _______________________________________________
>>> aqm mailing list
>>> aqm@ietf.org
>>> https://www.ietf.org/mailman/listinfo/aqm
>>> 
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm