Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-guidelines-11: (with DISCUSS and COMMENT)

>> -----Original Message-----
>> From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]
>> Sent: Monday, June 13, 2016 3:41 PM
> ...
>> Hi Al,
>>
>> I believe, we agree here. However, I’m not really sure what needs to be
>> changed/added in the draft now. The only concrete item I have is
>> replacing "application-level“ by "transport-layer payload“. Anything
>> else?
>>
>> Mirja
> [ACM]
> Thanks, that would resolve the biggest ambiguity for me.
> Like I said last week, I think we're done (with that change).
Thank you Al and Mirja.
I'll clear the DISCUSS on that basis, trusting the AD that the addition 
will be introduced.

Regards, Benoit
>
> Al
>
>>
>>> Am 10.06.2016 um 19:16 schrieb MORTON, ALFRED C (AL)
>> <acmorton@att.com>:
>>> more below, thanks for the clarifications, Mirja!
>>> Al
>>>
>>>> -----Original Message-----
>>>> From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]
>>>> Sent: Friday, June 10, 2016 12:55 PM
>>>> To: MORTON, ALFRED C (AL); Mirja Kühlewind; Benoit Claise
>>>> Cc: wes@mti-systems.com; aqm-chairs@ietf.org; The IESG; draft-ietf-
>> aqm-
>>>> eval-guidelines@ietf.org; Schulthess Nicolas (F&W); aqm@ietf.org
>>>> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
>>>> guidelines-11: (with DISCUSS and COMMENT)
>>>>
>>>> Hi Al,
>>>>
>>>> see below.
>>>>
>>>> On 10.06.2016 18:41, MORTON, ALFRED C (AL) wrote:
>>>>> Hi, see below,
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Mirja Kühlewind [mailto:ietf@kuehlewind.net]
>>>>>> Sent: Friday, June 10, 2016 9:15 AM
>>>>>> To: Benoit Claise; MORTON, ALFRED C (AL)
>>>>>> Cc: wes@mti-systems.com; aqm-chairs@ietf.org; The IESG; draft-ietf-
>>>> aqm-
>>>>>> eval-guidelines@ietf.org; Schulthess Nicolas (F&W); aqm@ietf.org
>>>>>> Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
>>>>>> guidelines-11: (with DISCUSS and COMMENT)
>>>>>>
>>>>>> Benoit,
>>>>>>
>>>>>> waiting for Al. But in the mean time see below.
>>>>>>
>>>>>> On 10.06.2016 11:57, Benoit Claise wrote:
>>>>>>> Al, assuming that someone would like to register this metric in a
>>>>>> registry
>>>>>>> (RFC6390), are they any grey areas in the performance metric
>>>>>> definitions in
>>>>>>> the draft?
>>>>>>>   From what I understand, a point such this one (from Al) is:
>>>>>>>
>>>>>>>      Because we are using Goodput, G, I take as given that there
>>>>>>>      must be a protocol with retransmission capability.
>>>>>>>      Otherwise, further simplification is possible (with dummy
>>>>>> traffic).
>>>>>>
>>>>>> Not really if you have not retransmission, simply your
>>>>>> goodout=throughput.
>>>>>> Don't see a problem here.
>>>>> [ACM]
>>>>> Although Goodput == Throughput for UDP, you can make a
>>>>> simpler measurement, you don't have to check for uniqueness.
>>>>
>>>> That's the view from someone measuring in the network. But if you do
>>>> simulations or have a controlled testbed, the easiest things is to
>>>> measure in
>>>> the application (and you automatically get the right thing). As we
>> don't
>>>> know
>>>> what exactly people do in the end, I think it is right to leave this
>>>> open
>>>> (and leave it as simple as possible in the description text).
>>> [ACM]
>>> Ok, but what layer of the application?  The raw media stream(s)?
>>> Or everything in the TCP/UDP payload?
>>>
>>> In lab benchmarking, it's sometimes about measuring at
>>> link speed x number of ports, so every operation makes a difference!
>>>
>>>>
>>>>>>>      But yes, Fs and G need to be reported on payload
>>>>>>>      at the same layer, so the protocol layer chosen is
>>>>>>>      an input parameter for this metric.
>>>>>> Yes, it need to be the same layer for all your tests; but the goal
>> is
>>>>>> not be
>>>>>> compatible with other tests. So it's your decision. It's guidance
>> how
>>>>>> you
>>>>>> would test AQMs to decide if you want to deploy them in the future
>>>> (or
>>>>>> to
>>>>>> show that your AQM has benefits compared to other AQMs such that
>>>> another
>>>>>> guy
>>>>>> might deploy this in future).
>>>>> [ACM]
>>>>>
>>>>> The current text mentions the "application layer" but needs to add
>> the
>>>> note
>>>>> that the layer chosen needs to be specified/included in with the
>>>> results, so that
>>>>> someone reading results later will know what was tested.
>>>> There actually is now a sentence saying:
>>>>
>>>> "Where flow size is the size of the application-level flow in bits
>> and
>>>> goodput is the application-level transfer time (described in
>>>> Section 2.5)."
>>>>
>>>> Is this sufficient?
>>> [ACM]
>>>
>>> I don't mean to prolong this, but I haven't been clear:
>>> The term "application-level" is ambiguous, it could be
>>> RTP, or some other container layer, or one of the MPEG layers,
>>> or the raw media/program stream (with our without meta data).
>>>
>>> If by saying "application-level", the transport-layer payload
>>> is meant, I suggest to say that.
>>>
>>> are we there yet? I know I am :-), it's 19:15 down the road in Geneva!
>>> Al
>>>
>>>> Mirja
>>>>
>>>>
>>>>> Al
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> aqm mailing list
>>>>> aqm@ietf.org
>>>>> https://www.ietf.org/mailman/listinfo/aqm
>>>>>
>>> _______________________________________________
>>> aqm mailing list
>>> aqm@ietf.org
>>> https://www.ietf.org/mailman/listinfo/aqm