Re: [ippm] Call for adoption of draft-olden-ippm-qoo

[Bjørn Ivar Teigen <[e-mail]>]

Hi Greg, IPPM

Thank you for your detailed review of our draft - much appreciated!
Your comments are well received, and I agree that these can be resolved in
the course of progressing the draft.

Regarding draft-teigen-ippm-app-quality-metric-reqs
<https://datatracker.ietf.org/doc/draft-olden-ippm-qoo/>: The documents are
indeed related, and in my opinion, it makes sense to work on them in
parallel. Perhaps I should have requested to present both documents in San
Francisco, but I was wary of taking up too much of the group's time. I
would be happy to present app-quality-metrics in Prague so we can discuss
eventually bringing both documents into the working group.

Regards,
Bjørn

On Mon, 11 Sept 2023 at 22:23, Greg Mirsky <[e-mail]> wrote:

> Hi Marcus,
> thank you for your kind consideration.
>
> Dear Authors,
> thank you for your dedication to this work. I found this document
> well-written and I support its adoption by the IPPM WG.
> Please find my notes and questions below. I believe that we can discuss
> and resolve them in the course of progressing the draft:
>
>    - In Abstract, as I understand it, you refer to the draft Requirements
>    for a Network Quality Framework Useful for Applications, Users, and
>    Operators
>    <https://datatracker.ietf.org/doc/draft-teigen-ippm-app-quality-metric-reqs/>.
>    Since HTML links are not used in Abstract, perhaps adding a clarification
>    like "draft" prior to the title of the draft is acceptable.
>    - Related to the above. What is your view of the state of
>    draft-teigen-ippm-app-quality-metric-reqs
>    <https://datatracker.ietf.org/doc/draft-olden-ippm-qoo/>? Do you
>    consider it ready for WG adoption? It seems to me that the requirements and
>    the framework documents are closely related, and it could be beneficial to
>    develop them concurrently.
>    - "The challenge lies in specifying how to simplify enough without
>    losing too much in terms of precision and accuracy." I agree that there
>    must be a balance among efforts to produce simpler performance metrics, on
>    the one hand, and accurately reflect the state of the network treatment of
>    the data that is presented as a set of measurable service level indicators.
>    What I wonder, is the distinction you see between precision and accuracy.
>    Is it in the measuring and calculating particular service level objectives?
>    - A somewhat general note. You might consider changing statements that
>    explain the intent and scope from "we believe", "we propose" to less
>    personal language.
>    -  "We propose representing network quality as minimum required
>    throughput and set of latency and loss percentiles." Do you see this list
>    of performance metrics as exhaustive or rather an exemplary composition of
>    what can be used to define the network quality?
>    - "Active probing with TWAMP Light / STAMP / IRTT". References and
>    expanded use of acronyms could be helpful to a reader.
>    - "A list of 10 percentiles in a logarithmic-esque fashion has already
>    been suggested in industry [0th, 10th, 25th, 50th, 75th, 90th, 95th, 99th,
>    99.9th, 100th] and seems adequate." Has such a list been documented? A
>    reference could be very helpful here.
>    - "By not requiring a specific number of samples, this framework
>    allows taking 10 samples and calling it a distribution, which of course is
>    not ideal. " That is an interesting point. What could happen if the number
>    of samples is variable, perhaps in a range or from a recommended set of
>    values, and an operator chooses depending on the operational considerations?
>    - "... standardized in BBF TR-452 (aka QED) framework [TR-452.1]"
>    Perhaps "standardized in the QED framework [TR-452.1]" with QED included in
>    Terminology and Acronyms section, can be used here.
>    - Thank you for providing the detailed Implementation status! It is a
>    very impressive list.
>    - Although this document, as Informative, may not need RFC 2119
>    disclaimer explaining the use of capitalized verbs, please add Terminology
>    and Acronyms section listing all acronyms and abbreviations used in the
>    document.
>
> Regards,
> Greg
>
> On Mon, Sep 4, 2023 at 10:32 PM Marcus Ihlar <[e-mail]>
> wrote:
>
>> Hi Greg,
>>
>> Your input is very much appreciated, extending the last call with one
>> week is fine.
>>
>>
>>
>> The new deadline for the last call is now *Tuesday September 12*.
>>
>>
>>
>> BR
>>
>> Marcus
>>
>>
>>
>> *From:* Greg Mirsky <[e-mail]>
>> *Sent:* Monday, 4 September 2023 16:31
>> *To:* Marcus Ihlar <[e-mail]>
>> *Cc:* IETF IPPM WG ([e-mail]) <[e-mail]>
>> *Subject:* Re: [ippm] Call for adoption of draft-olden-ippm-qoo
>>
>>
>>
>> Hi Marcus,
>>
>> I'd like to provide my comments on this work (I wholeheartedly support
>> its adoption by the group), but I am in a tight spot with the BBF Q3
>> meeting. Would it be possible to announce an extension of the WG AP for a
>> week?
>>
>>
>>
>> Regards,
>>
>> Greg
>>
>>
>>
>> On Tue, Aug 22, 2023 at 7:31 PM Marcus Ihlar <marcus.ihlar=
>> [e-mail]> wrote:
>>
>> Hello IPPM,
>>
>> This email starts an adoption call in the IPPM working group for
>> draft-olden-ippm-qoo.
>>
>>
>>
>> The draft describes a new network quality framework named Quality of
>> Outcome (QoO) and can be found here:
>>
>> https://datatracker.ietf.org/doc/draft-olden-ippm-qoo/
>>
>>
>>
>> The current version is here:
>>
>> https://www.ietf.org/archive/id/draft-olden-ippm-qoo-01.html
>>
>>
>>
>> Please reply to this email by *Tuesday September **5*, with any review
>> comments and whether you support adoption of this document.
>>
>>
>>
>> BR
>>
>> Marcus & Tommy
>>
>>
>>
>> _______________________________________________
>> ippm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/ippm
>>
>> _______________________________________________
> ippm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/ippm
>


-- 
Bjørn Ivar Teigen, Ph.D.
Head of Research
[phone] | [e-mail] | www.domos.ai