[ippm] Re: Gen-ART Last Call review of draft-ietf-ippm-qoo-06

[Ike Kunze <[e-mail]>]

Hi Paul,

Thank you very much for taking the time to review the draft!

You are right in that the measurements / reported percentiles of different network segments need to be the same to enable composability. Ensuring this alignment across different networks / operators might be challenging, especially considering that the draft does not impose standardized percentiles for the network performance requirements of applications.
The draft does follow BITAG recommendations regarding measurement percentiles, though, and I think that once we have a better understanding which measurement percentiles have the most meaning for applications (with application “profiles" then also using those percentiles), alignment across operators could very well just follow naturally.
Even if it does not follow, there is still a lot of value to be gained by just using the methodology on different network segments within the same network.

I believe we already have some limited (implicit) text along this line in the document. If you see value in clarifying possible deployment / adoption challenges beyond what we currently have, we could add some additional, explicit discussion focusing on this aspect in the operational consideration section.
What do you think?

Regarding the presentation of the formulas, you have probably also seen in the thread with Martin that we plan to (at the very least) implement his suggestion that you also refer to in your comment. Additionally, Martin has kindly provided me additional pointers for alternative forms of formula rendering and we plan to also explore these options to come up with a version that is as clear and easy to understand as possible.
I would very much appreciate your feedback on the final version once we have it.

Thank you again!

Cheers,
Ike


Friendly Note: I send emails at times that work best for me. Please don’t feel any pressure to reply outside your usual working hours.


[Background pattern  Description automatically generated]

Ike Kunze

Senior Researcher

Phone [phone]

CUJO AI<https://cujo.com/> | News <https://cujo.com/blog/> | LinkedIn<https://www.linkedin.com/company/cujoai>


From: Paul Kyzivat <[e-mail]>
Date: Monday, 9 February 2026 at 16:38
To: [e-mail] <[e-mail]>
Cc: General Area Review Team <[e-mail]>, [e-mail] <[e-mail]>, [e-mail] <[e-mail]>
Subject: Gen-ART Last Call review of draft-ietf-ippm-qoo-06

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.


I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.ietf.org%2Fen%2Fgroup%2Fgen%2FGenArtFAQ&data=05%7C02%7Cike.kunze%40cujo.com%7Cc05c24a2dabe489aaf4f08de67f1347e%7Ca5d2f1ce8ca649ce893ab4f922d7fc6b%7C0%7C0%7C639062482918728250%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=UPw6Ih9qEWczjMlQslXsnJu%2BtkPYy77aFYjavK%2FfSrs%3D&reserved=0<https://wiki.ietf.org/en/group/gen/GenArtFAQ>>.

Document: draft-ietf-ippm-qoo
Title: Quality of Outcome (QoO)
Reviewer: Paul Kyzivat
Review Date: 2026-02-09
IETF LC End Date: 2026-02-13
IESG Telechat date: ?

Summary: This draft is on the right track but has open issues, described
in the review.

Note: This reviewer has no experience in, or knowledge of, network
performance measurement. Hence I'm unqualified to comment on the
semantics of this draft. In such cases I usually fall back to reviewing
the form and syntax. Even that is hard here, since there is little in
the way of formal specification.

Issue:

My impression is that for QoO values to be composable, they must be
based on the same measurement percentiles. Achieving that across the
managers of connected networks seems difficult. Is it realistic?

Issue:

I had difficulty understanding the notation used in the formula in
section 7:

   QoO_latency = min_{i}(min(
      max((1 - ((ML_i - ROP_i) / (CPUP_i - ROP_i))) * 100, 0), 100))

Fortunately, Martin Thompson has posted an alternative rendition for
this formula, that I find much more understandable:

    for i in ROP:
      m = (ML[i] - ROP[i]) / (CPUP[i] - ROP[i])
      metrics[i] = clamp(0, m, 1)
    QoO_latency = find_min(metrics) * 100

(It is in his followup message to his ArtArt review of this document.
<https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailarchive.ietf.org%2Farch%2Fmsg%2Fart%2FBC-qxC6pD_63lfHfLd1R4n_KA8c%2F&data=05%7C02%7Cike.kunze%40cujo.com%7Cc05c24a2dabe489aaf4f08de67f1347e%7Ca5d2f1ce8ca649ce893ab4f922d7fc6b%7C0%7C0%7C639062482918760940%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=RWF9tZEOYG5pbms6GqROHFbOI%2BO7L9dQlRIvxCltmrY%3D&reserved=0<https://mailarchive.ietf.org/arch/msg/art/BC-qxC6pD_63lfHfLd1R4n_KA8c/>>)

Martin considers this a nit, but I consider it to be a significant
issue. It is the closest thing this document has to a normative
requirement, so it is essential that all readers understand it
consistently. I recommend reworking the rendition of this formula into a
form that will be clear to all who need to understand it.

I found Martin's full ArtArt review insightful. While it goes beyond my
understanding of the subject material, I agree with it to the extent of
my understanding.

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.