[ippm] Re: AD Review of draft-ietf-ippm-qoo

[Ike Kunze <[e-mail]>]

Hi Rüdiger,

Thank you very much for your feedback!

Regarding your main high-level comment that you

[woul]d appreciate the document to be written in a way to reflect that (i.e., it should be clear, that accuracy and precision only refer to the QoS metric QoO, but not to app QoE).

We have followed your suggestion and positioned QoO more explicitly on the QoS <-> QoE scale.
For example, we now state that the QoO framework itself does not have a fixed position on this scale but that it also depends on how exactly the thresholds are selected.
Also, we emphasize that QoO does not necessarily correlate to QoE and that more in-depth studies would be necessary to make such a claim.

For addressing this high-level comment, we have also implemented many of your suggestions; thank you for providing those!
Below, you can find how we have addressed your additional comments (our comments are marked with [IK]).

Should you see the need for additional changes, please let us know.

Thank you again!

Cheers,
Ike



Friendly Note: I send emails at times that work best for me. Please don’t feel any pressure to reply outside your usual working hours.


[Background pattern  Description automatically generated]

Ike Kunze

Senior Researcher

Phone [phone]

CUJO AI<https://cujo.com/> | News <https://cujo.com/blog/> | LinkedIn<https://www.linkedin.com/company/cujoai>


From: [e-mail] <[e-mail]>
Date: Monday, 17 November 2025 at 16:13
To: [e-mail] <[e-mail]>
Cc: [e-mail] <[e-mail]>, [e-mail] <[e-mail]>
Subject: [ippm] Re: AD Review of draft-ietf-ippm-qoo

[Cropped for conciseness; we describe how we address your high-level comments in the main body of this mail]

Some examples and proposals:

Scope
By leveraging the Quality Attenuation metric, QoO provides a unified method for defining and evaluating application-specific network requirements while ensuring actionable insights for network optimization and simple quality scores for end-users.

[RG] I’d be happier with
….., QoO provides a method for defining and evaluating application-specific network requirements while providing insights for network optimization and simple Quality of Service scores for end-users.

And so on…

[IK] We have implemented the essence of your suggestion with some additional changes based on the other received feedback, resulting in the following version:

QoO provides a method for defining and evaluating application-specific, quality-focused network performance requirements to enable insights for network optimization and simple Quality of Service scores for end-users.


2. Overview

  *
Maybe add a bullet point clarifying
QoO scores are expected to correlate with Quality of Experience metrics, like MOS scores, but they are not designed to deliver MOS or QoE scores.

[IK] Added this suggestion nearly 1:1.

3. Terminology

Text as is
Quality of Outcome (QoO): A network quality framework and metric designed to assess network performance based on the probability that applications will achieve successful outcomes.  QoO provides a unified approach that serves the needs of end-users, application developers, and network operators.

[RG]: I’m unhappy with the “will” in the statement. Please change to … probability that applications will likely achieve successful outcomes.

[IK] Based on other feedback, we have rephrased all statements pertaining to the “probability” of applications having good performance / successful outcome / etc.
In this particular context, the new version is:

A network quality framework and metric that evaluates network quality based on how closely measured network conditions meet application-specific, quality-focused performance requirements.

################

1. Introduction
4.2. Requirements
6.2. Describing Network Requirements

Example (here 4.2):
Flexibility in describing application requirements and the ability to capture the delay characteristics of the network in sufficient detail are necessary to compute application success with satisfactory accuracy and precision.

[RG] I prefer clarity, to which of the two QoO or QoE accuracy and precision refers throughout the doc, please reword these statements as in this case:
Flexibility in describing application requirements and the ability to capture an accurate and precise network service QoO score can be used to better estimate application success.

[IK] Again, based on other feedback, we have rephrased statements regarding application “success” but have generally followed your suggestion. New version is:

Flexibility in describing application requirements and the ability to capture the delay and loss characteristics of a network with sufficient accuracy and precision are necessary to compute a meaningful network quality QoO score that can be used to better estimate application performance.

[IK] Likewise, we have phrased more clearly what accuracy and precision refer to throughout the document.


##########

4.2.1. Requirements for end-users

Please provide a reference for all three requirements (you’ve added one):

Remote music collaboration: <20ms one-way latency [JamKazam<https://www.ietf.org/archive/id/draft-ietf-ippm-qoo-04.html#JamKazam>]
Online gaming: 6Mb/s downlink throughput and 30ms RTT to join a multiplayer game (based on requirements for popular online games)
Virtual reality: <20ms RTT from head motion to rendered update in VR (based on human perception studies for VR applications)

##########

10.3. Assuming Linear Relationship between Perfect and Unusable (and that it is not really a probability)

Please add as a first statement:
It has been shown, that, e.g., Interactivity can’t be modeled by a linear scale [ref G.1051], and thus the linear modeling proposed here adds an error in estimating QoE of interactive applications.

ITU-T G.1051, Latency measurement and interactivity scoring under real application data traffic patterns (03/2023), page 9.
https://www.itu.int/itu-t/recommendations/index.aspx?ser=G
Quote: “However, this dependency is not a simple linear function; rather there are saturation areas at both tails of the function, where no further change in perception happens even if the latency changes.”

The existing text may be maintained (no comment additional on it from my side).

[IK] We mainly followed your suggestion, again, with some modifications to accommodate other feedback.
The new version is

It has been shown that, for example, interactivity cannot be modeled by a linear scale [G.1051].  Thus, the linear modeling proposed here adds an error in estimating the perceived performance of interactive applications.

##########

9.2. Insights from user testing

Please clarify, how this test-setup and result evaluation relates to ITU-T standards P.800 and P.900 series, or  P.1401, https://www.itu.int/itu-t/recommendations/index.aspx?ser=P . The documents specify conditions and methods to plan, perform and evaluate subjective QoE measurement campaigns.
You can put subjective QoE testing as specified ITU-T P-series out of scope, so that a reader understands

While subjective QoE testing as specified ITU-T P-series is out of scope of this framework, a study involving 25…..Study] allowed for some insights.

[IK] Again, we followed your suggestion with small modifications:

While subjective QoE testing as specified in the ITU-T P-series recommendations ([P.800], [P.910], [P.1401]) is out of scope of this document, a study involving 25 participants tested the QoO framework in real-world settings [QoOUserStudy].



Current text:
These results provide supporting evidence for QoO's value as a user-focused tool, bridging technical metrics with real-world application performance to enhance end-user satisfaction.

RG:
These results indicate that users find it easier to correlate QoO scores with real-world application performance, then, e.g., a speed-test. QoO is expected to help bridging technical metrics with application performance.

[IK] Again, we followed your suggestion with small modifications:

These results indicate that users find it easier to correlate QoO scores with real-world application performance than, for example, a speed test.  As such, QoO is expected to help bridge technical metrics with application performance.


This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.