[xrblock] Adam Roach's No Objection on draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)

[Adam Roach <[e-mail]>]

Adam Roach has entered the following ballot position for
draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcweb-rtcp-xr-metrics/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks to the authors and contributors on this document. I have two substantive
comments, and a handful of minor nits.

---------------------------------------------------------------------------

§5.3.1:

>  application can use these metrics to calculate the Mean Opinion Score
>  (MoS) values or Media Delivery Index (MDI) for their services.

"...calculate Estimated Mean Opinion Score (eMOS) values..."

(MOS scores necessarily require human subjective evaluation of audio under
carefully controlled circumstances -- any computer-generated metrics are
inherently estimations, and should not be called MOS scores.)

---------------------------------------------------------------------------

§5.3.1:

>  Error-resilience mechanisms, like RTP retransmission or FEC, are
>  optional in RTCWEB because the overhead of the repair bits adding to
>  the original streams.

This is a little misleading. draft-ietf-rtcweb-rtp-usage says:

   WebRTC endpoints MUST follow the recommendations for FEC use given in
   [I-D.ietf-rtcweb-fec].

And, in turn, draft-ietf-rtcweb-fec says:

   To support the functionality recommended above, implementations MUST
   be able to receive and make use of the relevant FEC formats for their
   supported audio codecs, and MUST indicate this support, as described
   in Section 4.  Use of these formats when sending, as mentioned above,
   is RECOMMENDED.

Rather than trying to reiterate this somewhat nuanced situation in *this*
document, I recommend removing the entire sentence and fixing the rest of the
paragraph. If you choose to keep it, please make sure it more clearly explains
the level of FEC support required of RTCWEB implementations.


===========================================================================

The remainder of my comments are grammatical or editorial nits that should be
fixed before progressing the document.

---------------------------------------------------------------------------

§3:

>  The RTCP Sender Reports (SRs) and Receiver Reports (RRs) [RFC3550]
>  exposes the basic metrics...

"expose"

>  However, these metrics provides...

"provide"


>  For example, it may be useful to distinguish between
>  packets lost and packets discarded due to late arrival, even though
>  they have the same impact on the multimedia quality, it helps in
>  identifying and diagnosing issues.

This is a run-on sentence. Suggest: "...due to late arrival. Even though..."

>  The WebRTC application extracts the statistic from the browser by
>  querying the getStats() API [W3C.WD-webrtc-20161124], but the browser
>  currently only reports the local variables i.e., the statistics
>  related to the outgoing RTP media streams and the incoming RTP media
>  streams.

I don't think the "currently" in this sentence is actually true any longer; and
I expect it to get increasingly less true as time goes on. Consider rephrasing.

>  [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of
>  any RTCP XRs and since their usage is optional.

This is not a sentence. Maybe remove "since"?

---------------------------------------------------------------------------

§4:

>  For example, an application may choose
>  to poll the stack for statistics every 1 second, in this case the
>  underlying stack local will return the current snapshot of the local
>  statistics (for incoming and outgoing media streams).

This is a comma splice. Consider: "...every second. In this case..."

>  However it may
>  return the same remote statistics as before for the remote
>  statistics

Add a comma after "However".

---------------------------------------------------------------------------

§5:

>  Since following metrics are all defined in RTCP XR which is not
>  mandated in WebRTC, all of them are local.

"Since the following..."
       ^^^

"...RTCP XR, which is..."
           ^

>  However, if RTCP XR is
>  supported by negotiation between two browsers, following metrics can
>  also be generated remotely and be sent to local by RTCP XR packets.

"...two browsers, the following..."
                  ^^^

>  Following metrics are classified into 3 categories: network impact

"The following metrics..."
 ^^^


>  viewpoint of application, e.g., bit rate, frames rate or jitter

"...frame rate..."

>  WebRTC
>  application can use these metrics to calculate the Mean Opinion Score

"applications"


---------------------------------------------------------------------------

§5.1.1:

>  lost and duplicated packets.  Lost packets counts are useful for

"Lost packet counts..."

---------------------------------------------------------------------------

§5.1.3:

>  the two run length vectors to identify congestion-related losses,
>  i.e., a router queue became overloaded causing delays and then

Replace "i.e." ("that is") with "e.g." ("for example").

---------------------------------------------------------------------------

§5.2.1:

>  The metric reports the cumulative size of the packets discarded in
>  the interval, it is complementary to number of discarded packets.

"...the interval. It is complementary..."

---------------------------------------------------------------------------

§5.2.2:

>  For
>  audio streams, a single RTP packet may contain one or multiple audio
>  frames, each of which has a fixed length.

I don't think this is generally true. Even in the case of WebRTC, I believe the
default mode of operation for Opus is VBR, which results in frame sizes that
vary from frame to frame.

>  The metrics in this category includes: number of discarded key

"...category include:..."

---------------------------------------------------------------------------


§5.3.1:

>  The un-repaired packets count and repaired loss count defined in

"...unrepaired packet count..."

>  packets to lost packets.  Including this kind of metrics helps the

Choose either "...these kinds of metrics..." or "...this kind of metric..."

---------------------------------------------------------------------------

§5.3.2:

>  related losses, i.e., a router queue became overloaded causing delays

Replace "i.e." with "e.g."

---------------------------------------------------------------------------

§6:

>  identifiers relevant to RTP media in WebRTC.  These group of
>  identifiers are defined on a ReportGroup corresponding to an
>  Synchronization source (SSRC).  In practice the application need to

"This group of identifiers..."
 ^^^^

"...corresponding to a synchronization source..."
                     ^

"...the application needs to..."
                    ^^^^^

>  For XR metrics, it depends on
>  two factors: 1) if it measured at the endpoint, 2) if it reported by
>  the endpoint in an XR report.

"...if it is measured..."
          ^^

"...if it is reported..."
          ^^

---------------------------------------------------------------------------

§11.2:

>  [W3C.WD-webrtc-20161124]
>             Sporny, M. and D. Longley, "WebRTC 1.0: Real-time
>             Communication Between Browsers", World Wide Web Consortium
>             WD WD-webrtc-20161124, November 2016,
>             <https://www.w3.org/TR/2016/WD-webrtc-20161124>.

The authorship of this document is incorrect (neither Sporny nor Longley are
authors on WebRTC).

The URL points to an outdated version of the specification. Please update to
https://www.w3.org/TR/2017/CR-webrtc-20171102/


>  [W3C.WD-webrtc-stats-20161214]
>             Alvestrand, H. and V. Singh, "Identifiers for
>             WebRTC&#039;s Statistics API", World Wide Web Consortium
>             WD WD-webrtc-stats-20161214, December 2016,
>             <https://www.w3.org/TR/2016/WD-webrtc-stats-20161214>.

Please replace "&#039;" with an apostrophe.

Please update the URL to https://www.w3.org/TR/2018/WD-webrtc-stats-20180519/