Re: [xrblock] Adam Roach's No Objection on draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)

["Huangyihong (Rachel)" <[e-mail]>]

Hi Adam,

Thanks for the comments. Please see inline.

BR,
Rachel

> -----Original Message-----
> From: Adam Roach [mailto:[e-mail]]
> Sent: Tuesday, May 22, 2018 8:56 AM
> To: The IESG
> Cc: [e-mail]; Shida Schubert;
> [e-mail]; [e-mail]; [e-mail]
> Subject: Adam Roach's No Objection on
> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)
> 
> Adam Roach has entered the following ballot position for
> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: No Objection
> 
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
> 
> 
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
> 
> 
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcweb-rtcp-xr-metrics/
> 
> 
> 
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> Thanks to the authors and contributors on this document. I have two
> substantive
> comments, and a handful of minor nits.
> 
> ---------------------------------------------------------------------------
> 
> §5.3.1:
> 
> >  application can use these metrics to calculate the Mean Opinion Score
> >  (MoS) values or Media Delivery Index (MDI) for their services.
> 
> "...calculate Estimated Mean Opinion Score (eMOS) values..."
> 
> (MOS scores necessarily require human subjective evaluation of audio under
> carefully controlled circumstances -- any computer-generated metrics are
> inherently estimations, and should not be called MOS scores.)
> 

[Rachel]: You're right. Will change to "estimated Mean Opinion Score (MOS)" in the next version.

> ---------------------------------------------------------------------------
> 
> §5.3.1:
> 
> >  Error-resilience mechanisms, like RTP retransmission or FEC, are
> >  optional in RTCWEB because the overhead of the repair bits adding to
> >  the original streams.
> 
> This is a little misleading. draft-ietf-rtcweb-rtp-usage says:
> 
>    WebRTC endpoints MUST follow the recommendations for FEC use given in
>    [I-D.ietf-rtcweb-fec].
> 
> And, in turn, draft-ietf-rtcweb-fec says:
> 
>    To support the functionality recommended above, implementations MUST
>    be able to receive and make use of the relevant FEC formats for their
>    supported audio codecs, and MUST indicate this support, as described
>    in Section 4.  Use of these formats when sending, as mentioned above,
>    is RECOMMENDED.
> 
> Rather than trying to reiterate this somewhat nuanced situation in *this*
> document, I recommend removing the entire sentence and fixing the rest of
> the
> paragraph. If you choose to keep it, please make sure it more clearly explains
> the level of FEC support required of RTCWEB implementations.
> 

[Rachel]: Your concern makes sense to me. I'll delete it as following:

OLD

"
   Error-resilience mechanisms, like RTP retransmission or FEC, are
   optional in RTCWEB because the overhead of the repair bits adding to
   the original streams.  But they do help to greatly reduce the impact
   of packet loss and enhance the quality of transmission.  Web
   applications could support certain repair mechanism after negotiation
   between both sides of browsers when needed.  
"

NEW
"
Web applications could support certain RTP error-resilience mechanisms following the recommendations specified in [draft-ietf-rtcweb-rtp-usage].
"

> 
> ================================================================
> ===========
> 
> The remainder of my comments are grammatical or editorial nits that should
> be
> fixed before progressing the document.
> 
> ---------------------------------------------------------------------------
> 
> §3:
> 
> >  The RTCP Sender Reports (SRs) and Receiver Reports (RRs) [RFC3550]
> >  exposes the basic metrics...
> 
> "expose"
> 
> >  However, these metrics provides...
> 
> "provide"
> 
> 
> >  For example, it may be useful to distinguish between
> >  packets lost and packets discarded due to late arrival, even though
> >  they have the same impact on the multimedia quality, it helps in
> >  identifying and diagnosing issues.
> 
> This is a run-on sentence. Suggest: "...due to late arrival. Even though..."
> 
> >  The WebRTC application extracts the statistic from the browser by
> >  querying the getStats() API [W3C.WD-webrtc-20161124], but the browser
> >  currently only reports the local variables i.e., the statistics
> >  related to the outgoing RTP media streams and the incoming RTP media
> >  streams.
> 
> I don't think the "currently" in this sentence is actually true any longer; and
> I expect it to get increasingly less true as time goes on. Consider rephrasing.
> 
> >  [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of
> >  any RTCP XRs and since their usage is optional.
> 
> This is not a sentence. Maybe remove "since"?
> 
> ---------------------------------------------------------------------------
> 
> §4:
> 
> >  For example, an application may choose
> >  to poll the stack for statistics every 1 second, in this case the
> >  underlying stack local will return the current snapshot of the local
> >  statistics (for incoming and outgoing media streams).
> 
> This is a comma splice. Consider: "...every second. In this case..."
> 
> >  However it may
> >  return the same remote statistics as before for the remote
> >  statistics
> 
> Add a comma after "However".
> 
> ---------------------------------------------------------------------------
> 
> §5:
> 
> >  Since following metrics are all defined in RTCP XR which is not
> >  mandated in WebRTC, all of them are local.
> 
> "Since the following..."
>        ^^^
> 
> "...RTCP XR, which is..."
>            ^
> 
> >  However, if RTCP XR is
> >  supported by negotiation between two browsers, following metrics can
> >  also be generated remotely and be sent to local by RTCP XR packets.
> 
> "...two browsers, the following..."
>                   ^^^
> 
> >  Following metrics are classified into 3 categories: network impact
> 
> "The following metrics..."
>  ^^^
> 
> 
> >  viewpoint of application, e.g., bit rate, frames rate or jitter
> 
> "...frame rate..."
> 
> >  WebRTC
> >  application can use these metrics to calculate the Mean Opinion Score
> 
> "applications"
> 
> 
> ---------------------------------------------------------------------------
> 
> §5.1.1:
> 
> >  lost and duplicated packets.  Lost packets counts are useful for
> 
> "Lost packet counts..."
> 
> ---------------------------------------------------------------------------
> 
> §5.1.3:
> 
> >  the two run length vectors to identify congestion-related losses,
> >  i.e., a router queue became overloaded causing delays and then
> 
> Replace "i.e." ("that is") with "e.g." ("for example").
> 
> ---------------------------------------------------------------------------
> 
> §5.2.1:
> 
> >  The metric reports the cumulative size of the packets discarded in
> >  the interval, it is complementary to number of discarded packets.
> 
> "...the interval. It is complementary..."
> 
> ---------------------------------------------------------------------------
> 
> §5.2.2:
> 
> >  For
> >  audio streams, a single RTP packet may contain one or multiple audio
> >  frames, each of which has a fixed length.
> 
> I don't think this is generally true. Even in the case of WebRTC, I believe the
> default mode of operation for Opus is VBR, which results in frame sizes that
> vary from frame to frame.
> 
> >  The metrics in this category includes: number of discarded key
> 
> "...category include:..."
> 
> ---------------------------------------------------------------------------
> 
> 
> §5.3.1:
> 
> >  The un-repaired packets count and repaired loss count defined in
> 
> "...unrepaired packet count..."
> 
> >  packets to lost packets.  Including this kind of metrics helps the
> 
> Choose either "...these kinds of metrics..." or "...this kind of metric..."
> 
> ---------------------------------------------------------------------------
> 
> §5.3.2:
> 
> >  related losses, i.e., a router queue became overloaded causing delays
> 
> Replace "i.e." with "e.g."
> 
> ---------------------------------------------------------------------------
> 
> §6:
> 
> >  identifiers relevant to RTP media in WebRTC.  These group of
> >  identifiers are defined on a ReportGroup corresponding to an
> >  Synchronization source (SSRC).  In practice the application need to
> 
> "This group of identifiers..."
>  ^^^^
> 
> "...corresponding to a synchronization source..."
>                      ^
> 
> "...the application needs to..."
>                     ^^^^^
> 
> >  For XR metrics, it depends on
> >  two factors: 1) if it measured at the endpoint, 2) if it reported by
> >  the endpoint in an XR report.
> 
> "...if it is measured..."
>           ^^
> 
> "...if it is reported..."
>           ^^
> 
> ---------------------------------------------------------------------------
> 
> §11.2:
> 
> >  [W3C.WD-webrtc-20161124]
> >             Sporny, M. and D. Longley, "WebRTC 1.0: Real-time
> >             Communication Between Browsers", World Wide Web
> Consortium
> >             WD WD-webrtc-20161124, November 2016,
> >             <https://www.w3.org/TR/2016/WD-webrtc-20161124>.
> 
> The authorship of this document is incorrect (neither Sporny nor Longley are
> authors on WebRTC).
> 
> The URL points to an outdated version of the specification. Please update to
> https://www.w3.org/TR/2017/CR-webrtc-20171102/
> 
> 
> >  [W3C.WD-webrtc-stats-20161214]
> >             Alvestrand, H. and V. Singh, "Identifiers for
> >             WebRTC&#039;s Statistics API", World Wide Web Consortium
> >             WD WD-webrtc-stats-20161214, December 2016,
> >             <https://www.w3.org/TR/2016/WD-webrtc-stats-20161214>.
> 
> Please replace "&#039;" with an apostrophe.
> 
> Please update the URL to
> https://www.w3.org/TR/2018/WD-webrtc-stats-20180519/
> 

[Rachel]: They'll be fixed in the next version. Thanks.