Re: [xrblock] Adam Roach's No Objection on draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)

[Adam Roach <[e-mail]>]

Thanks! Your proposed resolutions look good to me.

/a

On 5/21/18 10:18 PM, Huangyihong (Rachel) wrote:
> Hi Adam,
>
> Thanks for the comments. Please see inline.
>
> BR,
> Rachel
>
>> -----Original Message-----
>> From: Adam Roach [mailto:[e-mail]]
>> Sent: Tuesday, May 22, 2018 8:56 AM
>> To: The IESG
>> Cc: [e-mail]; Shida Schubert;
>> [e-mail]; [e-mail]; [e-mail]
>> Subject: Adam Roach's No Objection on
>> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)
>>
>> Adam Roach has entered the following ballot position for
>> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: No Objection
>>
>> When responding, please keep the subject line intact and reply to all
>> email addresses included in the To and CC lines. (Feel free to cut this
>> introductory paragraph, however.)
>>
>>
>> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
>> for more information about IESG DISCUSS and COMMENT positions.
>>
>>
>> The document, along with other ballot positions, can be found here:
>> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcweb-rtcp-xr-metrics/
>>
>>
>>
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>
>> Thanks to the authors and contributors on this document. I have two
>> substantive
>> comments, and a handful of minor nits.
>>
>> ---------------------------------------------------------------------------
>>
>> §5.3.1:
>>
>>>   application can use these metrics to calculate the Mean Opinion Score
>>>   (MoS) values or Media Delivery Index (MDI) for their services.
>> "...calculate Estimated Mean Opinion Score (eMOS) values..."
>>
>> (MOS scores necessarily require human subjective evaluation of audio under
>> carefully controlled circumstances -- any computer-generated metrics are
>> inherently estimations, and should not be called MOS scores.)
>>
> [Rachel]: You're right. Will change to "estimated Mean Opinion Score (MOS)" in the next version.
>
>> ---------------------------------------------------------------------------
>>
>> §5.3.1:
>>
>>>   Error-resilience mechanisms, like RTP retransmission or FEC, are
>>>   optional in RTCWEB because the overhead of the repair bits adding to
>>>   the original streams.
>> This is a little misleading. draft-ietf-rtcweb-rtp-usage says:
>>
>>     WebRTC endpoints MUST follow the recommendations for FEC use given in
>>     [I-D.ietf-rtcweb-fec].
>>
>> And, in turn, draft-ietf-rtcweb-fec says:
>>
>>     To support the functionality recommended above, implementations MUST
>>     be able to receive and make use of the relevant FEC formats for their
>>     supported audio codecs, and MUST indicate this support, as described
>>     in Section 4.  Use of these formats when sending, as mentioned above,
>>     is RECOMMENDED.
>>
>> Rather than trying to reiterate this somewhat nuanced situation in *this*
>> document, I recommend removing the entire sentence and fixing the rest of
>> the
>> paragraph. If you choose to keep it, please make sure it more clearly explains
>> the level of FEC support required of RTCWEB implementations.
>>
> [Rachel]: Your concern makes sense to me. I'll delete it as following:
>
> OLD
>
> "
>     Error-resilience mechanisms, like RTP retransmission or FEC, are
>     optional in RTCWEB because the overhead of the repair bits adding to
>     the original streams.  But they do help to greatly reduce the impact
>     of packet loss and enhance the quality of transmission.  Web
>     applications could support certain repair mechanism after negotiation
>     between both sides of browsers when needed.
> "
>
> NEW
> "
> Web applications could support certain RTP error-resilience mechanisms following the recommendations specified in [draft-ietf-rtcweb-rtp-usage].
> "
>
>> ================================================================
>> ===========
>>
>> The remainder of my comments are grammatical or editorial nits that should
>> be
>> fixed before progressing the document.
>>
>> ---------------------------------------------------------------------------
>>
>> §3:
>>
>>>   The RTCP Sender Reports (SRs) and Receiver Reports (RRs) [RFC3550]
>>>   exposes the basic metrics...
>> "expose"
>>
>>>   However, these metrics provides...
>> "provide"
>>
>>
>>>   For example, it may be useful to distinguish between
>>>   packets lost and packets discarded due to late arrival, even though
>>>   they have the same impact on the multimedia quality, it helps in
>>>   identifying and diagnosing issues.
>> This is a run-on sentence. Suggest: "...due to late arrival. Even though..."
>>
>>>   The WebRTC application extracts the statistic from the browser by
>>>   querying the getStats() API [W3C.WD-webrtc-20161124], but the browser
>>>   currently only reports the local variables i.e., the statistics
>>>   related to the outgoing RTP media streams and the incoming RTP media
>>>   streams.
>> I don't think the "currently" in this sentence is actually true any longer; and
>> I expect it to get increasingly less true as time goes on. Consider rephrasing.
>>
>>>   [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of
>>>   any RTCP XRs and since their usage is optional.
>> This is not a sentence. Maybe remove "since"?
>>
>> ---------------------------------------------------------------------------
>>
>> §4:
>>
>>>   For example, an application may choose
>>>   to poll the stack for statistics every 1 second, in this case the
>>>   underlying stack local will return the current snapshot of the local
>>>   statistics (for incoming and outgoing media streams).
>> This is a comma splice. Consider: "...every second. In this case..."
>>
>>>   However it may
>>>   return the same remote statistics as before for the remote
>>>   statistics
>> Add a comma after "However".
>>
>> ---------------------------------------------------------------------------
>>
>> §5:
>>
>>>   Since following metrics are all defined in RTCP XR which is not
>>>   mandated in WebRTC, all of them are local.
>> "Since the following..."
>>         ^^^
>>
>> "...RTCP XR, which is..."
>>             ^
>>
>>>   However, if RTCP XR is
>>>   supported by negotiation between two browsers, following metrics can
>>>   also be generated remotely and be sent to local by RTCP XR packets.
>> "...two browsers, the following..."
>>                    ^^^
>>
>>>   Following metrics are classified into 3 categories: network impact
>> "The following metrics..."
>>   ^^^
>>
>>
>>>   viewpoint of application, e.g., bit rate, frames rate or jitter
>> "...frame rate..."
>>
>>>   WebRTC
>>>   application can use these metrics to calculate the Mean Opinion Score
>> "applications"
>>
>>
>> ---------------------------------------------------------------------------
>>
>> §5.1.1:
>>
>>>   lost and duplicated packets.  Lost packets counts are useful for
>> "Lost packet counts..."
>>
>> ---------------------------------------------------------------------------
>>
>> §5.1.3:
>>
>>>   the two run length vectors to identify congestion-related losses,
>>>   i.e., a router queue became overloaded causing delays and then
>> Replace "i.e." ("that is") with "e.g." ("for example").
>>
>> ---------------------------------------------------------------------------
>>
>> §5.2.1:
>>
>>>   The metric reports the cumulative size of the packets discarded in
>>>   the interval, it is complementary to number of discarded packets.
>> "...the interval. It is complementary..."
>>
>> ---------------------------------------------------------------------------
>>
>> §5.2.2:
>>
>>>   For
>>>   audio streams, a single RTP packet may contain one or multiple audio
>>>   frames, each of which has a fixed length.
>> I don't think this is generally true. Even in the case of WebRTC, I believe the
>> default mode of operation for Opus is VBR, which results in frame sizes that
>> vary from frame to frame.
>>
>>>   The metrics in this category includes: number of discarded key
>> "...category include:..."
>>
>> ---------------------------------------------------------------------------
>>
>>
>> §5.3.1:
>>
>>>   The un-repaired packets count and repaired loss count defined in
>> "...unrepaired packet count..."
>>
>>>   packets to lost packets.  Including this kind of metrics helps the
>> Choose either "...these kinds of metrics..." or "...this kind of metric..."
>>
>> ---------------------------------------------------------------------------
>>
>> §5.3.2:
>>
>>>   related losses, i.e., a router queue became overloaded causing delays
>> Replace "i.e." with "e.g."
>>
>> ---------------------------------------------------------------------------
>>
>> §6:
>>
>>>   identifiers relevant to RTP media in WebRTC.  These group of
>>>   identifiers are defined on a ReportGroup corresponding to an
>>>   Synchronization source (SSRC).  In practice the application need to
>> "This group of identifiers..."
>>   ^^^^
>>
>> "...corresponding to a synchronization source..."
>>                       ^
>>
>> "...the application needs to..."
>>                      ^^^^^
>>
>>>   For XR metrics, it depends on
>>>   two factors: 1) if it measured at the endpoint, 2) if it reported by
>>>   the endpoint in an XR report.
>> "...if it is measured..."
>>            ^^
>>
>> "...if it is reported..."
>>            ^^
>>
>> ---------------------------------------------------------------------------
>>
>> §11.2:
>>
>>>   [W3C.WD-webrtc-20161124]
>>>              Sporny, M. and D. Longley, "WebRTC 1.0: Real-time
>>>              Communication Between Browsers", World Wide Web
>> Consortium
>>>              WD WD-webrtc-20161124, November 2016,
>>>              <https://www.w3.org/TR/2016/WD-webrtc-20161124>.
>> The authorship of this document is incorrect (neither Sporny nor Longley are
>> authors on WebRTC).
>>
>> The URL points to an outdated version of the specification. Please update to
>> https://www.w3.org/TR/2017/CR-webrtc-20171102/
>>
>>
>>>   [W3C.WD-webrtc-stats-20161214]
>>>              Alvestrand, H. and V. Singh, "Identifiers for
>>>              WebRTC&#039;s Statistics API", World Wide Web Consortium
>>>              WD WD-webrtc-stats-20161214, December 2016,
>>>              <https://www.w3.org/TR/2016/WD-webrtc-stats-20161214>.
>> Please replace "&#039;" with an apostrophe.
>>
>> Please update the URL to
>> https://www.w3.org/TR/2018/WD-webrtc-stats-20180519/
>>
> [Rachel]: They'll be fixed in the next version. Thanks.