Re: [xrblock] Adam Roach's No Objection on draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)

["Huangyihong (Rachel)" <[e-mail]>]

Will do it today.

BR,
Rachel

> -----Original Message-----
> From: Ben Campbell [mailto:[e-mail]]
> Sent: Friday, May 25, 2018 3:27 AM
> To: [e-mail]; Huangyihong (Rachel)
> <[e-mail]>
> Cc: The IESG <[e-mail]>; [e-mail];
> [e-mail]; [e-mail]
> Subject: Re: Adam Roach's No Objection on
> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)
> 
> Hi,
> 
> It looks like the discussion on Adam’s comments have reached resolution, and
> I think that is also mostly true for the other IESG comments. Authors; Please
> submit a new revision when you are ready.
> 
> Thanks!
> 
> Ben.
> 
> > On May 21, 2018, at 10:20 PM, Adam Roach <[e-mail]> wrote:
> >
> > Thanks! Your proposed resolutions look good to me.
> >
> > /a
> >
> > On 5/21/18 10:18 PM, Huangyihong (Rachel) wrote:
> >> Hi Adam,
> >>
> >> Thanks for the comments. Please see inline.
> >>
> >> BR,
> >> Rachel
> >>
> >>> -----Original Message-----
> >>> From: Adam Roach [mailto:[e-mail]]
> >>> Sent: Tuesday, May 22, 2018 8:56 AM
> >>> To: The IESG
> >>> Cc: [e-mail]; Shida
> >>> Schubert; [e-mail]; [e-mail];
> >>> [e-mail]
> >>> Subject: Adam Roach's No Objection on
> >>> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: (with COMMENT)
> >>>
> >>> Adam Roach has entered the following ballot position for
> >>> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-09: No Objection
> >>>
> >>> When responding, please keep the subject line intact and reply to
> >>> all email addresses included in the To and CC lines. (Feel free to
> >>> cut this introductory paragraph, however.)
> >>>
> >>>
> >>> Please refer to
> >>> https://www.ietf.org/iesg/statement/discuss-criteria.html
> >>> for more information about IESG DISCUSS and COMMENT positions.
> >>>
> >>>
> >>> The document, along with other ballot positions, can be found here:
> >>> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcweb-rtcp-xr-m
> >>> etrics/
> >>>
> >>>
> >>>
> >>> --------------------------------------------------------------------
> >>> --
> >>> COMMENT:
> >>> --------------------------------------------------------------------
> >>> --
> >>>
> >>> Thanks to the authors and contributors on this document. I have two
> >>> substantive comments, and a handful of minor nits.
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5.3.1:
> >>>
> >>>>  application can use these metrics to calculate the Mean Opinion
> >>>> Score
> >>>>  (MoS) values or Media Delivery Index (MDI) for their services.
> >>> "...calculate Estimated Mean Opinion Score (eMOS) values..."
> >>>
> >>> (MOS scores necessarily require human subjective evaluation of audio
> >>> under carefully controlled circumstances -- any computer-generated
> >>> metrics are inherently estimations, and should not be called MOS
> >>> scores.)
> >>>
> >> [Rachel]: You're right. Will change to "estimated Mean Opinion Score (MOS)"
> in the next version.
> >>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5.3.1:
> >>>
> >>>>  Error-resilience mechanisms, like RTP retransmission or FEC, are
> >>>> optional in RTCWEB because the overhead of the repair bits adding
> >>>> to  the original streams.
> >>> This is a little misleading. draft-ietf-rtcweb-rtp-usage says:
> >>>
> >>>    WebRTC endpoints MUST follow the recommendations for FEC use
> given in
> >>>    [I-D.ietf-rtcweb-fec].
> >>>
> >>> And, in turn, draft-ietf-rtcweb-fec says:
> >>>
> >>>    To support the functionality recommended above, implementations
> MUST
> >>>    be able to receive and make use of the relevant FEC formats for their
> >>>    supported audio codecs, and MUST indicate this support, as described
> >>>    in Section 4.  Use of these formats when sending, as mentioned
> above,
> >>>    is RECOMMENDED.
> >>>
> >>> Rather than trying to reiterate this somewhat nuanced situation in
> >>> *this* document, I recommend removing the entire sentence and fixing
> >>> the rest of the paragraph. If you choose to keep it, please make
> >>> sure it more clearly explains the level of FEC support required of
> >>> RTCWEB implementations.
> >>>
> >> [Rachel]: Your concern makes sense to me. I'll delete it as following:
> >>
> >> OLD
> >>
> >> "
> >>    Error-resilience mechanisms, like RTP retransmission or FEC, are
> >>    optional in RTCWEB because the overhead of the repair bits adding to
> >>    the original streams.  But they do help to greatly reduce the impact
> >>    of packet loss and enhance the quality of transmission.  Web
> >>    applications could support certain repair mechanism after negotiation
> >>    between both sides of browsers when needed.
> >> "
> >>
> >> NEW
> >> "
> >> Web applications could support certain RTP error-resilience mechanisms
> following the recommendations specified in [draft-ietf-rtcweb-rtp-usage].
> >> "
> >>
> >>>
> ================================================================
> >>> ===========
> >>>
> >>> The remainder of my comments are grammatical or editorial nits that
> >>> should be fixed before progressing the document.
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §3:
> >>>
> >>>>  The RTCP Sender Reports (SRs) and Receiver Reports (RRs) [RFC3550]
> >>>> exposes the basic metrics...
> >>> "expose"
> >>>
> >>>>  However, these metrics provides...
> >>> "provide"
> >>>
> >>>
> >>>>  For example, it may be useful to distinguish between  packets lost
> >>>> and packets discarded due to late arrival, even though  they have
> >>>> the same impact on the multimedia quality, it helps in  identifying
> >>>> and diagnosing issues.
> >>> This is a run-on sentence. Suggest: "...due to late arrival. Even though..."
> >>>
> >>>>  The WebRTC application extracts the statistic from the browser by
> >>>> querying the getStats() API [W3C.WD-webrtc-20161124], but the
> >>>> browser  currently only reports the local variables i.e., the
> >>>> statistics  related to the outgoing RTP media streams and the
> >>>> incoming RTP media  streams.
> >>> I don't think the "currently" in this sentence is actually true any
> >>> longer; and I expect it to get increasingly less true as time goes on.
> Consider rephrasing.
> >>>
> >>>>  [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of  any RTCP
> >>>> XRs and since their usage is optional.
> >>> This is not a sentence. Maybe remove "since"?
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §4:
> >>>
> >>>>  For example, an application may choose  to poll the stack for
> >>>> statistics every 1 second, in this case the  underlying stack local
> >>>> will return the current snapshot of the local  statistics (for
> >>>> incoming and outgoing media streams).
> >>> This is a comma splice. Consider: "...every second. In this case..."
> >>>
> >>>>  However it may
> >>>>  return the same remote statistics as before for the remote
> >>>> statistics
> >>> Add a comma after "However".
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5:
> >>>
> >>>>  Since following metrics are all defined in RTCP XR which is not
> >>>> mandated in WebRTC, all of them are local.
> >>> "Since the following..."
> >>>        ^^^
> >>>
> >>> "...RTCP XR, which is..."
> >>>            ^
> >>>
> >>>>  However, if RTCP XR is
> >>>>  supported by negotiation between two browsers, following metrics
> >>>> can  also be generated remotely and be sent to local by RTCP XR
> packets.
> >>> "...two browsers, the following..."
> >>>                   ^^^
> >>>
> >>>>  Following metrics are classified into 3 categories: network impact
> >>> "The following metrics..."
> >>>  ^^^
> >>>
> >>>
> >>>>  viewpoint of application, e.g., bit rate, frames rate or jitter
> >>> "...frame rate..."
> >>>
> >>>>  WebRTC
> >>>>  application can use these metrics to calculate the Mean Opinion
> >>>> Score
> >>> "applications"
> >>>
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5.1.1:
> >>>
> >>>>  lost and duplicated packets.  Lost packets counts are useful for
> >>> "Lost packet counts..."
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5.1.3:
> >>>
> >>>>  the two run length vectors to identify congestion-related losses,
> >>>> i.e., a router queue became overloaded causing delays and then
> >>> Replace "i.e." ("that is") with "e.g." ("for example").
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5.2.1:
> >>>
> >>>>  The metric reports the cumulative size of the packets discarded in
> >>>> the interval, it is complementary to number of discarded packets.
> >>> "...the interval. It is complementary..."
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5.2.2:
> >>>
> >>>>  For
> >>>>  audio streams, a single RTP packet may contain one or multiple
> >>>> audio  frames, each of which has a fixed length.
> >>> I don't think this is generally true. Even in the case of WebRTC, I
> >>> believe the default mode of operation for Opus is VBR, which results
> >>> in frame sizes that vary from frame to frame.
> >>>
> >>>>  The metrics in this category includes: number of discarded key
> >>> "...category include:..."
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>>
> >>> §5.3.1:
> >>>
> >>>>  The un-repaired packets count and repaired loss count defined in
> >>> "...unrepaired packet count..."
> >>>
> >>>>  packets to lost packets.  Including this kind of metrics helps the
> >>> Choose either "...these kinds of metrics..." or "...this kind of metric..."
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §5.3.2:
> >>>
> >>>>  related losses, i.e., a router queue became overloaded causing
> >>>> delays
> >>> Replace "i.e." with "e.g."
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §6:
> >>>
> >>>>  identifiers relevant to RTP media in WebRTC.  These group of
> >>>> identifiers are defined on a ReportGroup corresponding to an
> >>>> Synchronization source (SSRC).  In practice the application need to
> >>> "This group of identifiers..."
> >>>  ^^^^
> >>>
> >>> "...corresponding to a synchronization source..."
> >>>                      ^
> >>>
> >>> "...the application needs to..."
> >>>                     ^^^^^
> >>>
> >>>>  For XR metrics, it depends on
> >>>>  two factors: 1) if it measured at the endpoint, 2) if it reported
> >>>> by  the endpoint in an XR report.
> >>> "...if it is measured..."
> >>>           ^^
> >>>
> >>> "...if it is reported..."
> >>>           ^^
> >>>
> >>> --------------------------------------------------------------------
> >>> -------
> >>>
> >>> §11.2:
> >>>
> >>>>  [W3C.WD-webrtc-20161124]
> >>>>             Sporny, M. and D. Longley, "WebRTC 1.0: Real-time
> >>>>             Communication Between Browsers", World Wide Web
> >>> Consortium
> >>>>             WD WD-webrtc-20161124, November 2016,
> >>>>             <https://www.w3.org/TR/2016/WD-webrtc-20161124>.
> >>> The authorship of this document is incorrect (neither Sporny nor
> >>> Longley are authors on WebRTC).
> >>>
> >>> The URL points to an outdated version of the specification. Please
> >>> update to https://www.w3.org/TR/2017/CR-webrtc-20171102/
> >>>
> >>>
> >>>>  [W3C.WD-webrtc-stats-20161214]
> >>>>             Alvestrand, H. and V. Singh, "Identifiers for
> >>>>             WebRTC&#039;s Statistics API", World Wide Web
> Consortium
> >>>>             WD WD-webrtc-stats-20161214, December 2016,
> >>>>
> <https://www.w3.org/TR/2016/WD-webrtc-stats-20161214>.
> >>> Please replace "&#039;" with an apostrophe.
> >>>
> >>> Please update the URL to
> >>> https://www.w3.org/TR/2018/WD-webrtc-stats-20180519/
> >>>
> >> [Rachel]: They'll be fixed in the next version. Thanks.
> >
> >