Re: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06

"Huangyihong (Rachel)" <rachel.huang@huawei.com> Thu, 02 November 2017 10:00 UTC

Return-Path: <rachel.huang@huawei.com>
X-Original-To: xrblock@ietfa.amsl.com
Delivered-To: xrblock@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C40B13F567; Thu, 2 Nov 2017 03:00:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.221
X-Spam-Level:
X-Spam-Status: No, score=-4.221 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id leZFRcDNVnZz; Thu, 2 Nov 2017 03:00:47 -0700 (PDT)
Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A526C13F4FA; Thu, 2 Nov 2017 03:00:43 -0700 (PDT)
Received: from 172.18.7.190 (EHLO lhreml705-cah.china.huawei.com) ([172.18.7.190]) by lhrrg01-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id DZC76982; Thu, 02 Nov 2017 10:00:41 +0000 (GMT)
Received: from NKGEML411-HUB.china.huawei.com (10.98.56.70) by lhreml705-cah.china.huawei.com (10.201.108.46) with Microsoft SMTP Server (TLS) id 14.3.361.1; Thu, 2 Nov 2017 10:00:39 +0000
Received: from NKGEML513-MBS.china.huawei.com ([169.254.2.198]) by nkgeml411-hub.china.huawei.com ([10.98.56.70]) with mapi id 14.03.0361.001; Thu, 2 Nov 2017 18:00:33 +0800
From: "Huangyihong (Rachel)" <rachel.huang@huawei.com>
To: Ben Campbell <ben@nostrum.com>, Varun Singh <varun@callstats.io>
CC: "draft-ietf-xrblock-rtcweb-rtcp-xr-metrics.all@ietf.org" <draft-ietf-xrblock-rtcweb-rtcp-xr-metrics.all@ietf.org>, xrblock <xrblock@ietf.org>
Thread-Topic: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06
Thread-Index: AQHTO8VfsJtQ+zH+8kexJpju540EL6LZcCcAgAbaxgCAIHyYkA==
Date: Thu, 02 Nov 2017 10:00:33 +0000
Message-ID: <51E6A56BD6A85142B9D172C87FC3ABBB9C5F8DEE@nkgeml513-mbs.china.huawei.com>
References: <99303AB6-D5EE-44EC-B515-DCB7966DADA6@nostrum.com> <CACHXSv7E+NhU_dvd63k57RfrVOAPoSFuOK8+-bRwsJTAcZ_ixQ@mail.gmail.com> <17BE8BAC-64E3-4A11-B3F0-C54E3D05244E@nostrum.com>
In-Reply-To: <17BE8BAC-64E3-4A11-B3F0-C54E3D05244E@nostrum.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.134.153.152]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-CFilter-Loop: Reflected
X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A090204.59FAECCA.0066, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=169.254.2.198, so=2013-06-18 04:22:30, dmn=2013-03-21 17:37:32
X-Mirapoint-Loop-Id: 370afb36fc3efbfb8515e93a700a4da6
Archived-At: <https://mailarchive.ietf.org/arch/msg/xrblock/Fh6GG2s1KYtCOHv_-4PILRt0yDM>
Subject: Re: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06
X-BeenThere: xrblock@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Metric Blocks for use with RTCP's Extended Report Framework working group discussion list <xrblock.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/xrblock>, <mailto:xrblock-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/xrblock/>
List-Post: <mailto:xrblock@ietf.org>
List-Help: <mailto:xrblock-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/xrblock>, <mailto:xrblock-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 02 Nov 2017 10:00:49 -0000

Hi Ben,

After discussion among the co-authors, we reached the consensus to make it informational. W3C stats API still refers to it, but in an informational way. We'll submit a new version to address your comments.

BR,
Rachel

> -----Original Message-----
> From: Ben Campbell [mailto:ben@nostrum.com]
> Sent: Friday, October 13, 2017 5:52 AM
> To: Varun Singh
> Cc: draft-ietf-xrblock-rtcweb-rtcp-xr-metrics.all@ietf.org; xrblock
> Subject: Re: [xrblock] AD Evaluation of
> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06
> 
> >
> > On Oct 8, 2017, at 8:11 AM, Varun Singh <varun@callstats.io> wrote:
> >
> > Hi Ben,
> >
> > See inline.
> >
> >> On Tue, Oct 3, 2017 at 12:28 AM, Ben Campbell <ben@nostrum.com>
> wrote:
> >> Hi,
> >>
> >> This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I
> would like to resolve at least the substantive comments prior to IETF LC.
> >>
> >> Thanks!
> >>
> >> Ben.
> >> —————————
> >>
> >> Substantive:
> >>
> >> - General: If I understand correctly, this document lists and describes XR
> metrics that a WebRTC application might choose to negotiate and expose via
> the WebRTC API. It does not define any new XR metrics. That brings up two
> questions:
> >>
> >> 1) Why is this specific to WebRTC? For the most parts, all the arguments
> here would apply to any sort of RTP endpoint.
> >>
> >
> > This draft was created initially as an input to RTCWEB and later as an
> > input to WebRTC Statistics API. Hence the use of WebRTC in the title
> > and the introduction. We could add to the introduction: "In general,
> > the metrics listed in this document can be exposed by any real-time
> > communications endpoint." Or something along those lines
> >
> >
> >> 2) Why is this standards track? This sort of material is typically informational,
> or occasionally a BCP. There are a small number of 2119 keywords, but I’m not
> sure they are needed or appropriate. (Specifics below.) My initial instinct is that
> this should be informational.
> >>
> >
> > The reason until recently (perhaps still exist) is that this was an
> > input from the IETF to W3C, I am not sure if there was an official
> > liaison statements sent to them? nonetheless, the W3C Statistics API
> > document needed a specification needed (a specification in any
> > standards body would suffice) for metrics to be accepted into the W3C
> > document.
> 
> To make sure I understand:That API document will reference this one? Do they
> require that specification to be standards track?
> 
> Was there discussion in xrblock about standards track verses BCP?
> 
> >
> > In practice, it helped that I was the co-author of both the IETF and
> > W3C document and hence was able to make the case for the metrics in
> > W3C.
> 
> Is that case already made? That is, does the need for a published RFC still
> exist?
> 
> >
> >> - General: This needs an IANA considerations section, even if it just contains
> the statement to the effect of “This document makes no requests for IANA”.
> >>
> >> - 1, last paragraph: “ The document also creates a registry containing
> identifiers from the metrics reported in the RTCP Sender, Receiver, and
> Extended Reports.”
> >
> > This should be removed. Thanks for pointing this out.
> >
> >>
> >> It doesn’t actually do that. I understand from the shepherd’s report that
> this was intentionally removed.
> >>
> >> — “All identifiers proposed in this document are RECOMMENDED to be
> >> implemented by an endpoint.  An endpoint MAY choose not to expose an
> >> identifier if it does not implement the corresponding RTCP Report. “
> >>
> >> Does the “RECOMMENDED” apply to all endpoints or just WebRTC
> >> endpoints? If the latter, doesn’t that requirement belong (or need to
> >> update) some requirement from an RTCWEB (maybe rtp-usage) document,
> >> or even the API itself? (I suspect that this draft does not have
> >> standing to state this normatively.) What is meant by “MAY choose not
> >> to expose”. Is that talking about via the WebRTC API? If so, isn’t
> >> that up to the API definition? (That is, it shouldn’t be normative
> >> here.)
> >>
> >
> > It should be applicable to any endpoint, although it is pertinent or
> > timely for the WebRTC endpoint to expose these.
> > The purpose of the statement was to highlight that if the
> > middlebox/endpoint sends RTCP XRs then it should expose the data
> > available via the API.
> >
> > I am open to changing to a non-normative lower case guidance if that
> > would be more suitable, although if this document is making
> > recommendations to the W3C document -- wouldn't we keep the uppercase?
> 
> In general, 2119 keywords are better used when we are talking about
> interoperability issues in a protocol. I suggest switching to lower case unless
> there’s some reason to believe the W3C won’t pay attention unless it’s in
> all-caps. (I don’t think that’s usually the case.)
> 
> >
> > co-authors, what do you think?
> >
> >> -2: If this document keeps the normative keywords, please use the
> >> updated boilerplate from RFC 8174. (I note that there are at least
> >> some uncapitalized “may”s that do not seem normative.)
> >>
> >
> > depending on the above, we should fix these to uppercase.
> >
> >> - 5.2.2, last paragraph: “The following metrics can also be considered…”
> >> Be considered by whom? Implementors? API designers?
> >>
> >
> > Both.
> >
> >> - 6, first paragraph: “In practice the application MUST be able to query the
> statistic identifiers on both an incoming (remote) and outgoing (local) media
> stream.”
> >> What does the MUST requirement apply to? The WebRTC API? This seems
> more like a statement of fact that “ the application needs to be able…”
> >>
> >
> > correct, "needs to" is better.
> >
> >> -8: "Therefore encryption procedures, such as the ones suggested for a
> Secure RTCP (SRTCP), need to be used.”
> >> The text should describe the reasons encryption is needed. Also, is this a
> new normative requirement, or a restatement of an existing requirement?
> >>
> >
> > I believe it is restatement alluding to the guidance in the
> > corresponding RTCP XRs.
> >
> >> Editorial:
> >>
> >> -1, paragraph 1:" If sufficient information (metrics or statistics)
> >> are provided to the applications, it can attempt to improve the media
> >> quality. “
> >>
> >> s / “are provided” / “is provided”
> >> s / applications / application
> >>
> >> -3, 2nd paragraph: It would help to clarify that the references to section 5
> and 6 are to this document, not RFC 3611.
> >>
> >> — 3rd paragraph: "At the moment…”
> >> Please clarify that is at the time of writing (not reading)
> >>
> >> -5, 2nd paragraph: “ Application impact metrics mainly collect the
> >> information in the viewpoint of application … “ s / in / from Also,
> >> please expand “FEC” on first mention.
> >>
> >> - 5.1.1, first paragraph: “ Duplicate packets may be a result of network
> delays, which causes the sender to retransmit the original packets.”
> >> s/ :delays, which” / “delays that”
> >>
> >> — last paragraph: "For those RTCWEB services with jitter buffer…”
> >> s/buffer/buffers
> >>
> >> - 5.1.2, first paragraph:
> >> First sentence is a comma splice.
> >> — “some transitory nature of the impairments”
> >> Should this say “the transitory nature of some impairments”?
> >> — “ Distributed burst provides a higher subjective quality”
> >> “Burst distribution …”?
> >> — Last paragraph: “ Hence, if WebRTC application”
> >> Missing article (or should application be plural?)
> >>
> >> - 5.2.1, title: Should “Discard” be “Discarded”?
> >>
> >> -7: Please clarify that this is at the time of this writing.
> >>
> >
> > Thanks for your feedback and suggestions, we will fix these.
> >
> > Regards,
> > Varun
> >
> >
> > --
> > Founder, CEO, callstats.io
> > http://www.callstats.io
> >
> > Interested in networking, media quality, and diagnostics.
> > We are hiring!: www.callstats.io/jobs/