Re: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06

Ben Campbell <ben@nostrum.com> Thu, 12 October 2017 21:51 UTC

Return-Path: <ben@nostrum.com>
X-Original-To: xrblock@ietfa.amsl.com
Delivered-To: xrblock@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 28D0C1344AD; Thu, 12 Oct 2017 14:51:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.878
X-Spam-Level:
X-Spam-Status: No, score=-1.878 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_QP_LONG_LINE=0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yA7L_7tElJXa; Thu, 12 Oct 2017 14:51:46 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C9FC2133071; Thu, 12 Oct 2017 14:51:46 -0700 (PDT)
Received: from [10.0.1.75] (cpe-66-25-7-22.tx.res.rr.com [66.25.7.22]) (authenticated bits=0) by nostrum.com (8.15.2/8.15.2) with ESMTPSA id v9CLpg2O047636 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 12 Oct 2017 16:51:43 -0500 (CDT) (envelope-from ben@nostrum.com)
X-Authentication-Warning: raven.nostrum.com: Host cpe-66-25-7-22.tx.res.rr.com [66.25.7.22] claimed to be [10.0.1.75]
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (1.0)
From: Ben Campbell <ben@nostrum.com>
X-Mailer: iPad Mail (15A421)
In-Reply-To: <CACHXSv7E+NhU_dvd63k57RfrVOAPoSFuOK8+-bRwsJTAcZ_ixQ@mail.gmail.com>
Date: Thu, 12 Oct 2017 16:51:42 -0500
Cc: draft-ietf-xrblock-rtcweb-rtcp-xr-metrics.all@ietf.org, xrblock <xrblock@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <17BE8BAC-64E3-4A11-B3F0-C54E3D05244E@nostrum.com>
References: <99303AB6-D5EE-44EC-B515-DCB7966DADA6@nostrum.com> <CACHXSv7E+NhU_dvd63k57RfrVOAPoSFuOK8+-bRwsJTAcZ_ixQ@mail.gmail.com>
To: Varun Singh <varun@callstats.io>
Archived-At: <https://mailarchive.ietf.org/arch/msg/xrblock/cs2P3pIwtI45t8skjzCpkO25640>
Subject: Re: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06
X-BeenThere: xrblock@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Metric Blocks for use with RTCP's Extended Report Framework working group discussion list <xrblock.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/xrblock>, <mailto:xrblock-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/xrblock/>
List-Post: <mailto:xrblock@ietf.org>
List-Help: <mailto:xrblock-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/xrblock>, <mailto:xrblock-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Oct 2017 21:51:49 -0000

> 
> On Oct 8, 2017, at 8:11 AM, Varun Singh <varun@callstats.io> wrote:
> 
> Hi Ben,
> 
> See inline.
> 
>> On Tue, Oct 3, 2017 at 12:28 AM, Ben Campbell <ben@nostrum.com> wrote:
>> Hi,
>> 
>> This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I would like to resolve at least the substantive comments prior to IETF LC.
>> 
>> Thanks!
>> 
>> Ben.
>> —————————
>> 
>> Substantive:
>> 
>> - General: If I understand correctly, this document lists and describes XR metrics that a WebRTC application might choose to negotiate and expose via the WebRTC API. It does not define any new XR metrics. That brings up two questions:
>> 
>> 1) Why is this specific to WebRTC? For the most parts, all the arguments here would apply to any sort of RTP endpoint.
>> 
> 
> This draft was created initially as an input to RTCWEB and later as an
> input to WebRTC Statistics API. Hence the use of WebRTC in the title
> and the introduction. We could add to the introduction: "In general,
> the metrics listed in this document can be exposed by any real-time
> communications endpoint." Or something along those lines
> 
> 
>> 2) Why is this standards track? This sort of material is typically informational, or occasionally a BCP. There are a small number of 2119 keywords, but I’m not sure they are needed or appropriate. (Specifics below.) My initial instinct is that this should be informational.
>> 
> 
> The reason until recently (perhaps still exist) is that this was an
> input from the IETF to W3C, I am not sure if there was an official
> liaison statements sent to them? nonetheless, the W3C Statistics API
> document needed a specification needed (a specification in any
> standards body would suffice) for metrics to be accepted into the W3C
> document.

To make sure I understand:That API document will reference this one? Do they require that specification to be standards track?

Was there discussion in xrblock about standards track verses BCP?

> 
> In practice, it helped that I was the co-author of both the IETF and
> W3C document and hence was able to make the case for the metrics in
> W3C.

Is that case already made? That is, does the need for a published RFC still exist?

> 
>> - General: This needs an IANA considerations section, even if it just contains the statement to the effect of “This document makes no requests for IANA”.
>> 
>> - 1, last paragraph: “ The document also creates a registry containing identifiers from the metrics reported in the RTCP Sender, Receiver, and Extended Reports.”
> 
> This should be removed. Thanks for pointing this out.
> 
>> 
>> It doesn’t actually do that. I understand from the shepherd’s report that this was intentionally removed.
>> 
>> — “All identifiers proposed in this document are RECOMMENDED to be implemented by an endpoint.  An endpoint MAY choose not to expose an identifier if it does not implement the corresponding RTCP Report. “
>> 
>> Does the “RECOMMENDED” apply to all endpoints or just WebRTC endpoints? If the latter, doesn’t that requirement belong (or need to update) some requirement from an RTCWEB (maybe rtp-usage) document, or even the API itself? (I suspect that this draft does not have standing to state this normatively.) What is meant by “MAY choose not to expose”. Is that talking about via the WebRTC API? If so, isn’t that up to the API definition? (That is, it shouldn’t be normative here.)
>> 
> 
> It should be applicable to any endpoint, although it is pertinent or
> timely for the WebRTC endpoint to expose these.
> The purpose of the statement was to highlight that if the
> middlebox/endpoint sends RTCP XRs then it should expose the data
> available via the API.
> 
> I am open to changing to a non-normative lower case guidance if that
> would be more suitable, although if this document is making
> recommendations to the W3C document -- wouldn't we keep the uppercase?

In general, 2119 keywords are better used when we are talking about interoperability issues in a protocol. I suggest switching to lower case unless there’s some reason to believe the W3C won’t pay attention unless it’s in all-caps. (I don’t think that’s usually the case.)

> 
> co-authors, what do you think?
> 
>> -2: If this document keeps the normative keywords, please use the updated boilerplate from RFC 8174. (I note that there are at least some uncapitalized “may”s that do not seem normative.)
>> 
> 
> depending on the above, we should fix these to uppercase.
> 
>> - 5.2.2, last paragraph: “The following metrics can also be considered…”
>> Be considered by whom? Implementors? API designers?
>> 
> 
> Both.
> 
>> - 6, first paragraph: “In practice the application MUST be able to query the statistic identifiers on both an incoming (remote) and outgoing (local) media stream.”
>> What does the MUST requirement apply to? The WebRTC API? This seems more like a statement of fact that “ the application needs to be able…”
>> 
> 
> correct, "needs to" is better.
> 
>> -8: "Therefore encryption procedures, such as the ones suggested for a Secure RTCP (SRTCP), need to be used.”
>> The text should describe the reasons encryption is needed. Also, is this a new normative requirement, or a restatement of an existing requirement?
>> 
> 
> I believe it is restatement alluding to the guidance in the
> corresponding RTCP XRs.
> 
>> Editorial:
>> 
>> -1, paragraph 1:" If sufficient information (metrics or statistics) are provided to the applications, it can attempt to improve the media quality. “
>> 
>> s / “are provided” / “is provided”
>> s / applications / application
>> 
>> -3, 2nd paragraph: It would help to clarify that the references to section 5 and 6 are to this document, not RFC 3611.
>> 
>> — 3rd paragraph: "At the moment…”
>> Please clarify that is at the time of writing (not reading)
>> 
>> -5, 2nd paragraph: “ Application impact metrics mainly collect the information in the viewpoint of application … “
>> s / in / from
>> Also, please expand “FEC” on first mention.
>> 
>> - 5.1.1, first paragraph: “ Duplicate packets may be a result of network delays, which causes the sender to retransmit the original packets.”
>> s/ :delays, which” / “delays that”
>> 
>> — last paragraph: "For those RTCWEB services with jitter buffer…”
>> s/buffer/buffers
>> 
>> - 5.1.2, first paragraph:
>> First sentence is a comma splice.
>> — “some transitory nature of the impairments”
>> Should this say “the transitory nature of some impairments”?
>> — “ Distributed burst provides a higher subjective quality”
>> “Burst distribution …”?
>> — Last paragraph: “ Hence, if WebRTC application”
>> Missing article (or should application be plural?)
>> 
>> - 5.2.1, title: Should “Discard” be “Discarded”?
>> 
>> -7: Please clarify that this is at the time of this writing.
>> 
> 
> Thanks for your feedback and suggestions, we will fix these.
> 
> Regards,
> Varun
> 
> 
> -- 
> Founder, CEO, callstats.io
> http://www.callstats.io
> 
> Interested in networking, media quality, and diagnostics.
> We are hiring!: www.callstats.io/jobs/