[rmcat] Roman Danyliw's Discuss on draft-ietf-rmcat-eval-criteria-12: (with DISCUSS and COMMENT)

Roman Danyliw via Datatracker <noreply@ietf.org> Tue, 03 March 2020 14:58 UTC

Return-Path: <noreply@ietf.org>
X-Original-To: rmcat@ietf.org
Delivered-To: rmcat@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 191463A21FE; Tue, 3 Mar 2020 06:58:42 -0800 (PST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
From: Roman Danyliw via Datatracker <noreply@ietf.org>
To: "The IESG" <iesg@ietf.org>
Cc: draft-ietf-rmcat-eval-criteria@ietf.org, rmcat-chairs@ietf.org, rmcat@ietf.org, Martin Stiemerling <mls.ietf@gmail.com>, Colin Perkins <csp@csperkins.org>, varun.singh@iki.fi, csp@csperkins.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.119.0
Auto-Submitted: auto-generated
Precedence: bulk
Reply-To: Roman Danyliw <rdd@cert.org>
Message-ID: <158324752207.7683.15663429456876412351@ietfa.amsl.com>
Date: Tue, 03 Mar 2020 06:58:42 -0800
Archived-At: <https://mailarchive.ietf.org/arch/msg/rmcat/Sd4VsIcDFagpxB2VKyk8XTVfY0w>
Subject: [rmcat] Roman Danyliw's Discuss on draft-ietf-rmcat-eval-criteria-12: (with DISCUSS and COMMENT)
X-BeenThere: rmcat@ietf.org
X-Mailman-Version: 2.1.29
List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." <rmcat.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rmcat>, <mailto:rmcat-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rmcat/>
List-Post: <mailto:rmcat@ietf.org>
List-Help: <mailto:rmcat-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rmcat>, <mailto:rmcat-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Mar 2020 14:58:42 -0000

Roman Danyliw has entered the following ballot position for
draft-ietf-rmcat-eval-criteria-12: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-rmcat-eval-criteria/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Section 5.2.  Per “Sample video test sequences are available at: [xiph-seq] and
[HEVC-seq].  The following two video streams are the recommended minimum for
testing: Foreman and FourPeople.”, these test sequences seems underspecified.

** Is the “recommended” here intended to be normative?  There is no RFC2119
boiler plate in this document to guide the parsing of the text.

** From the text, there wasn’t much precision in where to find these
recommended videos (Foreman and FourPeople).  At the url pointed to by
[HEVC-seq], I found the filenames “FourPeople_1280x720_60.yuv” and
“foreman15_4000.yuv”, is that them?

** Is it expected for
http://www.netlab.tkk.fi/~varun/test_sequences/foreman15_4000.yuv to be 0
bytes?  I tried on 03/03/2020 at ~0950 EST

** Give that that one of the recommended urls doesn’t work even before this
draft is published, I have great reservation with keeping a normative
“recommended” to such external repositories.  However, providing pointers to
repositories of “sample video test sequences” makes sense to me and is helpful.


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

** Section 3.1.  The purpose of this log isn’t clear.  What is the relationship
of it to the metrics described in the previous section?  Where does it fit into
the measurement workflow?  Is this constructed on a per packet capture file
basis?

** References:
-- Section 3.  Consider adding a citation for tcpdump and wireshark

-- Section 4.4.  Consider adding a citation for the “Bilbert-Elliot” model

** Editorial:
-- Section 3.  Recommend explicitly spelling out PCAP as packet capture.

-- Section 4.5. s/is is/is/