Re: [rmcat] Roman Danyliw's Discuss on draft-ietf-rmcat-eval-criteria-12: (with DISCUSS and COMMENT)

Joerg Ott <> Mon, 09 March 2020 19:56 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C7ECB3A1633; Mon, 9 Mar 2020 12:56:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id jQuoenA_czkh; Mon, 9 Mar 2020 12:56:09 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 65C133A1643; Mon, 9 Mar 2020 12:56:08 -0700 (PDT)
Received: by (Postfix, from userid 107) id 374CC1C0804; Mon, 9 Mar 2020 20:56:06 +0100 (CET)
Received: (Authenticated sender: ott) by (Postfix) with ESMTPSA id 4F5C61C07FF; Mon, 9 Mar 2020 20:56:03 +0100 (CET) (Extended-Queue-bit
To: Roman Danyliw <>, The IESG <>
Cc:,,, Martin Stiemerling <>, Colin Perkins <>,
References: <>
From: Joerg Ott <>
Message-ID: <>
Date: Mon, 9 Mar 2020 20:56:00 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.4.2
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: <>
Subject: Re: [rmcat] Roman Danyliw's Discuss on draft-ietf-rmcat-eval-criteria-12: (with DISCUSS and COMMENT)
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 09 Mar 2020 19:56:12 -0000

Dear Roman,

thanks much for your comments.  I just posted an updated draft (-13) to
address those.  See below.

> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> Section 5.2.  Per “Sample video test sequences are available at: [xiph-seq] and
> [HEVC-seq].  The following two video streams are the recommended minimum for
> testing: Foreman and FourPeople.”, these test sequences seems underspecified.
> ** Is the “recommended” here intended to be normative?  There is no RFC2119
> boiler plate in this document to guide the parsing of the text.

Recommended is recommended as per English language and it not used as a
verb but as an adjective, indicating a suggestion.

> ** From the text, there wasn’t much precision in where to find these
> recommended videos (Foreman and FourPeople).  At the url pointed to by
> [HEVC-seq], I found the filenames “FourPeople_1280x720_60.yuv” and
> “foreman15_4000.yuv”, is that them?

This is fixed in the rev now, also the link that disappeared is removed.

> ** Is it expected for
> to be 0
> bytes?  I tried on 03/03/2020 at ~0950 EST

The broken link is gone now.  Both videos can be found in a single place
now.  It is clearer specified where to find them.

> ** Give that that one of the recommended urls doesn’t work even before this
> draft is published, I have great reservation with keeping a normative
> “recommended” to such external repositories.  However, providing pointers to
> repositories of “sample video test sequences” makes sense to me and is helpful.

Should be addressed by now.

> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ** Section 3.1.  The purpose of this log isn’t clear.  What is the relationship
> of it to the metrics described in the previous section?  Where does it fit into
> the measurement workflow?  Is this constructed on a per packet capture file
> basis?

Added a clarifying sentence:

These data should suffice to compute the media-encoding independent
metrics described above.  Use of a common log will allow simplified
post-processing and analysis across different implementations.

> ** References:
> -- Section 3.  Consider adding a citation for tcpdump and wireshark


> -- Section 4.4.  Consider adding a citation for the “Bilbert-Elliot” model

We got this comment also from Mirja, to which I replied:

I could include a reference here for Gilbert-Elliot such as:

However, this not mandatory text and the second part of the sentence
is even less specific and does (deliberately) not give a specific
example.  We don't want to create any prejudice here beyond random
losses.  The Internet changes, after all.

In order to avoid this being a showstopper here (we don't have enough
hour tonight for another rev round, I am adding this in now.  Please
provide feedback if this is really needed.  I could still remove it
in two weeks when I-D submission reopens.

> ** Editorial:
> -- Section 3.  Recommend explicitly spelling out PCAP as packet capture.

This is spelled out in brackets; since this also refers to a commonly
used file format, I would keep it as is.

> -- Section 4.5. s/is is/is/


Thanks much,