Re: [rmcat] Roman Danyliw's Discuss on draft-ietf-rmcat-eval-criteria-12: (with DISCUSS and COMMENT)

[Joerg Ott <[e-mail]>]

Hi Roman,

thanks much, since the ref is already included in -13 we should be
happily done.

Best,
Jörg

On 19.03.20 00:18, Roman Danyliw wrote:
> Hi Joerg!
> 
> Thanks for the update.  -13 addresses my DISCUSS points.  One non-blocking comment on a COMMENT inline ...
> 
> Thanks,
> Roman
> 
> -----Original Message-----
> From: iesg <[e-mail]> On Behalf Of Joerg Ott
> Sent: Monday, March 9, 2020 3:56 PM
> To: Roman Danyliw <[e-mail]>; The IESG <[e-mail]>
> Cc: [e-mail]; [e-mail]; [e-mail]; Martin Stiemerling <[e-mail]>; [e-mail]; Colin Perkins <[e-mail]>
> Subject: Re: Roman Danyliw's Discuss on draft-ietf-rmcat-eval-criteria-12: (with DISCUSS and COMMENT)
> 
> Dear Roman,
> 
> thanks much for your comments.  I just posted an updated draft (-13) to address those.  See below.
> 
>> ----------------------------------------------------------------------
>> DISCUSS:
>> ----------------------------------------------------------------------
>>
>> Section 5.2.  Per “Sample video test sequences are available at:
>> [xiph-seq] and [HEVC-seq].  The following two video streams are the
>> recommended minimum for
>> testing: Foreman and FourPeople.”, these test sequences seems underspecified.
>>
>> ** Is the “recommended” here intended to be normative?  There is no
>> RFC2119 boiler plate in this document to guide the parsing of the text.
> 
> Recommended is recommended as per English language and it not used as a verb but as an adjective, indicating a suggestion.
> 
>> ** From the text, there wasn’t much precision in where to find these
>> recommended videos (Foreman and FourPeople).  At the url pointed to by
>> [HEVC-seq], I found the filenames “FourPeople_1280x720_60.yuv” and
>> “foreman15_4000.yuv”, is that them?
> 
> This is fixed in the rev now, also the link that disappeared is removed.
> 
>> ** Is it expected for
>> http://www.netlab.tkk.fi/~varun/test_sequences/foreman15_4000.yuv to
>> be 0 bytes?  I tried on 03/03/2020 at ~0950 EST
> 
> The broken link is gone now.  Both videos can be found in a single place now.  It is clearer specified where to find them.
> 
>> ** Give that that one of the recommended urls doesn’t work even before
>> this draft is published, I have great reservation with keeping a
>> normative “recommended” to such external repositories.  However,
>> providing pointers to repositories of “sample video test sequences” makes sense to me and is helpful.
> 
> Should be addressed by now.
> 
>> ----------------------------------------------------------------------
>> COMMENT:
>> ----------------------------------------------------------------------
>>
>> ** Section 3.1.  The purpose of this log isn’t clear.  What is the
>> relationship of it to the metrics described in the previous section?
>> Where does it fit into the measurement workflow?  Is this constructed
>> on a per packet capture file basis?
> 
> Added a clarifying sentence:
> 
> These data should suffice to compute the media-encoding independent metrics described above.  Use of a common log will allow simplified post-processing and analysis across different implementations.
> 
>> ** References:
>> -- Section 3.  Consider adding a citation for tcpdump and wireshark
> 
> done
> 
>> -- Section 4.4.  Consider adding a citation for the “Bilbert-Elliot”
>> model
> 
> We got this comment also from Mirja, to which I replied:
> 
> I could include a reference here for Gilbert-Elliot such as:
> http://www.ohohlfeld.com/paper/hasslinger_hohlfeld-mmb_2008.pdf
> 
> [Roman]  Since it would be informative, I don't think it would hurt to add this citation.
> 
> However, this not mandatory text and the second part of the sentence is even less specific and does (deliberately) not give a specific example.  We don't want to create any prejudice here beyond random losses.  The Internet changes, after all.
> 
> In order to avoid this being a showstopper here (we don't have enough hour tonight for another rev round, I am adding this in now.  Please provide feedback if this is really needed.  I could still remove it in two weeks when I-D submission reopens.
> 
>> ** Editorial:
>> -- Section 3.  Recommend explicitly spelling out PCAP as packet capture.
> 
> This is spelled out in brackets; since this also refers to a commonly used file format, I would keep it as is.
> 
>> -- Section 4.5. s/is is/is/
> 
> Fixed.
> 
> Thanks much,
> Jörg
> 
>