[video-codec] Mirja Kühlewind's Abstain on draft-ietf-netvc-testing-08: (with COMMENT)

[Mirja Kühlewind via Datatracker <[e-mail]>]

Mirja Kühlewind has entered the following ballot position for
draft-ietf-netvc-testing-08: Abstain

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-netvc-testing/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Update: This document has no security considerations section, while having this
section is required.

This document reads more like a user manual of the Daala tools repository
(together with the test sequences). I wonder why this is not simply achieved
within the repo? What’s the benefit of having this in an RFC? Especially I’m
worried that this document is basically useless in case the repo and test
sequences disappear, and are therefore not available anymore in future, or
change significantly. I understand that this is referenced by OAM and therefore
publication is desired, however, I don't think that makes my concern about the
standalone usefulness of this document invalid. If you really want to publish
in the RFC series, I would recommend to reduce the dependencies to these repos
and try to make this document more useful as a standalone test description
(which would probably mean removing most of section 4 and adding some
additional information to other parts).

Also, the shepherd write-up seems to indicate that this document has an IPR
disclosure that was filed after WG last call. Is the wg aware of this? Has this
been discussed in the wg?

Other more concrete comments:
1) Quick question on 2.1: Is the tester supposed to view one image after the
other or both at the same time? And if one ofter the other, could the order
impact the results (and should maybe be randomly chosen therfore)?

2) Sec 2.3: Would it make sense to provide a (normative) reference to MOS? Or
is that supposed to be so well know that that is not even necessary?

3) Sec 3.1: maybe spell out PSNR on first occurrence. And would it make sense
to provide a reference for PSNR?

4) Sec 3.2: “ The weights used by the dump_pnsrhvs.c tool in
   the Daala repository have been found to be the best match to real MOS
   scores.”
Maybe document these weights in this document as well…?

5) Sec 5.3: Maybe spell out CQP at first occurrence