[video-codec] Performances, measurement of same (Re: The Can Has Landed)

[Harald Alvestrand <[e-mail]>]

On 03/20/2015 01:41 AM, Thomas Daede wrote:
> On 03/19/2015 05:17 PM, Keith Winstein wrote:
>> But what are the merits of an IETF working group performing this kind of
>> high-risk, high-reward research, versus doing something much more boring
>> like "writing a specification for VP9 to enable interoperable
>> implementations, and then iterating on that technology"?
> If VP9 could be shown to meet our requirements for performance and
> licensing, I would be supportive of that path. However, I don't believe
> that it currently does.
>
> In the testing draft, while I specified methods of testing codecs, I did
> not specify an absolute performance goal. We will need to set one. Opus
> achieved state-of-the-art performance, which was highly beneficial to
> its adoption. VP9 does not achieve this yet.

Just as a matter of curiosity, which state of the art performance is it
that you think VP9 hasn't achieved yet?

I also wonder a bit about how we should treat evaluation of performance
- a traditional video codec development technique has been to develop a
codec that takes hours-per-second to encode a video, and then seek to
optimize it by a factor of a hundred before shipping it. But sometimes,
there's just limits to how much one can achieve.

How should we treat that aspect in evaluation?