Re: [rtcweb] VP8 vs H.264 - the core issue

[Harald Alvestrand <[e-mail]>]

On 10/24/2013 03:26 PM, Bo Burman wrote:
>
> 1) We do agree that H.264 Constrained Baseline and VP8 are comparable
> in terms of video quality. But do not forget that Constrained
> Baseline's twin sister H.264 Constrained High outperforms VP8 by a
> huge margin. This is also relevant.
>

I actually haven't seen a test I can quote on that - at least not with
parameters that make sense in a videoconferencing setting (use of rate
control, no B-frames and CBR mode).

There was a series of tests (both subjective and objective) done on both
High Profile and VP8 as part of the MPEG evaluation of VP8 as a response
to the last Call for Proposals, but after the discussions we had in
Incheon, MPEG decided to not publish the results of this viewing as a
"comparision", so I won't call it that.

>  
>
> 2) The "back-and forth of numbers" seems to refer to Ericsson's work
> where we tried to make a fair comparison to evaluate the extraordinary
> claims from Google that VP8 is 70 or 40 percent better than x264. We
> found serious issues with the way Google performed the test, maybe the
> most striking were the use of padding (+8% for x264) and excessive
> number of threads (+10% to +40% for x264) to add overhead to x264.
> That Google managed to remember the threading parameter only when it
> helped
>
> VP8 (the speed test) is also quite remarkable.
>

I was also astonished to learn that x264's default value for "threads"
gave a 10-40% performance penalty over "threads=1"!

On a speed test, we tried to do a bit of "like against like" - the VP8
encoder is by default single-threaded, so it seemed only reasonable to
run the other encoder single-threaded too. I haven't pursued the speed
tests since then; they are quite a bit harder to get consistent results
from than tests that have a deterministic result.

Yes, we should have tested each piece of the advice we got from the x264
developers on how to run x264 in something close to "constant bitrate
mode", instead of just dumping all the flags on our command line
uncritically. That was a mistake, and led to the nal-unit parameter,
whch we should never have used.

We have published our tests, exactly so that they can be evaluated.
We want to do a good job of incorporating feedback - but we will keep on
insisting that the tests should have some relationship to a realistic
WebRTC scenario.

In our opinion, that means working in one pass and using rate control.

>  
>
> 3) Regarding IPR, this is a difficult topic, but we're not at all
> convinced that VP8 is royalty free. For example, there is an IETF IPR
> disclosure (https://datatracker.ietf.org/ipr/2035/) where the IPR
> holder seems unwilling to license (on any terms), and
> http://www.fosspatents.com/2013/06/german-vp8-infringement-cases-show.html
> and
> http://www.fosspatents.com/2013/06/itc-institutes-investigation-of-nokias.html
> show that there are in fact ongoing litigations regarding VP8 - and
> this is only skimming the surface of what is available in public space.
>

If you have more specific information, feel free to share it.
The continued use of "there's more information out there" is not helpful
to anyone. Either you have specific information to bring, or you don't.

And since you bring a June mention of the German case, I should mention
this August note from the same case, which I also quoted in the VP8
internet-draft:

http://blog.webmproject.org/2013/08/good-news-from-germany.html

I refuse to comment on fosspatents.com's commentary.