Re: [rtcweb] New VP8 vs H.264 tests uploaded

Harald Alvestrand <harald@alvestrand.no> Thu, 04 April 2013 13:30 UTC

Return-Path: <harald@alvestrand.no>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11FBA21F8782 for <rtcweb@ietfa.amsl.com>; Thu, 4 Apr 2013 06:30:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.598
X-Spam-Level:
X-Spam-Status: No, score=-110.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wZUclUblZ378 for <rtcweb@ietfa.amsl.com>; Thu, 4 Apr 2013 06:30:52 -0700 (PDT)
Received: from eikenes.alvestrand.no (eikenes.alvestrand.no [158.38.152.233]) by ietfa.amsl.com (Postfix) with ESMTP id C6A2321F85EB for <rtcweb@ietf.org>; Thu, 4 Apr 2013 06:30:51 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 1C7D239E116 for <rtcweb@ietf.org>; Thu, 4 Apr 2013 15:30:49 +0200 (CEST)
X-Virus-Scanned: Debian amavisd-new at eikenes.alvestrand.no
Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1qTPbAz7e3ch for <rtcweb@ietf.org>; Thu, 4 Apr 2013 15:30:47 +0200 (CEST)
Received: from hta-dell.lul.corp.google.com (62-20-124-50.customer.telia.com [62.20.124.50]) by eikenes.alvestrand.no (Postfix) with ESMTPSA id 9208E39E056 for <rtcweb@ietf.org>; Thu, 4 Apr 2013 15:30:47 +0200 (CEST)
Message-ID: <515D8087.6080409@alvestrand.no>
Date: Thu, 04 Apr 2013 15:30:47 +0200
From: Harald Alvestrand <harald@alvestrand.no>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130308 Thunderbird/17.0.4
MIME-Version: 1.0
To: rtcweb@ietf.org
References: <CAPVCLWbajJNS-DbXS-AJjakwovBKhhpXAmBaR_LYKjCyk7UnYg@mail.gmail.com> <515D3FA1.6050305@gmail.com>
In-Reply-To: <515D3FA1.6050305@gmail.com>
Content-Type: multipart/alternative; boundary="------------020601080504010300060102"
Subject: Re: [rtcweb] New VP8 vs H.264 tests uploaded
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Apr 2013 13:30:53 -0000

On 04/04/2013 10:53 AM, Sergio Garcia Murillo wrote:
> Hi Adrian,
>
> Could you explain how the encoding parametrization is comparable?
>
> x264 --nal-hrd cbr --vbv-maxrate ${rate} --vbv-bufsize ${rate} \
>       --vbv-init 0.8 --bitrate ${rate} --fps ${frame_rate} \
>       --profile baseline --no-scenecut --keyint infinite --preset 
> veryslow \
>       --input-res ${width}x${height} \
>       --tune psnr \
>       -o ./encoded_clips/h264/${clip_stem}_${rate}kbps.mkv ${filename} \
>       2> ./logs/h264/${clip_stem}_${rate}kbps.txt
>
> vs:
>
>  ./bin/vpxenc --lag-in-frames=0 --target-bitrate=${rate} 
> --kf-min-dist=3000 \
>       --kf-max-dist=3000 --cpu-used=0 --fps=${frame_rate}/1 
> --static-thresh=0 \
>       --token-parts=1 --drop-frame=0 --end-usage=cbr --min-q=2 
> --max-q=56 \
>       --undershoot-pct=100 --overshoot-pct=15 --buf-sz=1000 \
>       --buf-initial-sz=800 --buf-optimal-sz=1000 --max-intra-rate=1200 \
>       --resize-allowed=0 --drop-frame=0 --passes=1 --good 
> --noise-sensitivity=0 \
>       -w ${width} -h ${height} ${filename} --codec=vp8 \
>       -o ./encoded_clips/vp8/${clip_stem}_${rate}kbps.webm \
>       &>./logs/vp8/${clip_stem}_${rate}kbps.txt

Both have the same target bitrate and the same resolution, and neither 
generates periodic keyframes.

Apart from that, I think they are comparable by virtue of being the 
parameters that were recommended for this test by people who like this 
particular codec implementation.

Was there any specific parameter or parameter set you were wondering about?


>
> Best regards
> Sergio
>
> El 03/04/2013 18:20, Adrian Grange escribió:
>> We have uploaded a new set of test results comparing VP8 to H.264. 
>> This latest set contains fixes for some of the problems in the 
>> previous set. We would like to extend our thanks to those who made 
>> suggestions as to how we could improve our methodology and encourage 
>> suggestions as to how we can make further improvements.
>>
>> In these tests we run x264 with the "veryslow" preset and VP8 with 
>> the "good, speed 0" setting in an attempt to produce comparable results.
>>
>> An overview of our results is available as follows:
>>
>> - A Quality comparison (psnr): 
>> http://downloads.webmproject.org/ietf_tests/vp8_vs_h264_quality.html
>>
>> - An Encode Speed comparison: 
>> http://downloads.webmproject.org/ietf_tests/vp8_vs_h264_speed.html
>>
>> - A comparison of the aggregate time required to decode all of the 
>> clips in the test: 
>> http://downloads.webmproject.org/ietf_tests/vp8vsh264-decodetime.txt
>>
>> All of our test scripts can either be downloaded from:
>> http://downloads.webmproject.org/ietf_tests/vp8_vs_h264.tar.xz
>> or checked out of our git/gerrit repository:
>> git clone http://git.chromium.org/webm/vpx_codec_comparison.git
>>
>> The file README.txt, contained within, presents details of how to 
>> build and run the tests.
>>
>> The compressed video files--the output from the quality tests--can 
>> also be downloaded:
>>
>> VP8:
>> http://downloads.webmproject.org/ietf_tests/vp8_videos 
>> <http://downloads.webmproject.org/ietf_tests/vp8_videos/>/index.html
>>
>> H.264:
>> http://downloads.webmproject.org/ietf_tests/h264_videos/index.html
>>
>> Adrian Grange
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> rtcweb mailing list
>> rtcweb@ietf.org
>> https://www.ietf.org/mailman/listinfo/rtcweb
>
>
>
> _______________________________________________
> rtcweb mailing list
> rtcweb@ietf.org
> https://www.ietf.org/mailman/listinfo/rtcweb