[rtcweb] Video codec quality evaluations (Re: Agenda time request for draft-dbenham-webrtcvideomti)

Harald Alvestrand <harald@alvestrand.no> Thu, 28 February 2013 15:31 UTC

Return-Path: <harald@alvestrand.no>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9DC7021F89BF for <rtcweb@ietfa.amsl.com>; Thu, 28 Feb 2013 07:31:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.299
X-Spam-Level:
X-Spam-Status: No, score=-110.299 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HDXtBEzOyUdj for <rtcweb@ietfa.amsl.com>; Thu, 28 Feb 2013 07:31:24 -0800 (PST)
Received: from eikenes.alvestrand.no (eikenes.alvestrand.no [158.38.152.233]) by ietfa.amsl.com (Postfix) with ESMTP id D0B2B21F88B4 for <rtcweb@ietf.org>; Thu, 28 Feb 2013 07:31:18 -0800 (PST)
Received: from localhost (localhost [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 4DF6A39E0E6 for <rtcweb@ietf.org>; Thu, 28 Feb 2013 16:31:14 +0100 (CET)
X-Virus-Scanned: Debian amavisd-new at eikenes.alvestrand.no
Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MPzI7XFJmszA for <rtcweb@ietf.org>; Thu, 28 Feb 2013 16:31:13 +0100 (CET)
Received: from [193.157.215.103] (unknown [193.157.215.103]) by eikenes.alvestrand.no (Postfix) with ESMTPSA id 23DBF39E056 for <rtcweb@ietf.org>; Thu, 28 Feb 2013 16:31:13 +0100 (CET)
Message-ID: <512F7840.6070407@alvestrand.no>
Date: Thu, 28 Feb 2013 16:31:12 +0100
From: Harald Alvestrand <harald@alvestrand.no>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130106 Thunderbird/17.0.2
MIME-Version: 1.0
To: rtcweb@ietf.org
References: <CD5381E5.95C4C%stewe@stewe.org>
In-Reply-To: <CD5381E5.95C4C%stewe@stewe.org>
Content-Type: multipart/mixed; boundary="------------070502000704030806080907"
Subject: [rtcweb] Video codec quality evaluations (Re: Agenda time request for draft-dbenham-webrtcvideomti)
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 28 Feb 2013 15:31:24 -0000

Sigh. I thought that after the drubbing draft-dbenham-webrtcvideomti got 
at the previous meeting, the authors would have either improved the 
attempts at quality evaluation or removed them.

It seemed to me that there was rough consensus on the mailing list 
earlier that the quality of the two codecs was close enough that this 
was not going to convince anyone who had already taken a strong position 
based on the IPR issues.

But if we are going to play the video codec quality evaluation game, I 
also have something I want to have on file here.

Google has submitted VP8 as a candidate for standardization in ISO/IEC 
JTC1 SC29 WG11 (better known as MPEG). As part of that submission, we 
submitted a quantitative evaluation of VP8's quality compared to the 
then-current "IVC Test Model", which also included numbers compared to 
the AVC Baseline "anchors" that were part of the project description for 
the IVC effort.

This was contributed to MPEG's January meeting in Geneva; the decision 
at that meeting was to continue the evaluation effort, with new data 
being made available before the next meeting in April.

I'm enclosing the report with the test results; the tests were not done 
by Google; the scripts are available if anyone wants to run them for 
themselves.

                        Harald