Re: [video-codec] Charter issues from BoF - testing

"Timothy B. Terriberry" <tterribe@xiph.org> Fri, 18 January 2013 14:38 UTC

Return-Path: <tterribe@xiph.org>
X-Original-To: video-codec@ietfa.amsl.com
Delivered-To: video-codec@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6B3FD21F8473 for <video-codec@ietfa.amsl.com>; Fri, 18 Jan 2013 06:38:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.582
X-Spam-Level:
X-Spam-Status: No, score=-2.582 tagged_above=-999 required=5 tests=[AWL=0.095, BAYES_00=-2.599, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nIMmhSePD6EJ for <video-codec@ietfa.amsl.com>; Fri, 18 Jan 2013 06:38:02 -0800 (PST)
Received: from smtp.mozilla.org (mx2.corp.phx1.mozilla.com [63.245.216.70]) by ietfa.amsl.com (Postfix) with ESMTP id 0972221F845E for <video-codec@ietf.org>; Fri, 18 Jan 2013 06:38:02 -0800 (PST)
Received: from [192.168.16.25] (pool-96-241-176-56.washdc.fios.verizon.net [96.241.176.56]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id 62193F2121 for <video-codec@ietf.org>; Fri, 18 Jan 2013 06:38:01 -0800 (PST)
Message-ID: <50F95E46.1040908@xiph.org>
Date: Fri, 18 Jan 2013 06:37:58 -0800
From: "Timothy B. Terriberry" <tterribe@xiph.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120625 SeaMonkey/2.10.1
MIME-Version: 1.0
To: video-codec@ietf.org
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 7bit
Subject: Re: [video-codec] Charter issues from BoF - testing
X-BeenThere: video-codec@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Video codec BoF discussion list <video-codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/video-codec>, <mailto:video-codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/video-codec>
List-Post: <mailto:video-codec@ietf.org>
List-Help: <mailto:video-codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/video-codec>, <mailto:video-codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 18 Jan 2013 14:38:02 -0000

> I'm pretty skeptical about PSNR but I was not really pushing on how we
> did testing here. I was more focused on how we will get any test data
> at all.
>
> Saying no feature goes in without test data in the charter would be a
> good way to address this. I'm sure there are others ways but what I
> want to avoid is a bunch of technical proposal which seem like they
> might be OK idea but actually don't gain much.

That seems like a fine idea, as long as it's worded generally enough 
that "compression performance" is not the only metric that can justify 
adding a feature.