IETF Logo Mail Archive

Re: [codec] Testing Codec Quality

Michael Knappe <mknappe@juniper.net> Wed, 12 January 2011 09:31 UTC

Return-Path: <mknappe@juniper.net>
X-Original-To: codec@core3.amsl.com
Delivered-To: codec@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 453BF28C11A for <codec@core3.amsl.com>; Wed, 12 Jan 2011 01:31:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.389
X-Spam-Level:
X-Spam-Status: No, score=-106.389 tagged_above=-999 required=5 tests=[AWL=0.210, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M6fQwx4+L+ph for <codec@core3.amsl.com>; Wed, 12 Jan 2011 01:31:55 -0800 (PST)
Received: from exprod7og115.obsmtp.com (exprod7og115.obsmtp.com [64.18.2.217]) by core3.amsl.com (Postfix) with ESMTP id 66C8228C118 for <codec@ietf.org>; Wed, 12 Jan 2011 01:31:55 -0800 (PST)
Received: from source ([66.129.224.36]) (using TLSv1) by exprod7ob115.postini.com ([64.18.6.12]) with SMTP ID DSNKTS11jx3lIFhYDafATrXgF3k7IN1YuG2S@postini.com; Wed, 12 Jan 2011 01:34:14 PST
Received: from EMBX02-HQ.jnpr.net ([fe80::18fe:d666:b43e:f97e]) by P-EMHUB02-HQ.jnpr.net ([fe80::88f9:77fd:dfc:4d51%11]) with mapi; Wed, 12 Jan 2011 01:32:37 -0800
From: Michael Knappe <mknappe@juniper.net>
To: Christian Hoene <hoene@uni-tuebingen.de>, "codec@ietf.org" <codec@ietf.org>
Date: Wed, 12 Jan 2011 01:32:35 -0800
Thread-Topic: [codec] Testing Codec Quality
Thread-Index: AcuS0pQ1qPinbi39R6KAd6QdOO+jfAfaRC0B
Message-ID: <C952B533.224F0%mknappe@juniper.net>
In-Reply-To: <000f01cb92d2$96d04490$c470cdb0$@de>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-Entourage/13.3.0.091002
acceptlanguage: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [codec] Testing Codec Quality
X-BeenThere: codec@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Codec WG <codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/codec>
List-Post: <mailto:codec@ietf.org>
List-Help: <mailto:codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Jan 2011 09:31:57 -0000

Christian - WOW, what a fantastic document!

I encourage everyone to read this through and provide comments back to the
mailing list. In particular, check out section 10.2 which lists detailed
considerations for the proposed semi-formal testing with the 6+ volunteer
'test houses'. One of the issues we've been dealing with in our discussions
is how to divide up testing of even a simplified list of codec configuration
and operational permutations (Christian easily came up with 48 different
test cases) between the test houses. The plan is to have a very short list
of basic codec test cases that every test house repeats, along with a longer
list of permutations that is divided among the test houses.


We do need to add a section outlining the proposed widescale informal
listening tests via a web based mushra app that will be done alongside the
semi-formal testing. Look for that app soon at www.open-codec.com, we're
working on getting that app up and running as we speak.

Thanks again to Christian for all the effort put into this doc.

Cheers, 

Mike




On 12/3/10 2:12 AM, "Christian Hoene" <hoene@uni-tuebingen.de> wrote:

> Hi,
> 
> one important question back to the days where iLBC was submitted to the IETF
> was: how do we know whether iLBC is a technical sound contribution to the
> IETF?
> 
> The draft http://www.ietf.org/id/draft-hoene-codec-quality-00.pdf addresses
> this question in the context of this WG. The first chapters describe well
> established procedures on how to test a codec (and goes a bit beyond that
> where needed). Finally, Chapter 10 suggests a procedure on how to test IIAC
> (or Opus).
> 
> Timely feedback is welcomed as the testing shall start soon.
> 
> With best regards,
> 
>  Christian
> 
> 
> ---------------------------------------------------------------
> Dr.-Ing. Christian Hoene
> Interactive Communication Systems (ICS), University of Tübingen
> Sand 13, 72076 Tübingen, Germany, Phone +49 7071 2970532
> http://www.net.uni-tuebingen.de/
> 
> 
> _______________________________________________
> codec mailing list
> codec@ietf.org
> https://www.ietf.org/mailman/listinfo/codec

v2.23.0 | Report a Bug | By Email