Re: [codec] Testing: A Novel Proposal

Anisse Taleb <> Tue, 19 April 2011 10:38 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id AA644E06D9 for <>; Tue, 19 Apr 2011 03:38:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -6.45
X-Spam-Status: No, score=-6.45 tagged_above=-999 required=5 tests=[AWL=0.149, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id kXqMANLxT17I for <>; Tue, 19 Apr 2011 03:38:36 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 0CADEE0680 for <>; Tue, 19 Apr 2011 03:38:36 -0700 (PDT)
Received: from (lhrga02-in []) by (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <> for; Tue, 19 Apr 2011 11:38:30 +0100 (BST)
Received: from ([]) by (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPS id <> for; Tue, 19 Apr 2011 11:38:29 +0100 (BST)
Received: from ( by ( with Microsoft SMTP Server (TLS) id; Tue, 19 Apr 2011 11:38:23 +0100
Received: from ([fe80::f93f:958b:5b06:4f36]) by ([::1]) with mapi id 14.01.0270.001; Tue, 19 Apr 2011 11:38:28 +0100
Date: Tue, 19 Apr 2011 10:38:28 +0000
From: Anisse Taleb <>
In-reply-to: <>
X-Originating-IP: []
To: Monty Montgomery <>
Message-id: <>
MIME-version: 1.0
Content-type: text/plain; charset="us-ascii"
Content-language: en-US
Content-transfer-encoding: 7bit
Accept-Language: en-GB, en-US
Thread-topic: [codec] Testing: A Novel Proposal
Thread-index: AQHL/mY5+1+5HZckN0mjUmLzLzCEJpRk9x/A
References: <>
Cc: "" <>
Subject: Re: [codec] Testing: A Novel Proposal
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Codec WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 19 Apr 2011 10:38:36 -0000

Dear Monty,
If you refer to the draft proposal, we are in a presence of a multimode multi-bandwidth codec, the per-bandwidth tests proposed are small in comparison to other tests for other codecs, however putting all these together makes the test quite large. For instance, ITU-T codecs usually address a single bandwidth.

It was never the intention to make this an impossible test/task neither was the intention to delay the publication of this codec. I am disappointed that some of the codec proponents and supporters are defensive and quite dismissive. This is an opportunity for everyone to show and demonstrate the quality of the codec in a hopefully well designed test that could be reproduced at will and hence not be easily dismissed or argued against.

As I repeatedly said this was an initial proposal and I value the technical comments and feedback by Jean-Marc, Roman, Stephen, Koen and others in and outside this mailing list. I am working on an update of that test plan and is hopeful to send a version soon. Please let me know if you have any comments such that I can accommodate your point of view too, even if you have little horns :-).

When it comes to your proposal,

"If it isn't good enough... Prove it."

I do not think I have anything to say about that except perhaps to remind you of your own statements 2 years ago, in response to Roni, 

"a lot of characterization, testing, and documentation needs to be done. We're aware of that and we have a lot of work to do."

Kind regards,