Re: [codec] draft test and processing plan for the IETF Codec

Jean-Marc Valin <> Thu, 14 April 2011 02:04 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 7C081E0781 for <>; Wed, 13 Apr 2011 19:04:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id qq9N-vLPMwuk for <>; Wed, 13 Apr 2011 19:04:19 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id AAEA1E05F5 for <>; Wed, 13 Apr 2011 19:04:19 -0700 (PDT)
MIME-version: 1.0
Content-transfer-encoding: 7bit
Content-type: text/plain; charset="ISO-8859-1"; format="flowed"
Received: from [] ([]) by (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTP id <> for; Wed, 13 Apr 2011 22:03:44 -0400 (EDT)
Message-id: <>
Date: Wed, 13 Apr 2011 22:04:08 -0400
From: Jean-Marc Valin <>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20110223 Thunderbird/3.1.8
To: Anisse Taleb <>
References: <> <> <>
In-reply-to: <>
Cc: "" <>
Subject: Re: [codec] draft test and processing plan for the IETF Codec
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Codec WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 14 Apr 2011 02:04:20 -0000

Hi Anisse,

Thanks for the answers. This clarifies a few things, though it makes 
other aspects more puzzling. From what I understand now, what you're 
proposing is that before Opus can be published, we need to verify that 
Opus 8 kb/s be better than Speex at 8 kb/s, no worse than iLBC, that 
Opus 12 kb/s be better than Speex at 12 kb/s, no worse than ilbc, no 
worse than AMR-NB at 12.2 kb/s, no worse than EVRC-NW at 8.3 kb/s, that 
Opus WB 12 kb/s be better than Speex at 12 kb/s, better than G.722 at 56 
kb/s, and so on.

In total, I count 27 codecs/bitrates combinations (excluding GSM-FR) we 
must be better than or no worse than. Considering the three proposed 
packet loss rates and two rate management options (CBR/VBR), we get a 
total of 162 conditions (75 BT and 87 NWT) that need to be met, 
involving 11 reference codecs. As a comparison, I looked at the example 
test plans that Paul sent. One has 4 reference codecs and the other has 
3. It's not clear from those documents how many BT and NWT comparisons 
are involved.



On 11-04-13 07:56 PM, Anisse Taleb wrote:
> Dear Jean Marc.
> Thank you for your valuable comments.
>> 1) How much is "very low delay" and how much is "medium delay"?
> This is based on section [5.1. Operating space] of the requirements document
> "medium delay" (20-30ms), while a few require a "very low delay" (<  10 ms). Would you like to contribute some precise delay operating points for Opus?
>> 2) What do "NWT" and "BT" mean?
> Sorry for this, I should have defined these abbreviations,
> NWT = Not Worse Than.
> BT = Better Than.
>> 3) What's the difference between the "Reference Codecs" column and the
>> "Requirements" column?
> I am not entirely happy with how this table looks like. Essentially, the first column is the collection of codecs, while the requirements column details the actual requirements. I will group these in a next version.
>> 4) How do you define the rates that do not exist in some codecs? For
>> example, I see a reference to 12 and 16 kb/s for Speex, which has no such
>> rates, and to 8, 12, 16 kb/s for iLBC, which does not have these rates
>> either.
> Thank you for pointing this out. If the rates are not supported, I would suggest to use closest bitrates.
> In this initial version, I didn't check if all codecs could be operated at the bitrates being proposed.
> Would you like to suggest on which rates ilbc and speex operated?
> Kind regards,
> /Anisse
> _______________________________________________
> codec mailing list