Re: [codec] draft test and processing plan for the IETF Codec

Hi Anisse,

Thanks for the answers. This clarifies a few things, though it makes 
other aspects more puzzling. From what I understand now, what you're 
proposing is that before Opus can be published, we need to verify that 
Opus 8 kb/s be better than Speex at 8 kb/s, no worse than iLBC, that 
Opus 12 kb/s be better than Speex at 12 kb/s, no worse than ilbc, no 
worse than AMR-NB at 12.2 kb/s, no worse than EVRC-NW at 8.3 kb/s, that 
Opus WB 12 kb/s be better than Speex at 12 kb/s, better than G.722 at 56 
kb/s, and so on.

In total, I count 27 codecs/bitrates combinations (excluding GSM-FR) we 
must be better than or no worse than. Considering the three proposed 
packet loss rates and two rate management options (CBR/VBR), we get a 
total of 162 conditions (75 BT and 87 NWT) that need to be met, 
involving 11 reference codecs. As a comparison, I looked at the example 
test plans that Paul sent. One has 4 reference codecs and the other has 
3. It's not clear from those documents how many BT and NWT comparisons 
are involved.

Cheers,

    Jean-Marc

On 11-04-13 07:56 PM, Anisse Taleb wrote:
> Dear Jean Marc.
>
> Thank you for your valuable comments.
>
>> 1) How much is "very low delay" and how much is "medium delay"?
>
> This is based on section [5.1. Operating space] of the requirements document http://www.ietf.org/id/draft-ietf-codec-requirements-02.txt.
> "medium delay" (20-30ms), while a few require a "very low delay" (<  10 ms). Would you like to contribute some precise delay operating points for Opus?
>
>> 2) What do "NWT" and "BT" mean?
> Sorry for this, I should have defined these abbreviations,
>
> NWT = Not Worse Than.
> BT = Better Than.
>
>> 3) What's the difference between the "Reference Codecs" column and the
>> "Requirements" column?
>
> I am not entirely happy with how this table looks like. Essentially, the first column is the collection of codecs, while the requirements column details the actual requirements. I will group these in a next version.
>
>> 4) How do you define the rates that do not exist in some codecs? For
>> example, I see a reference to 12 and 16 kb/s for Speex, which has no such
>> rates, and to 8, 12, 16 kb/s for iLBC, which does not have these rates
>> either.
>
> Thank you for pointing this out. If the rates are not supported, I would suggest to use closest bitrates.
> In this initial version, I didn't check if all codecs could be operated at the bitrates being proposed.
> Would you like to suggest on which rates ilbc and speex operated?
>
> Kind regards,
> /Anisse
>
> _______________________________________________
> codec mailing list
> codec@ietf.org
> https://www.ietf.org/mailman/listinfo/codec
>
>