Re: [codec] A concrete proposal for requirements and testing

[Jean-Marc Valin <[e-mail]>]

Hi,

Sorry, we should have made this clearer that when we mean "better" we  
mean quality vs bitrate coding efficiency. Otherwise nobody can ever  
beat the "16-bit PCM" codec (except maybe the 24-bit PCM codec) ;-)

Or are you suggesting that we compare G.722 to Opus running at 64  
kb/s? In that case should we be running Opus at its optimal point,  
which would probably be fullband stereo?

Cheers,

	Jean-Marc


Roman Shpount <[e-mail]> a écrit :

> I might be missing something, but how is it that G.722 is clearly
> out-performed by G.722.1? As far as I know G.722 at 64 KB has higher quality
> then wideband G.722.1 at any rate. Obviously, G.722 has higher bandwidth
> requirements, but has higher quality and lower bandwidth. Outperforming
> G.722 at 16KHz for all languages, and especially for multiple simultaneous
> speakers or non-speech sources is not an easy thing and should be tested
> against.
> _____________
> Roman Shpount
>
>
> On Thu, Mar 31, 2011 at 10:53 AM, Jean-Marc Valin <
> [e-mail]> wrote:
>
>>  Hi,
>>
>> Following the meeting and post-meeting discussions about requirements and
>> testing, we would like to make the following proposal which addresses the
>> opposing views which prevented consensus in the meeting today.
>>
>> First, we propose to remove the following codecs from the requirements:
>>
>> - GSM-FR, based on consensus from the list
>> - G.722, based on being clearly out-performed by G.722.1
>> - Speex-UWB, based on the fact that the author himself does not recommend
>> it being used :-)
>>
>> We can keep the other reference codecs as minimum quality requirement and
>> include being no worse than AMR-NB and AMR-WB as "objectives" that are "nice
>> to have", but not hard requirements.
>>
>> From there and based on the listening tests presented by Jan Skoglund
>> today, let's see what we can already conclude and what still needs more
>> testing:
>>
>> 1) The narrowband test showed that Opus had higher quality than Speex at 11
>> kb/s. Does anyone disagree that this is sufficient to meet the Sec 4.2
>> requirement of out-performing Speex in narrowband mode?
>>
>> 2) The narrowband test showed that Opus had higher quality at 11 kb/s than
>> iLBC at 15 kb/s. Does anyone disagree that this is sufficient to meet the
>> Sec 4.2 requirement of out-performing iLBC.
>>
>> 3) There have been no formal comparison with AMR-NB yet. What do you think
>> would be sufficient to assess the quality of Opus compared to AMR-NB?
>>
>> 4) The wideband test showed that Opus at 19.85 kb/s had higher quality than
>> Speex-WB at 24 kb/s. Does anyone disagree that this is sufficient to meet
>> the Sec 4.2 requirement of out-performing Speex in wideband mode?
>>
>> 5) The wideband test showed that Opus at 19.85 kb/s had higher quality than
>> G.722.1 at 24 kb/s. Does anyone disagree that this is sufficient to meet the
>> Sec 4.2 requirement of out-performing G.722.1?
>>
>> 6) The wideband test showed that Opus at 19.85 kb/s had higher quality than
>> AMR-WB at 19.85 kb/s. Does anyone disagree that this is sufficient to
>> concluded that the proposed "nice to have" objective of "no worse than
>> AMR-WB" is met?
>>
>> 7) The fullband test showed that Opus at 32 kb/s had higher quality than
>> G.719 at 32 kb/s. Does anyone disagree that this is sufficient to meet the
>> Sec 4.2 requirement of out-performing G.722.1C, considering that G.719 has
>> already been shown to out-perform G.722.1C
>>
>> If you disagree with any of the points above -- as may very well be the
>> case -- please do provide a concrete test proposal that would be sufficient
>> to convince you.
>>
>> Cheers,
>>
>>     Jean-Marc and Koen
>>
>> _______________________________________________
>> codec mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/codec
>>
>>
>