Re: [codec] A concrete proposal for requirements and testing

[Roman Shpount <[e-mail]>]

I would like to see a comparison of G.722 to Opus running to fullband mono
at its optimum point. To be honest, my interests are somewhat skewed towards
16 Khz audio, so comparison of G.722 to Opus at the optimal point for 16KHz
mono audio is also of great interest to me and my company.
_____________
Roman Shpount


2011/3/31 Jean-Marc Valin <[e-mail]>

> Hi,
>
> Sorry, we should have made this clearer that when we mean "better" we mean
> quality vs bitrate coding efficiency. Otherwise nobody can ever beat the
> "16-bit PCM" codec (except maybe the 24-bit PCM codec) ;-)
>
> Or are you suggesting that we compare G.722 to Opus running at 64 kb/s? In
> that case should we be running Opus at its optimal point, which would
> probably be fullband stereo?
>
> Cheers,
>
>        Jean-Marc
>
>
> Roman Shpount <[e-mail]> a écrit :
>
>
>  I might be missing something, but how is it that G.722 is clearly
>> out-performed by G.722.1? As far as I know G.722 at 64 KB has higher
>> quality
>> then wideband G.722.1 at any rate. Obviously, G.722 has higher bandwidth
>> requirements, but has higher quality and lower bandwidth. Outperforming
>> G.722 at 16KHz for all languages, and especially for multiple simultaneous
>> speakers or non-speech sources is not an easy thing and should be tested
>> against.
>> _____________
>> Roman Shpount
>>
>>
>> On Thu, Mar 31, 2011 at 10:53 AM, Jean-Marc Valin <
>> [e-mail]> wrote:
>>
>>   Hi,
>>>
>>> Following the meeting and post-meeting discussions about requirements and
>>> testing, we would like to make the following proposal which addresses the
>>> opposing views which prevented consensus in the meeting today.
>>>
>>> First, we propose to remove the following codecs from the requirements:
>>>
>>> - GSM-FR, based on consensus from the list
>>> - G.722, based on being clearly out-performed by G.722.1
>>> - Speex-UWB, based on the fact that the author himself does not recommend
>>> it being used :-)
>>>
>>> We can keep the other reference codecs as minimum quality requirement and
>>> include being no worse than AMR-NB and AMR-WB as "objectives" that are
>>> "nice
>>> to have", but not hard requirements.
>>>
>>> From there and based on the listening tests presented by Jan Skoglund
>>> today, let's see what we can already conclude and what still needs more
>>> testing:
>>>
>>> 1) The narrowband test showed that Opus had higher quality than Speex at
>>> 11
>>> kb/s. Does anyone disagree that this is sufficient to meet the Sec 4.2
>>> requirement of out-performing Speex in narrowband mode?
>>>
>>> 2) The narrowband test showed that Opus had higher quality at 11 kb/s
>>> than
>>> iLBC at 15 kb/s. Does anyone disagree that this is sufficient to meet the
>>> Sec 4.2 requirement of out-performing iLBC.
>>>
>>> 3) There have been no formal comparison with AMR-NB yet. What do you
>>> think
>>> would be sufficient to assess the quality of Opus compared to AMR-NB?
>>>
>>> 4) The wideband test showed that Opus at 19.85 kb/s had higher quality
>>> than
>>> Speex-WB at 24 kb/s. Does anyone disagree that this is sufficient to meet
>>> the Sec 4.2 requirement of out-performing Speex in wideband mode?
>>>
>>> 5) The wideband test showed that Opus at 19.85 kb/s had higher quality
>>> than
>>> G.722.1 at 24 kb/s. Does anyone disagree that this is sufficient to meet
>>> the
>>> Sec 4.2 requirement of out-performing G.722.1?
>>>
>>> 6) The wideband test showed that Opus at 19.85 kb/s had higher quality
>>> than
>>> AMR-WB at 19.85 kb/s. Does anyone disagree that this is sufficient to
>>> concluded that the proposed "nice to have" objective of "no worse than
>>> AMR-WB" is met?
>>>
>>> 7) The fullband test showed that Opus at 32 kb/s had higher quality than
>>> G.719 at 32 kb/s. Does anyone disagree that this is sufficient to meet
>>> the
>>> Sec 4.2 requirement of out-performing G.722.1C, considering that G.719
>>> has
>>> already been shown to out-perform G.722.1C
>>>
>>> If you disagree with any of the points above -- as may very well be the
>>> case -- please do provide a concrete test proposal that would be
>>> sufficient
>>> to convince you.
>>>
>>> Cheers,
>>>
>>>    Jean-Marc and Koen
>>>
>>> _______________________________________________
>>> codec mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/codec
>>>
>>>
>>>
>>
>
>
>