Re: [codec] #1: Application: 2.1. Point to point calls supporting transcoding?

Quoting Koen Vos <koen.vos@skype.net>:

> Any testing of tandem coding can only be about the audio quality.  
> There is no binary question about whether "it works" or not, because  
> any two codecs do indeed work together when connected through audio.
>
> I don't see a need to add tandem testing to the requirements.   
> Quality will be high when operating towards the high end of the  
> range in bitrates mentioned in the requirements, and tandem quality  
> will be limited by the other codec. For lower bitrates the quality  
> naturally goes down.. not sure what to test?

Assuming two codecs, for example, CELT and AMR-WB, you must compare  
all CELT modes (about 16) against all AMR-WB modes (about 16), using  
different sample contents (4), and with noise and without (2).  
Typically, you need 8 ratings from different persons to get precise  
results. Actually, one needs to make the tests in at least two  
different labs. Thus, only 65536 listening-only ratings are required.  
These results you put in a document that is not public accessible and  
that nobody will read.

Then everybody will believe that transcoding is a good idea and should  
be done at least twice in every call.

  Christian

>
> koen.
>
>
> Quoting stephen botzko:
>> I personally believe testing tandem operation is a MUST, since devices using
>> this CODEC will certainly connect into networks where it is not supported.
>> The idea below that this will "just work" with every ITU codec is
>> incorrect..
>>
>> In addition, tandem operation occurs in virtually all conference bridges
>> (including back-to-back encode/decode with CODEC itself).  So surely
>> self-tandeming has to be tested anyway.  (Though of course is is common for
>> devices in the same conference to be using different codecs).
>>
>> I do not favor just copying the tandem requirements from some other codec's
>> test plan - unless we are sure that it is in essentially the same ecosystem
>> as ours.  Ideally the community would identify common use cases, and the
>> test cases would be derived from those cases.
>>
>> Stephen Botzko
>>
>> On Sat, Apr 3, 2010 at 1:39 AM, codec issue tracker  
>> <trac@tools.ietf.org>wrote:
>>
>>> #1: Application: 2.1.  Point to point calls supporting transcoding?
>>>
>>> ------------------------------------+---------------------------------------
>>> Reporter:  hoene@...                 |       Owner:
>>>    Type:  enhancement             |      Status:  new
>>> Priority:  major                   |   Milestone:
>>> Component:  requirements            |     Version:
>>> Severity:  Active WG Document      |    Keywords:
>>>
>>> ------------------------------------+---------------------------------------
>>>
>>> Comment(by hoene@...):
>>>
>>> No consensus either until now. Again two opinions:
>>>
>>> a) "To have similar requirements as G.718 and to consider at least cross
>>> tandeming conditions with G722.2@12.65 in the characterization phase."
>>>
>>> b) "I think that transcoding must be an explicit non-goal for this codec."
>>> "Sorry, but I still do not understand the need for this testing.  In a
>>> cell network to VoIP topology, the codec selected must be an ITU codec
>>> which we already know are working fine for transcoding.  In a VoIP to VoIP
>>> topology the codec selected is the one defined by this WG and by
>>> definition no transcoding is needed.  And that's it, no test needed,
>>> because nobody will use it this way."
>>>
>>> Again, I would suggest to look for volunteers who want to make the
>>> subjective tests with G.722.2 and CODEC.
>>>
>>> --
>>> Ticket URL: <https://svn.tools.ietf.org/wg/codec/trac/ticket/1#comment:1>
>>> codec <http://tools.ietf.org/codec/>
>>>
>>> _______________________________________________
>>> codec mailing list
>>> codec@ietf.org
>>> https://www.ietf.org/mailman/listinfo/codec
>>>
>>
>
>
> _______________________________________________
> codec mailing list
> codec@ietf.org
> https://www.ietf.org/mailman/listinfo/codec
>