Re: [codec] A concrete proposal for requirements and testing

[Anisse Taleb <[e-mail]>]

Dear Ron,

> If we can take that as meaning that nobody actually does
> disagree with the points that Jean-Marc asked to clarify, then I think
> we can take that as meaning this (original) thread has actually achieved
> its purpose and we now do have consensus on those points.

With all the "background noise" you may have missed my reply to JM. I do not think we have consensus.

> I don't think anyone disagrees on what testing means.

We definitely do. I have heard many voices stating the codec has been extensively tested, and consider the tests conducted during the development of the codec as representative of what a "codec standardization test" really means.  

> And in the face of the test results that
> we do already have

What test results you speak of ? 
- Tests that have been conducted on various versions of the codec and hence not allowing to look at the results side by side.
- Admittedly inadequate methodologies for certain tests
- Objective measurements (oh please !)
- Low number of subjects
- Lack of items 
- No cross-checks

Don't get me wrong, I really appreciate the efforts of the developers and the testers in their continuous effort to improve and bug-fix the codec, but this is a natural part of codec development and a necessary burden to produce a decent codec. However, I strongly disagree that the results seen so far represent more than a simple indication of a solid performance that qualify the codec to be considered for requirement testing. 

> 
> Such tests may still be interesting to see (I'd be interested in seeing
> them for one), but I strongly disagree that they are at all serving the
> aims of the WG, and don't think we should delay its progress if we don't
> have them in time.

And what does serve the aims of the WG ?

The charter clearly states the goals of the WG. Producing an un-encumbered high quality audio codec that is optimized for internet applications. How can you speak of serving the aims of the WG by rushing an encumbered codec of questionable testing with no consideration of FER and delay jitter subjective tests into WGLC. 

Kind regards,
/Anisse