Re: [codec] Testing: A Novel Proposal

[Anisse Taleb <[e-mail]>]

Dear Monty,
 
If you refer to the draft proposal, we are in a presence of a multimode multi-bandwidth codec, the per-bandwidth tests proposed are small in comparison to other tests for other codecs, however putting all these together makes the test quite large. For instance, ITU-T codecs usually address a single bandwidth.

It was never the intention to make this an impossible test/task neither was the intention to delay the publication of this codec. I am disappointed that some of the codec proponents and supporters are defensive and quite dismissive. This is an opportunity for everyone to show and demonstrate the quality of the codec in a hopefully well designed test that could be reproduced at will and hence not be easily dismissed or argued against.

As I repeatedly said this was an initial proposal and I value the technical comments and feedback by Jean-Marc, Roman, Stephen, Koen and others in and outside this mailing list. I am working on an update of that test plan and is hopeful to send a version soon. Please let me know if you have any comments such that I can accommodate your point of view too, even if you have little horns :-).

When it comes to your proposal,

"If it isn't good enough... Prove it."

I do not think I have anything to say about that except perhaps to remind you of your own statements 2 years ago, in response to Roni, 

"a lot of characterization, testing, and documentation needs to be done. We're aware of that and we have a lot of work to do."

Kind regards,
/Anisse