[codec] comparitive quality testing

[Gregory Maxwell <[e-mail]>]

Roni Even [[e-mail]] wrote:
> I do not mind if the WG will decide to remove the quality claim and continue
> with developing  a royalty free codec with "good enough" quality not saying
> it is better than other codecs.
> I just think that it should be clear from the charter and requirements what
> is the purpose of the work.

It's funny how we can argue and argue, only to later realize that it comes
down to a simple mutual misunderstanding.

I thought everyone was already on the same page with respect to the 
goals: it's good to be as good as possible, but the chartered purpose
of the WG was only to do a "good quality" codec that was suited
to the listed applications and deployments.

As a developer I know that quality testing is important, and of course
we've done a lot of it of various types.  I strongly believe in scientific
testing, so of course my first instinct would have been to do it here,
but perhaps the reality of the consensus process makes that less
reasonable—as others have pointed out, most other WGs don't really
do anything comparable to quality testing.

Likewise, making sure the outcome is as legally unencumbered as I can
is also very important to me,  but because of the vulgarities of the
process and the law, this isn't something that the working group itself
makes promises about.

So, perhaps it makes sense for the working group to not make any quality
promises in the same way it makes no promises about patents.

It seems clear enough to me now that we can much more easily come to
consensus about achieving good-enough status than about formal testing
gates and requirements.

We should accept your suggestion—drop all the comparative quality
requirements from the requirements draft, and stop discussing comparative
quality here—and then make some progress on technology, rather than
continue bickering about details where we are not going to come to
consensus.

The market can figure out the comparative quality question on its own.