[codec] Compare tool sensitivity

[Koen Vos <[e-mail]>]

(Changed the thread subject)

Ron wrote:
> There is one requirement.  Good interoperability.

I agree completely.  All compliant future encoders and decoders should work well together.  This also prevents embrace-and-extend tactics.

However, within the space of guaranteed interop, there are benefits to being lenient:

1. The decoder may be improved for specific use cases, by finding a different tradeoff between quality, delay, complexity, etc.  For instance in a non-real time application it may help to have a resampler with higher quality, at the cost of higher delay and complexity.

2. A limited decoder may be better.  For instance, an Opus-to-PSTN gateway only needs a decoder to generate 8 kHz output signals.  Allowing such a decoder enables optimizations in terms of memory and CPU.  (Of course the decoder would still have to decode any Opus bitstream, including one with more than NB content.)  

3. The current test disqualifies a huge space of decoders that sound (virtually) identical.  Simply getting the sign wrong of the output signal shouldn't make a company liable for a patent lawsuit.

A non-goal for the WG, in my opinion, is to ensure that all users get the quality they are deemed to deserve.  While nobody wants decoders that sound like crap, few users will stick with an application that plays out a jingle whenever an Opus bitstream is received.  And we already allow crappy _encoders_.  I prefer the market place to weed out crap over threats of litigation.

While there's no perfect solution, I think Jean-Marc and I will come up with a compare tool that provides a small -but useful- amount of freedom to change the decoder without breaking the interop guarantee.

koen.


----- Original Message -----
From: "Ron" <[e-mail]>
To: [e-mail]
Sent: Thursday, November 17, 2011 3:18:14 AM
Subject: Re: [codec] WGLC #2 for draft-ietf-codec-opus-10

On Wed, Nov 16, 2011 at 11:37:12AM +0100, Christian Hoene wrote:
> Hi,
> 
> I would suggest to split two things: 
> 
> First, we need a notion for standard conformance because of the legal IPR
> requirements (can be more relaxing). 
> Second, we need some tests in respect to interoperability (should be more
> straight).
> 
>  Both tests need to fulfill different requirements.

I don't see these as different requirements at all.

The IPR in this case is not being used to protect a revenue stream garnered
from unwitting users of the technology - it is being used to protect the
interoperability guarantees that can be offered to end users.

Separating these two things, and watering down the protection offered by
the IPR does not offer any benefit to users, only to implementers who might
be tempted to compromise on interoperability.  Possibly for 'practical'
reasons, and possibly for 'commercial' reasons of their own - but their
reasons don't matter, only the end effect, a degraded user experience, is
the important consideration (to avoid) here.

There is one requirement.  Good interoperability.  IPR grant conformance
is the only binding method we have of ensuring this.  As heretical as it
may seem to the FRAND profit seekers - the real profit to be made here
from patents is an enhanced and assured user experience.

Of course, since I don't actually own any of the IPR, I can't speak for
the people who do, or for their motives - but this is what I see as their
most valuable gift to us, above and beyond the actual technology itself.


> Also, I like Erik's idea to allow for algorithms to replace patented stuff
> in the codec - even if the quality is a bit worse.

I think this is a fallacy.  It was argued that this extra freedom might
allow people to produce a 'better' decoder.  That seems like nonsense to
me, the decoder is never going to be able to recover information that
simply isn't there - the best it can do is recover all of it with tight
fidelity.  Which is what the current conformance test tries to measure.

It was also argued that this freedom might allow people to produce a
worse decoder.  That seems like something we should strongly avoid.
There seem to be only three reasons to do this:

 - Save cycles on underpowered systems.
   But the requirements already include assessing complexity guarantees
   are within reasonable bounds.

 - Sidestep the IPR to produce systems that are not fully interoperable
   but can still claim (some semblance of) conformance.

 - Provide distortive 'enhancements' that some users may assess as being
   pleasing on a first brief listen, but which reduce the real fidelity
   of the output.   (eg. simulate mp3 distortion for the people who
   listening tests actually prove like that more than lossless original)

Both those latter cases carry a danger of sounding like crap if future
(real) enhancements are made to the encoder which actually does increase
the real fidelity and information content of the encoded signal.

The likelihood of such improvements being made to the encoder is high,
so I think this is a real jeopardy which we should not inflict upon
end users or limit future encoder work by.

The decoder should decode the bitstream accurately.  If it cannot, then
it should not claim to decode Opus.  People are always free to sidestep
the IPR and make something that sounds worse.  They just shouldn't be
encouraged or allowed to then also misrepresent that as Opus and mislead
the users that this group set out to serve.  As some of the liaisons
have reminded us, we're here because we wanted to *raise* the bar, not
to just continue Business As Usual in patented codec proliferation.
So we should do just that.  And make everyone happy :)

IMO

 Ron


_______________________________________________
codec mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/codec