Re: [codec] A concrete proposal for requirements and testing

["Roni Even" <[e-mail]>]

Hi,

According to the proponents there is nothing preventing you from using OPUS
for testing or in your product. What is being discussed is if the current
version of the codec answers the current charter of the CODEC WG.

 

I see in the email discussion two issues which may need to be resolved.

 

The first is if the codec meets the requirements, but the requirements are
not finalized yet. I can see that agreement can be reached if there is a
focus to finish the requirement document. Note that the objectives are "
Designing for use in interactive applications (examples include, but are not
limited to, point-to-point voice calls, multi-party voice conferencing,
telepresence, teleoperation, in-game voice chat, and live music
performance)" so there are claims that you need to compare OPUS to codecs
used in this application spaces. Another way to address it is to limit the
applications in the charter. One example may be to remove the telepresence
application case.

 

 

The second issue is the objective that  "The goal of this working group is
to ensure the existence of a single high-quality audio codec that is
optimized for use over the Internet and that can be widely implemented and
easily distributed among application developers, service operators, and end
users." Here again we may need to address the new IPR claims. It can be done
by re-charter or by changing the codec around the IPR. In the mean time some
view that in the current state, the OPUS codec should be compared to
existing royalty baring codecs.
 
Regards
Roni Even

 

From: [e-mail] [mailto:[e-mail]] On Behalf Of
Kavan Seggie
Sent: Saturday, April 09, 2011 9:15 AM
To: Ron
Cc: [e-mail]
Subject: Re: [codec] A concrete proposal for requirements and testing

 

> We don't have just a handful
> of companies responsible for signing off on a spec, and making do if
> it later proves insufficient - we have millions of them, most of which
> haven't even heard of this group yet.  And the best way to engage them
> is to provide a proposed standard which they can work to and assess
> for their own uses.

 

+1

 

I am the Founder and CEO of a startup who has been quietly observing this WG
and the development of CELT for almost two years.

 

It would be great if we could progress to real world testing and get the
product out to the people. For us, SILK is an amazing codec, better than any
voice codec we have tried including proprietary technology from the best
vendors. CELT, sounds excellent too. We do not have the resources to run a
formal test plan so we trust the ears of our team, friends and family and we
think they are really solid.

 

IMHO the next step is to provide a release version of Opus to the wider
community for real world testing. They'll let you know if there are any real
issues far quicker than more formal testing.

 

Regards

 

Kavan

 

On Sat, Apr 9, 2011 at 4:06 AM, Ron <[e-mail]> wrote:


On Fri, Apr 08, 2011 at 11:32:33PM +0300, Roni Even wrote:
> And as stated in the charter on deliverables:
>
> " Specification of a codec that meets the agreed-upon requirements"
>
> I think there are no agreed-upon requirements yet and this is what I see
as
> the  current discussion and not if the codec has fulfilled the
requirements.

Thanks for the change of tack.  Since it seems unlikely that the WG
document draft-ietf-codec-requirements-02 would be amended to impose
a quality requirement of "Better than transparent", this current
thread has clearly been sailing a bit too close to the wind so far.

Which parts of that document do you still dispute or think need
improving?

That seems like precisely the sort of thing we should be finalising
now, rather than embarking upon arduous additional quality testing
that all indications are will only tell us things we already know.



> From: [e-mail] [mailto:[e-mail]] On Behalf Of
Paul Coverdale
> Sent: Friday, April 08, 2011 9:32 PM

> Koen Vos wrote:
> > quality has been shown to be good enough for the codec to be useful...
>
> I think that's where some people have difficulty. There's been no
systematic
> attempt to evaluate Opus against the performance requirements given in the
> codec requirements document (as thin as it is) in a controlled and
repeatable
> manner.

I think that's a bit disingenuous to the careful methodology that was
employed by the developers, and to the people who so far have done
blind and scientific tests of their results, but without dwelling on
that slur:

There's also been no, even informal, results presented to suggest that
it does not exceed even the most ambitious performance requirement
expectations held at the outset, by a rather surprising margin.

There has certainly been a systematic attempt to cast doubt upon the
results that we do have, but so far not a shred of even questionable
evidence to back those doubts up.

Do you have some results to share, that back up the claims of the
people "having difficulty" accepting what has been achieved to date?


While I respect your adherence to processes that are indeed necessary
for the ITU to licence a technology and deploy it in the telephone
network, one of the very reasons for forming this group under the
auspices of the IETF was that the reality of developing and deploying
internet services is quite different.  We don't have just a handful
of companies responsible for signing off on a spec, and making do if
it later proves insufficient - we have millions of them, most of which
haven't even heard of this group yet.  And the best way to engage them
is to provide a proposed standard which they can work to and assess
for their own uses.

If we'd had to wait for DNS and IPv6 to mature before continuing that
process, our jaws wouldn't be dropping at what this group has produced
for its first proposed release, like they are today.

We have some very clever and very experienced people here.  If none of
them can present us with actual engineering faults (as opposed to minor
procedural details), then it's time to put our work to the real test.

Every day that goes by in which such faults aren't found and presented
is further evidence to support the appropriateness of the Last Call.


Cheers,
Ron



_______________________________________________
codec mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/codec




-- 
Kavan Seggie

SayMama <http://www.saymama.com>  - The video chat social network.

 

Find us on Facebook: facebook.com/officialsaymama

Follow us on Twitter: twitter.com/saymama