Re: [codec] A concrete proposal for requirements and testing
Roman Shpount <roman@telurix.com> Thu, 31 March 2011 16:18 UTC
Return-Path: <roman@telurix.com>
X-Original-To: codec@core3.amsl.com
Delivered-To: codec@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3444E3A69BF for <codec@core3.amsl.com>; Thu, 31 Mar 2011 09:18:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.976
X-Spam-Level:
X-Spam-Status: No, score=-2.976 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jjFoDbVXHrlc for <codec@core3.amsl.com>; Thu, 31 Mar 2011 09:18:02 -0700 (PDT)
Received: from mail-iw0-f172.google.com (mail-iw0-f172.google.com [209.85.214.172]) by core3.amsl.com (Postfix) with ESMTP id EDA553A684E for <codec@ietf.org>; Thu, 31 Mar 2011 09:18:01 -0700 (PDT)
Received: by iwn39 with SMTP id 39so2949449iwn.31 for <codec@ietf.org>; Thu, 31 Mar 2011 09:19:41 -0700 (PDT)
Received: by 10.42.221.136 with SMTP id ic8mr1318062icb.487.1301588381510; Thu, 31 Mar 2011 09:19:41 -0700 (PDT)
Received: from mail-iy0-f172.google.com (mail-iy0-f172.google.com [209.85.210.172]) by mx.google.com with ESMTPS id i20sm831785iby.65.2011.03.31.09.19.39 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 31 Mar 2011 09:19:40 -0700 (PDT)
Received: by iye19 with SMTP id 19so2955090iye.31 for <codec@ietf.org>; Thu, 31 Mar 2011 09:19:39 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.231.194.34 with SMTP id dw34mr2573415ibb.117.1301588379206; Thu, 31 Mar 2011 09:19:39 -0700 (PDT)
Received: by 10.231.19.11 with HTTP; Thu, 31 Mar 2011 09:19:39 -0700 (PDT)
In-Reply-To: <64212FE1AE068044AD567CCB214073F123A10234@MAIL2.octasic.com>
References: <64212FE1AE068044AD567CCB214073F123A10234@MAIL2.octasic.com>
Date: Thu, 31 Mar 2011 12:19:39 -0400
Message-ID: <AANLkTinrS6e+i_VE5Q91gbNV5yhwi3Ad+wZOUsKYGNZn@mail.gmail.com>
From: Roman Shpount <roman@telurix.com>
To: Jean-Marc Valin <jean-marc.valin@octasic.com>
Content-Type: multipart/alternative; boundary="005045016245fc828a049fc9a8bc"
Cc: "codec@ietf.org" <codec@ietf.org>
Subject: Re: [codec] A concrete proposal for requirements and testing
X-BeenThere: codec@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Codec WG <codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/codec>
List-Post: <mailto:codec@ietf.org>
List-Help: <mailto:codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 31 Mar 2011 16:18:03 -0000
I might be missing something, but how is it that G.722 is clearly out-performed by G.722.1? As far as I know G.722 at 64 KB has higher quality then wideband G.722.1 at any rate. Obviously, G.722 has higher bandwidth requirements, but has higher quality and lower bandwidth. Outperforming G.722 at 16KHz for all languages, and especially for multiple simultaneous speakers or non-speech sources is not an easy thing and should be tested against. _____________ Roman Shpount On Thu, Mar 31, 2011 at 10:53 AM, Jean-Marc Valin < jean-marc.valin@octasic.com> wrote: > Hi, > > Following the meeting and post-meeting discussions about requirements and > testing, we would like to make the following proposal which addresses the > opposing views which prevented consensus in the meeting today. > > First, we propose to remove the following codecs from the requirements: > > - GSM-FR, based on consensus from the list > - G.722, based on being clearly out-performed by G.722.1 > - Speex-UWB, based on the fact that the author himself does not recommend > it being used :-) > > We can keep the other reference codecs as minimum quality requirement and > include being no worse than AMR-NB and AMR-WB as "objectives" that are "nice > to have", but not hard requirements. > > From there and based on the listening tests presented by Jan Skoglund > today, let's see what we can already conclude and what still needs more > testing: > > 1) The narrowband test showed that Opus had higher quality than Speex at 11 > kb/s. Does anyone disagree that this is sufficient to meet the Sec 4.2 > requirement of out-performing Speex in narrowband mode? > > 2) The narrowband test showed that Opus had higher quality at 11 kb/s than > iLBC at 15 kb/s. Does anyone disagree that this is sufficient to meet the > Sec 4.2 requirement of out-performing iLBC. > > 3) There have been no formal comparison with AMR-NB yet. What do you think > would be sufficient to assess the quality of Opus compared to AMR-NB? > > 4) The wideband test showed that Opus at 19.85 kb/s had higher quality than > Speex-WB at 24 kb/s. Does anyone disagree that this is sufficient to meet > the Sec 4.2 requirement of out-performing Speex in wideband mode? > > 5) The wideband test showed that Opus at 19.85 kb/s had higher quality than > G.722.1 at 24 kb/s. Does anyone disagree that this is sufficient to meet the > Sec 4.2 requirement of out-performing G.722.1? > > 6) The wideband test showed that Opus at 19.85 kb/s had higher quality than > AMR-WB at 19.85 kb/s. Does anyone disagree that this is sufficient to > concluded that the proposed "nice to have" objective of "no worse than > AMR-WB" is met? > > 7) The fullband test showed that Opus at 32 kb/s had higher quality than > G.719 at 32 kb/s. Does anyone disagree that this is sufficient to meet the > Sec 4.2 requirement of out-performing G.722.1C, considering that G.719 has > already been shown to out-perform G.722.1C > > If you disagree with any of the points above -- as may very well be the > case -- please do provide a concrete test proposal that would be sufficient > to convince you. > > Cheers, > > Jean-Marc and Koen > > _______________________________________________ > codec mailing list > codec@ietf.org > https://www.ietf.org/mailman/listinfo/codec > >
- [codec] A concrete proposal for requirements and … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Roman Shpount
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Roman Shpount
- Re: [codec] A concrete proposal for requirements … Benjamin M. Schwartz
- Re: [codec] A concrete proposal for requirements … Roni Even
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Jan Skoglund
- Re: [codec] A concrete proposal for requirements … Anisse Taleb
- Re: [codec] A concrete proposal for requirements … Erik Norvell
- Re: [codec] A concrete proposal for requirements … Anisse Taleb
- Re: [codec] A concrete proposal for requirements … Anisse Taleb
- Re: [codec] A concrete proposal for requirements … Paul Coverdale
- Re: [codec] A concrete proposal for requirements … Benjamin M. Schwartz
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Paul Coverdale
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Anisse Taleb
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Paul Coverdale
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Anisse Taleb
- Re: [codec] A concrete proposal for requirements … Paul Coverdale
- Re: [codec] A concrete proposal for requirements … Peter Saint-Andre
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Benjamin M. Schwartz
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Peter Saint-Andre
- Re: [codec] A concrete proposal for requirements … Timothy B. Terriberry
- Re: [codec] A concrete proposal for requirements … Stephan Wenger
- Re: [codec] A concrete proposal for requirements … Monty Montgomery
- Re: [codec] A concrete proposal for requirements … Timothy B. Terriberry
- Re: [codec] A concrete proposal for requirements … Stephan Wenger
- Re: [codec] A concrete proposal for requirements … Gregory Maxwell
- Re: [codec] A concrete proposal for requirements … Monty Montgomery
- Re: [codec] A concrete proposal for requirements … Koen Vos
- Re: [codec] A concrete proposal for requirements … Roman Shpount
- Re: [codec] A concrete proposal for requirements … Koen Vos
- Re: [codec] A concrete proposal for requirements … Stephan Wenger
- Re: [codec] A concrete proposal for requirements … Paul Coverdale
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Gregory Maxwell
- Re: [codec] A concrete proposal for requirements … Roman Shpount
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Koen Vos
- Re: [codec] A concrete proposal for requirements … Paul Coverdale
- Re: [codec] A concrete proposal for requirements … Roni Even
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Kavan Seggie
- Re: [codec] A concrete proposal for requirements … Roni Even
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Roni Even
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Roni Even
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Stephan Wenger
- Re: [codec] A concrete proposal for requirements … Kat Walsh
- Re: [codec] A concrete proposal for requirements … Stefan Hacker
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Paul Coverdale
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Serge Smirnoff
- Re: [codec] A concrete proposal for requirements … Anisse Taleb
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Jean-Marc Valin
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Ron
- Re: [codec] A concrete proposal for requirements … Stephen Botzko
- Re: [codec] A concrete proposal for requirements … Anisse Taleb
- [codec] Chairs and consensus Cullen Jennings