Re: [codec] comparitive quality testing: vote to add iLBC in

Roman Shpount <roman@telurix.com> Fri, 29 April 2011 23:15 UTC

Return-Path: <roman@telurix.com>
X-Original-To: codec@ietfa.amsl.com
Delivered-To: codec@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 612E0E06AF for <codec@ietfa.amsl.com>; Fri, 29 Apr 2011 16:15:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.976
X-Spam-Level:
X-Spam-Status: No, score=-2.976 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W4k+WVRWpd-J for <codec@ietfa.amsl.com>; Fri, 29 Apr 2011 16:14:59 -0700 (PDT)
Received: from mail-ew0-f44.google.com (mail-ew0-f44.google.com [209.85.215.44]) by ietfa.amsl.com (Postfix) with ESMTP id 4223FE0698 for <codec@ietf.org>; Fri, 29 Apr 2011 16:14:58 -0700 (PDT)
Received: by ewy19 with SMTP id 19so1550425ewy.31 for <codec@ietf.org>; Fri, 29 Apr 2011 16:14:58 -0700 (PDT)
Received: by 10.213.96.216 with SMTP id i24mr2397422ebn.76.1304118897945; Fri, 29 Apr 2011 16:14:57 -0700 (PDT)
Received: from mail-ew0-f44.google.com (mail-ew0-f44.google.com [209.85.215.44]) by mx.google.com with ESMTPS id g1sm742242een.3.2011.04.29.16.14.56 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 29 Apr 2011 16:14:56 -0700 (PDT)
Received: by ewy19 with SMTP id 19so1550421ewy.31 for <codec@ietf.org>; Fri, 29 Apr 2011 16:14:56 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.14.11.26 with SMTP id 26mr2333492eew.114.1304118896184; Fri, 29 Apr 2011 16:14:56 -0700 (PDT)
Received: by 10.14.127.3 with HTTP; Fri, 29 Apr 2011 16:14:55 -0700 (PDT)
In-Reply-To: <C9E050C0.15C38%jf.mule@cablelabs.com>
References: <260B5AEE-577C-478B-9335-553C7E9ABD8F@telio.no> <C9E050C0.15C38%jf.mule@cablelabs.com>
Date: Fri, 29 Apr 2011 19:14:55 -0400
Message-ID: <BANLkTi=6pb8BngAtwfDyBjOhq+2Vh++FaQ@mail.gmail.com>
From: Roman Shpount <roman@telurix.com>
To: Jean-Francois Mule <jf.mule@cablelabs.com>
Content-Type: multipart/alternative; boundary=0016364c7bd58d4da204a216d77b
Cc: "codec@ietf.org" <codec@ietf.org>
Subject: Re: [codec] comparitive quality testing: vote to add iLBC in
X-BeenThere: codec@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Codec WG <codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/codec>, <mailto:codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/codec>
List-Post: <mailto:codec@ietf.org>
List-Help: <mailto:codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Apr 2011 23:15:02 -0000

Jean-Francois,

My argument against iLBC essentially comes down to two points:

1. It is argued that we should compare with iLBC since it is IPR free and
unencumbered CODEC. What I am not sure about is the current license for the
iLBC source code. RFC 3951 specifies that the code belongs to IETF.
iLBCFreeware.org states that the code is covered by "GIPS Royalty Free
License", which is strange since you normally should mention this in the
code to be binding. At the same time "GIPS Royalty Free License" gives no
rights to use GIPS IP which is part of iLBC. It only covers the code. IPR
release specifies that GIPS will provide a license for the codec IP upon
written request, but this license is not publicly available. GIPS is now
Google, and even if Google will still provide a royalty free license per
IPR, this license can have other terms in it that might restrict iLBC use.
It would be very interesting to see this license. Second problem are IPR
disclosures. None of them have been updated when RFC was accepted and all of
them only mention the provisional patents. It would be nice to have actual
patent numbers if the patents where granted or if they still apply to the
final RFC. Finally, based on the latest Qualcomm disclosure in regard to
Opus, it is unclear why similar disclosures weren't filed against Silk and
iLBC. It is unclear if this because the patents do not apply or is this due
to some other reason. My assumption was that iLBC was not extremely
important to the market so that no one followed up to clarify the above
mentioned points. In any case, the reasoning for this is not important, but
the resulting iLBC licensing status is.

2. iLBC is not a bit exact codec. You can have multiple standard compliant
implementations that would produce audio of different quality. Reference
implementation is a lot more computationally demanding then G.729A. There
are proprietary implementations on the market that are comparable in their
computational requirements to G.729A, but producing different encoded audio
then the reference. No open source fixed point or otherwise optimized iLBC
implementations are available, so there is nothing that we can use to
estimate the audio quality of an optimized iLBC implementation. The
reasoning for not having such implementation is unclear, but I assumed this
was due to the luck of interest in the market. Once again, the reasoning is
irrelevant. What this means, that if we are testing against reference iLBC
implementation built from the RFC code even though it might be producing a
better quality audio then the optimized implementations. If we are comparing
against the reference we need to specify compiler and compilation options
used to build the codec, since codec output changes based on it. These
changes might not affect the quality, but would produce an unreproducible
test. In some sense, stating in the requirements that Opus should be better
then iLBC is the same as stating it should be better then MP3 without
specifying the encode or the options used.

>From my point of view we can spend time and clarify these two issues (or at
least the second issue). Alternatively we can remove the requirement to test
against iLBC, and spend our limited resource testing Opus against other
codecs.
_____________
Roman Shpount


On Fri, Apr 29, 2011 at 6:06 AM, Jean-Francois Mule
<jf.mule@cablelabs.com>wrote:

> Hi, Alan and Roman,
>
>   I am not sure what this thread leads the working group but I'd like to
> say a few points about iLBC.
>
> Roman wrote:
> > One can argue that we should also compare Opus with patent free codecs,
> > which adds iLBC and Speex to the list, but I personally see this as
> > less of a requirement. iLBC never managed to get market traction
> > outside of the open source world,
> Doing comparative quality testing against G.711, iLBC, G.729, G.722, and
> G.722.2 AMR-WB provides considerable values to many vendor companies that
> have implemented these codecs and to numerous service providers that have
> deployed them (or are considering to).  I'd like to see iLBC added to the
> list.
>
>
> My team at CableLabs did quite a bit of work on iLBC and other codecs
> almost 10 years ago.
> Around 2002, we launched a project to find a royalty-free, narrowband
> codec that could be implemented in PacketCable(tm) networks and that could
> be submitted to our IPR pool including codec implementation references.
> All we had when we started was G.711.  I'll skip the multi-year process...
> iLBC was selected by CableLabs after some extensive testing outsourced to
> two well-known and independent testing labs staffed with what I considered
> at the time worldwide codec experts having worked on many codecs and
> implementations (including some well-known standard codecs).  iLBC became
> part of our PacketCable 1.1 now called PacketCable 1.5 specifications
> prior to the codec becoming an IETF Experimental RFC.
> In terms of voice codec deployments in cable networks, the fact is that it
> is mostly G.711 today in the field - no questions there. Each cable
> operator makes its own decision. The reasons for this lack of iLBC
> deployments in cable networks are numerous and for the most part, outside
> codec considerations.
>
> That said, iLBC has been implemented by many as noted on the list.
> I'm not sure if this was mentioned but my wireshark capture of an Apple
> Facetime call shows a SIP invite with support for iLBC in addition to
> X-AAC_ELD, X-G722 and PCMU.
> --- Facetime call capture done in Oct 2010
> User-Agent: Viceroy 1.4.1/GK
>        Content-Type: application/sdp
>        Content-Length: 587
>    Message Body
>        Session Description Protocol
>            Session Description Protocol Version (v): 0
>            Owner/Creator, Session Id (o): GKVoiceChatService 0 0 IN IP4
> 192.160.73.209
>                Owner Username: GKVoiceChatService
>                Session ID: 0
>                Session Version: 0
>
> ...
>
> Media Attribute (a): rtpmap:124 iLBC/8000
>
>
>
> Jean-Francois.
> jfm@cablelabs.com
>
>
> On 04/26/2011 23:14, "Alan Duric" <Alan.Duric@telio.no> wrote:
>
> >Dear Roman,
> >
> >unfortunately, You are continuing with "say so" statements ...  Please
> >read my postings before You reply and use facts and sources, rather then
> >throwing FUD!
> >
> >Basically, You have expanded our initial conversation about iLBC
> >deployments with 2 additional topics. I would like to focus on iLBC
> >deployments and close on that subject on this thread. As per other newly
> >introduced topics:
> >1) iLBC implementations and quality
> >- my response to You is that it depends on how good work is done with the
> >implementation. There is a number of very good and excellent
> >implemenations (e.g by silicon & gateway vendors and a number of ATA/IP
> >Phone vendors), which run in a similar manner as implemented and
> >"advertised" by GIPS at the time.
> >
> >2) iLBC license
> >- this new introduced topic I propose that we take as separate thread and
> >discussion point, not confuse it with this thread. However, there is so
> >far no charges pressed to anyone, years after it has been deployed. As
> >per G.729A, I am also very glad that it has been released on royalty free
> >basis to the community, which was also pretty much happened due to
> >availability of iLBC.
> >
> >As per iLBC deployments, please find response inline:
> >
> >>
> >> This discussion is a bit unrelated to the overall discussion in the
> >>group, but I would still like to clarify some of my points.
> >>
> >> As far as iLBC market penetration is concerned, what I know (and please
> >>correct me if I am wrong), none of the major US VoIP carriers (AT&T,
> >>Verizon, Qwest, Level 3, XO, GlobalCorssing) offer iLBC termination to
> >>their customers. It is a choice of G.711 or G.729 A or B only.
> >
> >Yes, You are wrong! One of the listed carriers (which is present in
> >Europe) is offering it. According to Sonus there is 8 of major US
> >carriers using iLBC!
> >
> >> This is either caused by the luck of market demand, or by them using
> >>Sonus gateways which, as far as I know, do not support iLBC.
> >
> >Where do You get this info??? Sonus supports iLBC on all of their
> >gateways since GSX 6.5.0R0 (to be more exact under a feature PCR 111).
> >This has been available for a number of years - the best illustration is
> >that this software version is now going out of life ... Source is Sonus
> >and my company, we are Sonus customers since end of 2005 and running
> >billions of minutes of traffic on Sonus.
> >>
> >> Once again, as far as I know Skype is not using iLBC for anything. It
> >>is using Silk, G.711. G.729A, and ISAC.
> >
> >And again ... where do You get all this info from??? A number of wrong
> >statements in only one sentence (BTW, SILK has phased out pretty much
> >completely ISAC a long time ago).
> >>
> >> As far as I know none of the cable VoIP operators in US use iLBC, it is
> >>all G.711.
> >
> >This is also not true, i trust that someone from CableLabs that follows
> >the list will react on this (providing us with first hand info).
> >
> >>
> >> As far as hardware vendors are concerned, Cisco does support iLBC on
> >>its gateways a quite a few (but I do not think all) of its phones. Sonus
> >>only supports iLBC in its SBC, but not in gateways. Polycom, Snom,
> >>Aastra do not seem to support iLBC in their phones.
> >
> >As Cullen mentioned it is supported by Cisco, some Polycoms and also a
> >whole range of Asian vendors that by now own quite a bit of the market
> >around the world. I know that SNOM 870 also supports iLBC and I trust a
> >number of their phones  ... Where do You get all this??
> >
> >As mentioned in my previous email, if You disagree please provide us with
> >facts and sources, otherwise it is just "say so" and should be treated as
> >such ...
> >
> >Best regards,
> >Alan
> >
> >
> >Alan Duric
> >CTO and Cofounder
> >Telio Holding ASA | Oslo exchange: TELIO
> >
> >twitter:       alanduric
> >linkedin.com/in/alanduric
> >
>
>