Re: [codec] Some listening test results

Ron <ron@debian.org> Fri, 26 July 2013 20:37 UTC

Return-Path: <ron@debian.org>
X-Original-To: codec@ietfa.amsl.com
Delivered-To: codec@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5F88711E816E for <codec@ietfa.amsl.com>; Fri, 26 Jul 2013 13:37:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.423
X-Spam-Level:
X-Spam-Status: No, score=-1.423 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FH_HOST_EQ_D_D_D_D=0.765, HOST_MISMATCH_NET=0.311, RDNS_DYNAMIC=0.1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mvPHzSQBJ+-9 for <codec@ietfa.amsl.com>; Fri, 26 Jul 2013 13:37:21 -0700 (PDT)
Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [IPv6:2001:44b8:8060:ff02:300:1:2:7]) by ietfa.amsl.com (Postfix) with ESMTP id 0D28511E8176 for <codec@ietf.org>; Fri, 26 Jul 2013 13:37:17 -0700 (PDT)
Received: from ppp118-210-216-146.lns20.adl6.internode.on.net (HELO audi.shelbyville.oz) ([118.210.216.146]) by ipmail07.adl2.internode.on.net with ESMTP; 27 Jul 2013 06:07:16 +0930
Received: from localhost (localhost [127.0.0.1]) by audi.shelbyville.oz (Postfix) with ESMTP id 82DA24F8F3; Sat, 27 Jul 2013 06:07:12 +0930 (CST)
X-Virus-Scanned: Debian amavisd-new at audi.shelbyville.oz
Received: from audi.shelbyville.oz ([127.0.0.1]) by localhost (audi.shelbyville.oz [127.0.0.1]) (amavisd-new, port 10024) with LMTP id RD2sFZizb6vt; Sat, 27 Jul 2013 06:06:52 +0930 (CST)
Received: by audi.shelbyville.oz (Postfix, from userid 1000) id 3498C4F902; Sat, 27 Jul 2013 06:06:52 +0930 (CST)
Date: Sat, 27 Jul 2013 06:06:52 +0930
From: Ron <ron@debian.org>
To: Christian Hoene <christian.hoene@symonics.com>
Message-ID: <20130726203652.GM25981@audi.shelbyville.oz>
References: <006001ce8a0d$89a2aa30$9ce7fe90$@symonics.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <006001ce8a0d$89a2aa30$9ce7fe90$@symonics.com>
User-Agent: Mutt/1.5.20 (2009-06-14)
Cc: Michael Haun <michael.haun@symonics.com>, opus@xiph.org, codec@ietf.org, patrick.schreiner@symonics.com
Subject: Re: [codec] Some listening test results
X-BeenThere: codec@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Codec WG <codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/codec>, <mailto:codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/codec>
List-Post: <mailto:codec@ietf.org>
List-Help: <mailto:codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 26 Jul 2013 20:37:25 -0000

Hi Christian,

Thanks for publishing this.  Do you happen to have some numbers on
what the expected bit error rates are for a "typical AMR-WB usage
scenario"?

I'm curious about that because you note that the Opus bitstream
wasn't designed to be inherently robust against a bit error at
any point in the packet (and some places will obviously effect
it worse than others) - however it's not quite true that it
'cannot tolerate them'.

Opus requires framing for transport, and if the transport itself
doesn't ensure data integrity then it's reasonably easy for the
framing to do so (such as Ogg does with its CRC).

In which case a bit error simply becomes a packet loss, handled
by the normal Opus PLC/FEC mechanisms.  So I'm curious about how
the results you saw for various packet loss rates might map to
expected service quality, even without taking steps to make the
link itself more robust than it already is.

Given the increasing importance of data over cellular links,
I'd expect the current BER targets are already fairly low, though
I don't seem to be able to find any numbers on current expectation.
Just some papers that note "it's much lower than people think",
and "approaching the rates seen on wired networks" - though of
course it's much more likely to be bursty when errors do occur,
and going to vary widely by locale.

  Cheers,
  Ron


On Fri, Jul 26, 2013 at 04:36:36PM +0200, Christian Hoene wrote:
> Hi,
> 
> a first set of results can be downloaded here
> 
>  <http://trac.tools.ietf.org/wg/codec/trac/wiki/TestingResults>
> http://trac.tools.ietf.org/wg/codec/trac/wiki/TestingResults
> 
> More will be added soon.
> Thanks to Patrick Schreiner and Alfons Martin, who did the tests.
> 
> With best regards,
> Christian Hoene