Re: [codec] #15: Efficiently combine pre-encoded audio

Brian Rosen <br@brianrosen.net> Wed, 12 May 2010 15:36 UTC

Return-Path: <br@brianrosen.net>
X-Original-To: codec@core3.amsl.com
Delivered-To: codec@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5A4FC28C187 for <codec@core3.amsl.com>; Wed, 12 May 2010 08:36:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.978
X-Spam-Level:
X-Spam-Status: No, score=0.978 tagged_above=-999 required=5 tests=[AWL=-1.353, BAYES_50=0.001, IP_NOT_FRIENDLY=0.334, J_CHICKENPOX_72=0.6, MIME_QP_LONG_LINE=1.396]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TMZgEyR7FQXa for <codec@core3.amsl.com>; Wed, 12 May 2010 08:36:52 -0700 (PDT)
Received: from ebru.winwebhosting.com (ebru.winwebhosting.com [67.18.150.162]) by core3.amsl.com (Postfix) with ESMTP id CB25428C2D4 for <codec@ietf.org>; Wed, 12 May 2010 08:25:34 -0700 (PDT)
Received: from neustargw.va.neustar.com ([209.173.53.233] helo=[192.168.128.74]) by ebru.winwebhosting.com with esmtpa (Exim 4.69) (envelope-from <br@brianrosen.net>) id 1OCDoC-0006bL-AO; Wed, 12 May 2010 10:25:16 -0500
User-Agent: Microsoft-Entourage/12.24.0.100205
Date: Wed, 12 May 2010 11:25:17 -0400
From: Brian Rosen <br@brianrosen.net>
To: Cullen Jennings <fluffy@cisco.com>, <codec@ietf.org>
Message-ID: <C810409D.33E4E%br@brianrosen.net>
Thread-Topic: [codec] #15: Efficiently combine pre-encoded audio
Thread-Index: Acrx51NBPNPYs3gITUaVqNolKeFZIg==
In-Reply-To: <1F68067D-33B9-4F0C-B31B-B3A56A72DBA4@cisco.com>
Mime-version: 1.0
Content-type: text/plain; charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - ebru.winwebhosting.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - brianrosen.net
Subject: Re: [codec] #15: Efficiently combine pre-encoded audio
X-BeenThere: codec@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Codec WG <codec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/codec>
List-Post: <mailto:codec@ietf.org>
List-Help: <mailto:codec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/codec>, <mailto:codec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 12 May 2010 15:36:53 -0000

Excellent idea.  Been there, never really did it.  It's complex.
Effectively, you need a distributed adaptive threshold mechanism.

However, if you had it, user experience in multispeaker environments gets a
win.

Brian


On 5/12/10 11:05 AM, "Cullen Jennings" <fluffy@cisco.com> wrote:

> 
> For conference bridges, it's probably more important to be able to decide who
> the active speakers are with low CPU complexity than the actually act of
> mixing the the selected speakers. Consider a typical call with 7 people who
> might be speakers and  the 3 most active are selected and mixed. In many
> systems today, most the MIPS goes to decoding all 7 streams to do speaker
> detection before the resulting 4 streams are formed and encoded. If there was
> a cheap way to figure out who the active speakers were without doing a full
> decode of all 7 streams, that would be sort of nice the for conferences
> bridges. 
> 
> 
> 
> 
> On May 1, 2010, at 8:24 AM, codec issue tracker wrote:
> 
>> #15: Efficiently combine pre-encoded audio
>> ------------------------------------+---------------------------------------
>> Reporter:  hoene@Š                 |       Owner:
>>     Type:  enhancement             |      Status:  new
>> Priority:  minor                   |   Milestone:
>> Component:  requirements            |     Version:
>> Severity:  Active WG Document      |    Keywords:
>> ------------------------------------+---------------------------------------
>> 
>> Comment(by hoene@Š):
>> 
>> [Colin:]
>>> [...] conferences implemented using multicast require
>>> end system mixing of potentially large numbers of active audio
>>> streams, whereas those implemented using conference bridges do the
>>> mixing in a single central location, and generally suppress all but
>>> one speaker. The differences in mixing and the number of simultaneous
>>> active streams that might be received potentially affect the design of
>>> the codec.
>> 
>> [Raymond]: I would like to take this opportunity to express my view that
>> although codec complexity isn¹t much of an issue for PC-to-PC calls where
>> there are GHz of processing power available, the codec complexity is an
>> important issue in certain application scenarios.  The following are just
>> some examples.
>> 1) If a conference bridge has to decode a large number of voice channels,
>> mix, and re-encode, and if compressed-domain mixing cannot be done (which
>> is usually the case), then it is important to keep the decoder complexity
>> low.
>> 
>> [JM]: The decoder complexity is very important. Not only because of mixing
>> issue, but also because the decoder is generally not allowed to take
>> shortcuts to save on complexity (unlike the encoder). As for compressed-
>> domain mixing, as you say it is not always available, but *if* we can do
>> it (even if only partially), then that can result in a "free" reduction in
>> decoder complexity for mixing.
>> 
>> [Christian]: Scalable [conferencing]
>> -       Receiver side activity detection for music and voice having low
>> complexity (for the conference bridge)
>> -       Efficient mixing of two to four(?) active flows (is this
>> achievable without the complete process of decoding and encoding again?)
>> 
>> -- 
>> Ticket URL: <http://trac.tools.ietf.org/wg/codec/trac/ticket/15#comment:1>
>> codec <http://tools.ietf.org/codec/>
>> 
>> _______________________________________________
>> codec mailing list
>> codec@ietf.org
>> https://www.ietf.org/mailman/listinfo/codec
> 
> 
> Cullen Jennings
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/index.html
> 
> 
> 
> _______________________________________________
> codec mailing list
> codec@ietf.org
> https://www.ietf.org/mailman/listinfo/codec