Return-Path: X-Original-To: codec@core3.amsl.com Delivered-To: codec@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 08AB53A6932 for ; Wed, 12 May 2010 08:58:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.141 X-Spam-Level: X-Spam-Status: No, score=-0.141 tagged_above=-999 required=5 tests=[AWL=-0.743, BAYES_50=0.001, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gLnjBoOGYW0c for ; Wed, 12 May 2010 08:58:51 -0700 (PDT) Received: from mail-ww0-f44.google.com (mail-ww0-f44.google.com [74.125.82.44]) by core3.amsl.com (Postfix) with ESMTP id EE2143A690C for ; Wed, 12 May 2010 08:46:03 -0700 (PDT) Received: by wwb28 with SMTP id 28so108315wwb.31 for ; Wed, 12 May 2010 08:45:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=Y9wnSe7Ll29KcGNuN6nBqTmg9regpb3bTwcTO0eoa7s=; b=n7ZZemVllNMLknCEzZPSQzEe46kXBMJT8DZyFtPAdz214L+9mLYH3aRXC0FcIYdj0H R0TxiMZ0KxN4JV5qO9cRqI0vaCiD+s4RayYC7rAtsaHANZt9Fp5z6VqTep0S9SctRldV 5UsGjdD/C2eyWd7P8NvrTWqC20NZ/4Np9ChY0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=inXWKReFtDnO6l5BDuz++L381OYHfn5mR6WzPeRky1oNIh9DNjYQVxwA2dhO5+rhHu 13iI9rBj8PigD3pz7njHYAuhHK9+nOwjg3jt5Na+fnib6evhWVG+ECpnZZMVzjRs/dg6 8d2PRnMvwN2s6sq36BxunuTv9GgDJ9KaGWIS8= MIME-Version: 1.0 Received: by 10.216.87.132 with SMTP id y4mr4217495wee.174.1273679150529; Wed, 12 May 2010 08:45:50 -0700 (PDT) Received: by 10.216.23.5 with HTTP; Wed, 12 May 2010 08:45:50 -0700 (PDT) In-Reply-To: References: <1F68067D-33B9-4F0C-B31B-B3A56A72DBA4@cisco.com> Date: Wed, 12 May 2010 11:45:50 -0400 Message-ID: From: stephen botzko To: Brian Rosen Content-Type: multipart/alternative; boundary=0016e6d99ba7533ca904866789ca Cc: codec@ietf.org Subject: Re: [codec] #15: Efficiently combine pre-encoded audio X-BeenThere: codec@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Codec WG List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2010 15:58:53 -0000 --0016e6d99ba7533ca904866789ca Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable The adaptive threshold doesn't have to be distributed, as the conference bridge is selecting the highest scores. You do need a consistent way to compute the scores in the endpoints, ideall= y using a method which is not simply energy. I realize the bridge can alternatively generate scores from bitstream information; I am thinking that is equivalent to including a metric in the RTP payload. Stephen Botzko On Wed, May 12, 2010 at 11:25 AM, Brian Rosen wrote: > Excellent idea. Been there, never really did it. It's complex. > Effectively, you need a distributed adaptive threshold mechanism. > > However, if you had it, user experience in multispeaker environments gets= a > win. > > Brian > > > On 5/12/10 11:05 AM, "Cullen Jennings" wrote: > > > > > For conference bridges, it's probably more important to be able to deci= de > who > > the active speakers are with low CPU complexity than the actually act o= f > > mixing the the selected speakers. Consider a typical call with 7 people > who > > might be speakers and the 3 most active are selected and mixed. In man= y > > systems today, most the MIPS goes to decoding all 7 streams to do speak= er > > detection before the resulting 4 streams are formed and encoded. If the= re > was > > a cheap way to figure out who the active speakers were without doing a > full > > decode of all 7 streams, that would be sort of nice the for conferences > > bridges. > > > > > > > > > > On May 1, 2010, at 8:24 AM, codec issue tracker wrote: > > > >> #15: Efficiently combine pre-encoded audio > >> > ------------------------------------+------------------------------------= --- > >> Reporter: hoene@=C5=A0 | Owner: > >> Type: enhancement | Status: new > >> Priority: minor | Milestone: > >> Component: requirements | Version: > >> Severity: Active WG Document | Keywords: > >> > ------------------------------------+------------------------------------= --- > >> > >> Comment(by hoene@=C5=A0): > >> > >> [Colin:] > >>> [...] conferences implemented using multicast require > >>> end system mixing of potentially large numbers of active audio > >>> streams, whereas those implemented using conference bridges do the > >>> mixing in a single central location, and generally suppress all but > >>> one speaker. The differences in mixing and the number of simultaneous > >>> active streams that might be received potentially affect the design o= f > >>> the codec. > >> > >> [Raymond]: I would like to take this opportunity to express my view th= at > >> although codec complexity isn=C2=B9t much of an issue for PC-to-PC cal= ls > where > >> there are GHz of processing power available, the codec complexity is a= n > >> important issue in certain application scenarios. The following are > just > >> some examples. > >> 1) If a conference bridge has to decode a large number of voice > channels, > >> mix, and re-encode, and if compressed-domain mixing cannot be done > (which > >> is usually the case), then it is important to keep the decoder > complexity > >> low. > >> > >> [JM]: The decoder complexity is very important. Not only because of > mixing > >> issue, but also because the decoder is generally not allowed to take > >> shortcuts to save on complexity (unlike the encoder). As for compresse= d- > >> domain mixing, as you say it is not always available, but *if* we can = do > >> it (even if only partially), then that can result in a "free" reductio= n > in > >> decoder complexity for mixing. > >> > >> [Christian]: Scalable [conferencing] > >> - Receiver side activity detection for music and voice having lo= w > >> complexity (for the conference bridge) > >> - Efficient mixing of two to four(?) active flows (is this > >> achievable without the complete process of decoding and encoding again= ?) > >> > >> -- > >> Ticket URL: < > http://trac.tools.ietf.org/wg/codec/trac/ticket/15#comment:1> > >> codec > >> > >> _______________________________________________ > >> codec mailing list > >> codec@ietf.org > >> https://www.ietf.org/mailman/listinfo/codec > > > > > > Cullen Jennings > > For corporate legal information go to: > > http://www.cisco.com/web/about/doing_business/legal/cri/index.html > > > > > > > > _______________________________________________ > > codec mailing list > > codec@ietf.org > > https://www.ietf.org/mailman/listinfo/codec > > > _______________________________________________ > codec mailing list > codec@ietf.org > https://www.ietf.org/mailman/listinfo/codec > --0016e6d99ba7533ca904866789ca Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable The adaptive threshold doesn't have to be distributed, as the conferenc= e bridge is selecting the highest scores.=C2=A0

You do need a consi= stent way to compute the scores in the endpoints, ideally using a method wh= ich is not simply energy.

I realize the bridge can alternatively generate scores from bitstream i= nformation; I am thinking that is equivalent to including a metric in the R= TP payload.

Stephen Botzko

On Wed,= May 12, 2010 at 11:25 AM, Brian Rosen <br@brianrosen.net> wrote:
Excellent idea. = =C2=A0Been there, never really did it. =C2=A0It's complex.
Effectively, you need a distributed adaptive threshold mechanism.

However, if you had it, user experience in multispeaker environments gets a=
win.

Brian


On 5/12/10 11:05 AM, "Cullen Jennings" <fluffy@cisco.com> wrote:

>
> For conference bridges, it's probably more important to be able to= decide who
> the active speakers are with low CPU complexity than the actually act = of
> mixing the the selected speakers. Consider a typical call with 7 peopl= e who
> might be speakers and =C2=A0the 3 most active are selected and mixed. = In many
> systems today, most the MIPS goes to decoding all 7 streams to do spea= ker
> detection before the resulting 4 streams are formed and encoded. If th= ere was
> a cheap way to figure out who the active speakers were without doing a= full
> decode of all 7 streams, that would be sort of nice the for conference= s
> bridges.
>
>
>
>
> On May 1, 2010, at 8:24 AM, codec issue tracker wrote:
>
>> #15: Efficiently combine pre-encoded audio
>> ------------------------------------+-----------------------------= ----------
>> Reporter: =C2=A0hoene@=C5=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2=A0 Owner:
>> =C2=A0 =C2=A0 Type: =C2=A0enhancement =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2=A0Status: =C2=A0new
>> Priority: =C2=A0minor =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 | =C2=A0 Milestone:
>> Component: =C2=A0requirements =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0| =C2=A0 =C2=A0 Version:
>> Severity: =C2=A0Active WG Document =C2=A0 =C2=A0 =C2=A0| =C2=A0 = =C2=A0Keywords:
>> ------------------------------------+-----------------------------= ----------
>>
>> Comment(by hoene@=C5=A0):
>>
>> [Colin:]
>>> [...] conferences implemented using multicast require
>>> end system mixing of potentially large numbers of active audio=
>>> streams, whereas those implemented using conference bridges do= the
>>> mixing in a single central location, and generally suppress al= l but
>>> one speaker. The differences in mixing and the number of simul= taneous
>>> active streams that might be received potentially affect the d= esign of
>>> the codec.
>>
>> [Raymond]: I would like to take this opportunity to express my vie= w that
>> although codec complexity isn=C2=B9t much of an issue for PC-to-PC= calls where
>> there are GHz of processing power available, the codec complexity = is an
>> important issue in certain application scenarios. =C2=A0The follow= ing are just
>> some examples.
>> 1) If a conference bridge has to decode a large number of voice ch= annels,
>> mix, and re-encode, and if compressed-domain mixing cannot be done= (which
>> is usually the case), then it is important to keep the decoder com= plexity
>> low.
>>
>> [JM]: The decoder complexity is very important. Not only because o= f mixing
>> issue, but also because the decoder is generally not allowed to ta= ke
>> shortcuts to save on complexity (unlike the encoder). As for compr= essed-
>> domain mixing, as you say it is not always available, but *if* we = can do
>> it (even if only partially), then that can result in a "free&= quot; reduction in
>> decoder complexity for mixing.
>>
>> [Christian]: Scalable [conferencing]
>> - =C2=A0 =C2=A0 =C2=A0 Receiver side activity detection for music = and voice having low
>> complexity (for the conference bridge)
>> - =C2=A0 =C2=A0 =C2=A0 Efficient mixing of two to four(?) active f= lows (is this
>> achievable without the complete process of decoding and encoding a= gain?)
>>
>> --
>> Ticket URL: <http://trac.tools.ietf.org/wg/code= c/trac/ticket/15#comment:1>
>> codec <http://tools.ietf.org/codec/>
>>
>> _______________________________________________
>> codec mailing list
>> codec@ietf.org
>> https://www.ietf.org/mailman/listinfo/codec
>
>
> Cullen Jennings
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/le= gal/cri/index.html
>
>
>
> _______________________________________________
> codec mailing list
> codec@ietf.org
> https://www.ietf.org/mailman/listinfo/codec


_______________________________________________
codec mailing list
codec@ietf.org
h= ttps://www.ietf.org/mailman/listinfo/codec

--0016e6d99ba7533ca904866789ca--