Re: [codec] #15: Efficiently combine pre-encoded audio

["Benjamin M. Schwartz" <[e-mail]>]

On 05/12/2010 12:03 PM, Jean-Marc Valin wrote:
> Benjamin M. Schwartz wrote:
>> but how is decoder VAD
>> better than encoder VAD? Encoder VAD saves even more CPU, saves
>> bandwidth, and enables easier jitter buffering.
>
> There's a few reasons why I think decoder-side is better:
> - The decision for an encoder-size VAD would take some amount of space
> in the bit-stream

I think I failed to communicate that by VAD I mean _not sending packets_ 
during inactivity.  For the packets that are sent, the overhead should 
average much less than 1 bit per frame.

I'm not suggesting sending 200 packets a second containing a flag 
indicating no voice activity, followed by carefully coded background 
noise.  That would be silly.

> - If we make an encode-size VAD mandatory, then all encoders will have
> to spend the CPU cycles, even when it's not needed. If it's not
> mandatory, then the decoder cannot rely on it, so it still needs to
> implement a VAD

I don't see this as "mandatory".  The encoder can turn off VAD, and 
probably should for full-quality applications.

> - A decoder VAD does not need to be specified in an exact way, so
> implementers can choose different implementations depending on that
> information they need.

The only thing that needs exact specification is the signalling.  The 
encoder may use it or not use it as it pleases.

> - You cannot "game" a decode-size VAD.

I don't know what this means.

>> Are you thinking about some sort of adaptive thresholding that requires
>> knowing all streams' volume levels?
>
> Well, knowing the relative amplitudes of each stream can allow you to
> take more intelligent decisions, e.g. when you have to choose the "most
> active speaker". That's something you can't really get from an encoder VAD.
>
>> Anyway, VAD can run on both encode and decode sides at the same time.
>
> That would just mean nobody would bother implementing the encode side.

I expect encode-side VAD on a conference call to save more than a factor 
of 2 in bandwidth, which makes it very desirable, especially for large 
deployments.  People will use it to save bandwidth (especially if it's on 
by default in the reference implementation).  The decode-side CPU savings 
are just a minor bonus side-effect.

--Ben