Re: [codec] #14: VAD and CNG?

["Christian Hoene" <[e-mail]>]

Hello,

I do not like the idea that a user should decide on parameters that he does not understand.
Can't we develop some intelligent, automatic decision on when to use DTX? For example, switch VAD on if
a) if bandwidth is limited (dynamically controlled)
b) if transmission energy needs to be saved on mobile devices (controlled/negotiated for both sender and receiver)
c) if a lot of background noise is present (sender initiated, no negotiation required)
d) if the background noise is important (sender initiated, e.g. emergency call, but then again, the sender can amplify the
background noise instead)
Shall the controlling/negotiation take place in-band or using SDP?

With best regards,

 Christian

---------------------------------------------------------------
Dr.-Ing. Christian Hoene
Interactive Communication Systems (ICS), University of Tübingen 
Sand 13, 72076 Tübingen, Germany, Phone [phone] 
http://www.net.uni-tuebingen.de/


>-----Original Message-----
>From: Michael Knappe [mailto:[e-mail]]
>Sent: Monday, May 24, 2010 6:06 PM
>To: Brian Rosen; [e-mail]; [e-mail]
>Subject: Re: [codec] #14: VAD and CNG?
>
>I agree that VAD (or DTX) needs to be negotiable.
>
>Mike
>
>
>On 5/24/10 7:48 AM, "Brian Rosen" <[e-mail]> wrote:
>
>> I hope we are assuming VAD (or DTX) is negotiable.  It is essential (for
>> emergency calls) that VAD is disabled.  While in many cases the endpoint
>> will know that it is in an emergency call, and disable VAD, it is not always
>> possible to know.
>>
>> Brian
>>
>>
>> On 5/24/10 10:22 AM, "codec issue tracker" <[e-mail]> wrote:
>>
>>> #14: VAD and
>>> CNG?
>> ------------------------------------+------------------------------------
>>> ---
>>  Reporter:  hoene@Š                 |       Owner:
>>      Type:  defect
>>> |      Status:  new
>>  Priority:  major                   |   Milestone:
>>>
>> Component:  requirements            |     Version:
>>  Severity:  -
>>> |    Keywords:
>>>
>> ------------------------------------+---------------------------------------
>>
>>>
>> Comment(by hoene@Š):
>>
>>  [Raymond]: I think comfort noise is more than just to
>>> let the telephone
>>  user know that the connection is not dead.  If voice
>>> packets are sent only
>>  during active voice regions of the signal (in DTX) and
>>> not during
>>  silence/background noise regions, and if there is audible
>>> background noise
>>  and comfort noise is not added during background noise
>>> regions, at the
>>  receiving end the background noise will be "on" only during
>>> active voice
>>  and "off" otherwise.  This frequent on-off switching will make
>>> the
>>  background noise sound unnatural and will bother many users.  Adding
>>
>>> comfort noise during background noise regions will make such background
>>  noise
>>> sound more natural.
>>
>>  [Benjamin]:  The cheapest solution, of course, is
>>> transmit-side activity
>>  detection.
>>  Maybe we need to specify a way for a
>>> receiver to request that the
>>  transmitter employ (or not employ) VAD.
>>
>>
>>
>>> [Benjamin]:
>>  I know that CELT makes decoder VAD very efficient, but how is
>>> decoder VAD
>>  better than encoder VAD?  Encoder VAD saves even more CPU, saves
>>
>>> bandwidth, and enables easier jitter buffering.
>>  Are you thinking about some
>>> sort of adaptive thresholding that requires
>>  knowing all streams' volume
>>> levels?
>>  Anyway, VAD can run on both encode and decode sides at the same
>>> time.
>>
>>
>>  [Stephen]: It might be valuable to be able to obtain the VAD without
>>
>>> having to decrypt the bitstream, since that also can become a problem as
>>  the
>>> number of streams grows.
>>  There is a security consideration (traffic
>>> analysis), however, it still
>>  might be worth thinking about.
>>
>>  [JM]:
>>> I know
>>> that CELT makes decoder VAD very efficient,
>>  Not only CELT. You can do that
>>> with an LPC-based codec too.
>>
>>> but how is decoder VAD
>>> better than encoder
>>> VAD?  Encoder VAD saves even more CPU, saves
>>> bandwidth, and enables easier
>>> jitter buffering.
>>
>>  There's a few reasons why I think decoder-side is better:
>>
>>> - The decision for an encoder-size VAD would take some amount of space in the
>>> bit-stream
>>  - If we make an encode-size VAD mandatory, then all encoders will
>>> have to
>>  spend the CPU cycles, even when it's not needed. If it's not
>>> mandatory,
>>  then the decoder cannot rely on it, so it still needs to implement
>>> a VAD
>>  - A decoder VAD does not need to be specified in an exact way, so
>>
>>> implementers can choose different implementations depending on that
>>
>>> information they need.
>>  - You cannot "game" a decode-size VAD.
>>
>>> Are you
>>> thinking about some sort of adaptive thresholding that
>>> requires knowing all
>>> streams' volume levels?
>>
>>  Well, knowing the relative amplitudes of each stream
>>> can allow you to take
>>  more intelligent decisions, e.g. when you have to
>>> choose the "most active
>>  speaker". That's something you can't really get from
>>> an encoder VAD.
>>
>>> Anyway, VAD can run on both encode and decode sides at the
>>> same time.
>>
>>  That would just mean nobody would bother implementing the encode
>>> side.
>>
>>
>>  [Benjamin]:
>>
>>  I think I failed to communicate that by VAD I mean _not
>>> sending packets_
>>  during inactivity.  For the packets that are sent, the
>>> overhead should
>>  average much less than 1 bit per frame.
>>
>>  I'm not suggesting
>>> sending 200 packets a second containing a flag
>>  indicating no voice activity,
>>> followed by carefully coded background
>>  noise.  That would be silly.
>>
>>> - If
>>> we make an encode-size VAD mandatory, then all encoders will have
>>> to spend
>>> the CPU cycles, even when it's not needed. If it's not
>>> mandatory, then the
>>> decoder cannot rely on it, so it still needs to
>>> implement a VAD
>>
>>  I don't
>>> see this as "mandatory".  The encoder can turn off VAD, and
>>  probably should
>>> for full-quality applications.
>>
>>> - A decoder VAD does not need to be
>>> specified in an exact way, so
>>> implementers can choose different
>>> implementations depending on that
>>> information they need.
>>
>>  The only thing
>>> that needs exact specification is the signalling.  The
>>  encoder may use it or
>>> not use it as it pleases.
>>
>>> - You cannot "game" a decode-size VAD.
>>
>>  I don't
>>> know what this means.
>>
>>>> Are you thinking about some sort of adaptive
>>> thresholding that
>>>> requires knowing all streams' volume levels?
>>>
>>> Well,
>>> knowing the relative amplitudes of each stream can allow you to
>>> take more
>>> intelligent decisions, e.g. when you have to choose the
>>> "most active
>>> speaker". That's something you can't really get from an
>>  encoder VAD.
>>>
>>>>
>>> Anyway, VAD can run on both encode and decode sides at the same time.
>>>
>>>
>>> That would just mean nobody would bother implementing the encode side.
>>
>>  I
>>> expect encode-side VAD on a conference call to save more than a factor
>>  of 2
>>> in bandwidth, which makes it very desirable, especially for large
>>
>>> deployments.  People will use it to save bandwidth (especially if it's on by
>>> default in the reference implementation).  The decode-side CPU savings
>>  are
>>> just a minor bonus side-effect.
>>
>>  [JM]:
>>  What you're describing is called DTX
>>> (discontinuous transmission). This
>>  can be useful feature, but it's very
>>> different from what we were
>>  originally talking about in terms of conference
>>> mixing.
>>
>>  [Ben]: Oops. Right.  What I'm trying to say is that DTX, based on
>>> encoder-
>>  side VAD, also greatly reduces the (average) computational burden on
>>> a
>>  conference mixer.  Of course, if everyone's really talking at once then
>>
>>> VAD can't help.
>>
>>  [Roman]: There is one more application to efficiently
>>> combining pre-
>>  encoded
>>  audio: playing announcements or recorded audio.
>>> Standard network or IVR
>>  announcements can be encoded once and efficiently
>>> inserted or combined
>>  into audio stream. If pre-encoded audio is supported and
>>> the client
>>  supports AVT tones, it is trivial to develop a very efficient IVR
>>> server
>>  which does not require any CODEC encoding or decoding.
>>
>>  Efficient
>>> decoder side VAD is also very helpful in case of speech
>>  recognition, where it
>>> allows to save cycles in end-pointer. This way audio
>>  only needs to be decoded
>>> and passed to the speech recognition system only
>>  when voice is present.
>>
>>
>>> Bottom line, if we have both efficient decoder side VAD and combining pre-
>>
>>> encoded audio we can develop some very efficient VXML servers, voice mail and
>>> IVR system, not just conferencing servers.
>>
>>  Of course, this works well only
>>> if the background noise is relatively
>>  stationary.  If the background noise is
>>> dynamically changing, then comfort
>>  noise can't really sound like the true
>>> background noise.  Today I was put
>>  on hold in a phone call with music
>>> playing.  Apparently the system treated
>>  some parts of the music as background
>>> noise and replaced them with comfort
>>  noise. The result is pretty annoying, or
>>> amusing, depending on which way
>>  you look at it.
>>
>>  Therefore, I think comfort
>>> noise has its value if DTX is used and the
>>  background noise is fairly
>>> stationary, but if the background noise or
>>  music is changing dynamically and
>>> high audio quality is desired, then the
>>  DTX and comfort noise should be
>>> turned off and all parts of the signal
>>  need to be transmitted.