Re: [rtcweb] Mute implementations (Re: More Comments on draft-ietf-rtcweb-use-cases-and-requirements-07)

[Randell Jesup <[e-mail]>]

On 6/14/2012 3:36 PM, Justin Uberti wrote:
> For "mute", stopping sending of RTP may trigger concealment on the
> remote side. We currently send silent audio when muted to prevent this.
> For "hold", you are notifying the remote side (e.g. a=inactive), so it
> makes sense to stop sending RTP.

I would also suggest that for video-mute that instead of stopping video 
or sending black, the application should send a "mute image".  This 
could be done by changing the video source to be a fixed image (or small 
looped animation from a <video> element.  This greatly reduces confusion 
for users ("you just went black!" "where did you go?", etc)

This is largely an application issue, so long as you can redirect the 
source of input MediaStream for the outgoing video.

See the message stream around 2/13/2012 titled "Mute and MediaStream 
repointing".  One message:
http://www.w3.org/mid/[e-mail]

A quote:
------------------------------------

Since Hold is a slightly different concept, I believe 'muting' an 
outgoing stream should generate silence for audio, and some type of 
'muted' video message.  We used a camera icon with a circle-slash. 
Bandwidth for a static image can be dropped almost as much as you can 
for black.  I dislike (strongly) using 'black' to signal Mute because of 
codecs and the like in the path (and things like insertions of video 
'bugs' or other manipulation after the MediaStream generation (such as 
through the MediaStream Processing API proposal) might mess it up).

** This brings up an important point about repointing a MediaStream **

However, exactly what should be sent for Mute is probably an application 
issue - for video mute, the application should probably replace one 
source with a static (or not) image.  This speaks more to the 
MediaStream Processing API, since you don't want to renegotiate in order 
to change the tracks - you want to change the source of an existing track.

So, basically you want a MediaStream from getUserMedia() (call it 
MSCamera), a MediaStream from a static image or pre-recorded video (call 
it MSMute), and a MediaStream that can be either (MSMerged).  We can't 
send a track, just a stream, and again we don't want to renegotiate, so 
we need to create MSMerged from either MSCamera or MSMute.  The 
MediaStream Processing API should be able to handle this, much as it can 
handle smooth cross-fades of audio and stitching video clips together - 
see demos at:

http://robert.ocallahan.org/2012/01/mediastreams-processing-demos.html

This could be a canned video (or audio), a fixed image, or something 
algorithmically generated by the application using Workers ("You are 
second in line" "Your wait time is 3 minutes" "Please see our new fall 
line at http://bigstore.com/"), and equivalently-generated audio.

This would allow you to change the outgoing video source without 
triggering re-negotiations, which is very important.

Alternatively we'd need a way to override the source for individual 
tracks (and restore the default value), and a way to create a track as a 
discrete object, etc.

---------------------------------
end quote

-- 
Randell Jesup
[e-mail]