Re: [rtcweb] Mute implementations (Re: More Comments on draft-ietf-rtcweb-use-cases-and-requirements-07)

[Stefan Hakansson LK <[e-mail]>]

On 06/14/2012 10:01 PM, Hutton, Andrew wrote:
> I agree with Justin that mute is a local action and should not stop the
> RTP or change the state of the remote track.
>
> Regards
> Andy

First, a short explanation of (my understanding of) what is in the API now:

* each track has a muted/unmuted and a enabled/disabled state
* enable state can be controlled by the application
* muted state can only be controlled by the browser
* muted overrides enabled (so a track that is enabled will still not 
play if it is muted)

The idea is that "mute" state is out of the control of the app (so the 
browser can override - and the user if the browser chrome allows). There 
has been some discussion indicating that the above is not sufficient and 
that we need to add more.

I think we're talking about different "mutes" in this thread.

One that is that you mute using e.g. a button on your headset. This 
would lead to audio samples with amplitude of zero being generated. This 
should not change any state on the track (locally or remotely) IMO, and 
silence/cn should be sent in the RTP.

Another is that the app has a "mute" button. Clicking this would 
(depending of the app) lead to the corresponding track being "disabled" 
in the local MediaStream. For the remote MediaStream, this is basically 
the same as "mute in the browser chrome" in the local case (since the 
source stops producing data), and I think the state there should change 
to "muted".

A third is that you mute the device via the browser chrome (I assume 
this possibility will exist). In this case the track state changes to 
muted, and I really think this should propagate to the other end of a 
PeerConnection.

Stefan


>
>
>
> On 14 Jun 2012, at 20:37, "Justin Uberti" <[e-mail]
> <mailto:[e-mail]>> wrote:
>
>> For "mute", stopping sending of RTP may trigger concealment on the
>> remote side. We currently send silent audio when muted to prevent this.
>> For "hold", you are notifying the remote side (e.g. a=inactive), so it
>> makes sense to stop sending RTP.
>>
>> On Thu, Jun 14, 2012 at 9:10 AM, Stefan Hakansson LK
>> <[e-mail]
>> <mailto:[e-mail]>> wrote:
>>
>>     On 06/14/2012 11:21 AM, Harald Alvestrand wrote:
>>
>>         Changing the subject line again, since we're discussing how to do
>>         mute, not whether to do it.....
>>
>>         On 06/14/2012 11:15 AM, Hutton, Andrew wrote:
>>
>>             On 13 June 2012 15:37 Stefan Hakansson wrote:
>>
>>                 This is my personal understanding:
>>
>>                 - the API (and this part has been there for a long
>>                 time) allows
>>                 the application to disable/enable individual tracks of
>>                 MediaStreams - So say you have a MediaStream (one
>>                 audio track,
>>                 one video track in this example) that is rendered
>>                 using a video
>>                 element; disabling audio would lead to silence,
>>                 disabling video
>>                 to that the video stops - The question is: what should
>>                 happen if
>>                 the MediaStream is connected to a PeerConnection and
>>                 sent to a
>>                 peer? - using recvonly does not seem appropriate as
>>                 that would
>>                 stop all ssrc:s belonging to the same m-line - and the
>>                 app only
>>                 disabled one - one option could be to simply stop
>>                 sending the RTP
>>                 packets for this track. I don't know how reliable that
>>                 would be.
>>
>>             In the case of outgoing media then I think mute should simply
>>             disconnect the source (e.g. microphone) and there is no
>>             impact on
>>             RTP other than depending on the codec silence will be
>>             transmitted
>>             or SID frame etc. There is also no impact on RTCP which
>>             continues.
>>
>>             This is different to setting the direction to be recvonly
>>             which
>>             would completely stop the outgoing RTP packets and would
>>             normally
>>             be signaled to the remote party. This should of course be
>>             achieved
>>             using setLocalDescription.
>>
>>
>>         Just to add to the complexity of the "mute" case:
>>
>>         When an audio stream is muted, should there be comfort noise
>>         sent? If
>>         yes, what should determine the noise level?
>>
>>
>>     To me it feels as that ideally we should stop sending RTP (and
>>     make the other end know that the track is disabled). If we go to
>>     comfort noise state, the track in the MediaStream at the other end
>>     would still be active (even though what is played could be very
>>     low level comfort noise), but to reflect the local state the
>>     remote track should go to "muted", shouldn't it?
>>
>>
>>
>>
>>         (people who've never had to deal with comfort noise can safely
>>         tune
>>         out at this point)
>>
>>
>>
>>                 - I think there has been proposals signaling this in
>>                 SDP or in
>>                 RTCP, but I don't think there was ever any conclusion
>>
>>             If we are talking about a local mute then I don't see any
>>             need to
>>             signal this but again this is different to changing the
>>             direction
>>             attribute in SDP.
>>
>>                 For the other part, muting incoming media, this is
>>                 simple: the
>>                 app (at receiving browser) could just disable the
>>                 tracks it wants
>>                 in the incoming MediaStreams. A related question is if
>>                 this
>>                 should be signaled back to the sender (so it could
>>                 stop sending).
>>                 This is essentially a question of
>>                 efficiency/optimization, and
>>                 nothing stops the app from signaling that now using an app
>>                 specific protocol (perhaps carried over the data
>>                 channel), but
>>                 perhaps this should be brought into SDP or RTCP.
>>
>>             If the application wants to ask the remote party to pause
>>             sending
>>             of RTP then it should surely do this in SDP by setting the
>>             direction attribute appropriately (i.e. sendonly or inactive).
>>
>>             Regards Andy
>>
>>             _________________________________________________ rtcweb
>>             mailing
>>             list [e-mail] <mailto:[e-mail]>
>>             https://www.ietf.org/mailman/__listinfo/rtcweb
>>             <https://www.ietf.org/mailman/listinfo/rtcweb>
>>
>>
>>         _________________________________________________ rtcweb
>>         mailing list
>>         [e-mail] <mailto:[e-mail]>
>>         https://www.ietf.org/mailman/__listinfo/rtcweb
>>         <https://www.ietf.org/mailman/listinfo/rtcweb>
>>
>>
>>     _________________________________________________
>>     rtcweb mailing list
>>     [e-mail] <mailto:[e-mail]>
>>     https://www.ietf.org/mailman/__listinfo/rtcweb
>>     <https://www.ietf.org/mailman/listinfo/rtcweb>
>>
>>
>> _______________________________________________
>> rtcweb mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/rtcweb