Re: [rtcweb] Mute implementations (Re: More Comments ondraft-ietf-rtcweb-use-cases-and-requirements-07)

Harald Alvestrand <harald@alvestrand.no> Sat, 16 June 2012 20:34 UTC

Return-Path: <harald@alvestrand.no>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 61EDE21F84FD for <rtcweb@ietfa.amsl.com>; Sat, 16 Jun 2012 13:34:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.599
X-Spam-Level:
X-Spam-Status: No, score=-110.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FlOm3wRwAH9O for <rtcweb@ietfa.amsl.com>; Sat, 16 Jun 2012 13:34:13 -0700 (PDT)
Received: from eikenes.alvestrand.no (eikenes.alvestrand.no [158.38.152.233]) by ietfa.amsl.com (Postfix) with ESMTP id 63DF521F84FA for <rtcweb@ietf.org>; Sat, 16 Jun 2012 13:34:13 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 7332A39E07C; Sat, 16 Jun 2012 22:34:12 +0200 (CEST)
X-Virus-Scanned: Debian amavisd-new at eikenes.alvestrand.no
Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iL5e7rN4tKj5; Sat, 16 Jun 2012 22:34:10 +0200 (CEST)
Received: from [192.168.11.193] (c213-89-143-9.bredband.comhem.se [213.89.143.9]) by eikenes.alvestrand.no (Postfix) with ESMTPSA id 69ACF39E04C; Sat, 16 Jun 2012 22:34:10 +0200 (CEST)
Message-ID: <4FDCEDC2.2040903@alvestrand.no>
Date: Sat, 16 Jun 2012 22:34:10 +0200
From: Harald Alvestrand <harald@alvestrand.no>
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1
MIME-Version: 1.0
To: Jim Barnett <Jim.Barnett@genesyslab.com>
References: <5A2F246E-9D77-4B72-B3F1-681B00FA99FD@cisco.com><F3D4ABD6AB47084B84337CF4F3446A464B2F11B52E@ESESSCMS0362.eemea.ericsson.se><AAB6A6B6-BEC7-436C-B1EB-2C117596C7C6@cisco.com><F3D4ABD6AB47084B84337CF4F3446A464B2F2631AC@ESESSCMS0362.eemea.ericsson.se><9F33F40F6F2CD847824537F3C4E37DDF021B53@MCHP04MSX.global-ad.net><CA+9kkMCL1k3x2hYi632nDU2g6CtOq3O7risDi38+7FTeWkxcGA@mail.gmail.com><9F33F40F6F2CD847824537F3C4E37DDF0223ED@MCHP04MSX.global-ad.net><4FD89773.3090506@ericsson.com><9F33F40F6F2CD847824537F3C4E37DDF02594D@MCHP04MSX.global-ad.net><4FD9AD1E.6060504@alvestrand.no> <4FD9E2BE.2080907@ericsson.com><CAOJ7v-18EynuscX3tLRryzqKxyXwYzES9K4zroE-5=FL4o8WzA@mail.gmail.com><4FDB9E20.8070408@jesup.org><CABkgnnVqeyuayFsiA7YD+OFnyCi8tqode6VdypeaHKT+Z73KWw@mail.gmail.com><4FDBA6AC.2080107@jesup.org> <4FDCE4D4.7070909@alvestrand.no> <E17CAD772E76C742B645BD4DC602CD8106581BE5@NAHALD.us.int.genesyslab.com>
In-Reply-To: <E17CAD772E76C742B645BD4DC602CD8106581BE5@NAHALD.us.int.genesyslab.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Cc: rtcweb@ietf.org
Subject: Re: [rtcweb] Mute implementations (Re: More Comments ondraft-ietf-rtcweb-use-cases-and-requirements-07)
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Jun 2012 20:34:14 -0000

On 06/16/2012 10:07 PM, Jim Barnett wrote:
> Harald,
>    Your description below makes me nervous:  with getUserMedia, the user
> grants permission to _specific_ media streams (for example audio, or
> front camera, but not rear).
Yes, he grants permission to use of the camera. That permission persists 
until <we have to define this point in time>. There's some interesting 
language there now about "strong" and "weak" references to video streams.

>    If the system wants " to change the source
> of a video stream after its creation", shouldn't it be required to ask
> permission again?  In this case,  common sense says that it's not
> necessary, but that's because we're not changing to an arbitrary
> alternative the source (e.g., suddenly switching on the rear camera) but
> rather substituting some sort of 'null' or 'dummy' source.  Can we think
> of a different way to phrase it?
There are multiple ways to implement a requirement, as always.

One possibility is the simple one:

getUserMedia(..., function(strream) {
       outgoingstream = stream;
})
getStreamFromPicture(image, function(stream) {
       
outgoingstream.videoStreams(0).replaceContent(stream.videoStreams(0));
}..)

whatever the "replace" call does.
There may be a requirement that we keep a reference around that keeps 
the video stream, so that we can restore it, in order that the camera 
not close and can be used later:

getUserMedia(..., function(strream) {
       copyOfCameraStream = stream;
       outgoingstream = MediaStream(stream.audioTracks, stream.videoTracks);

})

getStreamFromPicture(image, function(stream) {
       
outgoingstream.videoStreams(0).replaceContent(stream.videoStreams(0));
}..)

restorePicture = function() {
      
outgoingtstream.videoStreams(0).replaceContent(copyOfCameraStream.video.Streams(0));
}

But there are surely multiple other ways to define it. What I want to 
make sure we have done first is to capture the requirement accurately.

>
> - Jim
>
> -----Original Message-----
> From: rtcweb-bounces@ietf.org [mailto:rtcweb-bounces@ietf.org] On Behalf
> Of Harald Alvestrand
> Sent: Saturday, June 16, 2012 3:56 PM
> To: rtcweb@ietf.org
> Subject: Re: [rtcweb] Mute implementations (Re: More Comments
> ondraft-ietf-rtcweb-use-cases-and-requirements-07)
>
> On 06/15/2012 11:18 PM, Randell Jesup wrote:
>> On 6/15/2012 4:49 PM, Martin Thomson wrote:
>>> On 15 June 2012 13:42, Randell Jesup<randell-ietf@jesup.org>   wrote:
>>>> I would also suggest that for video-mute that instead of stopping
>>>> video or sending black, the application should send a "mute image".
>>> The question I have is whether there is: a) a fixed image/video that
>>> is serialized to RTP b) video "comfort noise", or c) local mute
>>> image/video playback.  If it can't be c, then you might need to
>>> explain yourself.
>> Either we build it into the system (which I dislike - SetMuteImage()
>> or whathaveyou, though we could), or we put it in the application
>> domain and suggest or assist implementation.
>>
>> The simple way to handle Mute is to revector the input to a "mute" or
>> "hold" source, and they may not be the same (hold would have "Please
>> Hold" and music, perhaps; mute would have silence and "Muted" or a
>> camera with a circle/slash, etc).  You can also signal that out of
>> band to let the other end take smart action (remove muted video image,
>> display "muted" next to a name, etc).  This can be done via
>> DataChannels, for example.  You may then reinvite for Hold to make it
>> one-way, but that's mostly optional.
> The resulting requirements seem to me to be:
>
> 1) To be able to use a still image as source for a video stream
> 2) To be able to change the source of a video stream after its creation
>
> Both are requirements on the GetUserMedia API. Can we link these to a
> specific use case, or do we need to modify the use case document so that
> the requirements are clear?
>
>                Harald
>
>
>
>
> _______________________________________________
> rtcweb mailing list
> rtcweb@ietf.org
> https://www.ietf.org/mailman/listinfo/rtcweb