Re: [rtcweb] Proposal for a JS API for NoPlan (adding multiple sources without encoding them in SDP)

Peter Thatcher <pthatcher@google.com> Mon, 17 June 2013 19:54 UTC

Return-Path: <pthatcher@google.com>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC08121F9CF3 for <rtcweb@ietfa.amsl.com>; Mon, 17 Jun 2013 12:54:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.406
X-Spam-Level:
X-Spam-Status: No, score=-1.406 tagged_above=-999 required=5 tests=[AWL=-0.330, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, J_CHICKENPOX_55=0.6, MIME_8BIT_HEADER=0.3]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A-4+bj2xFeU4 for <rtcweb@ietfa.amsl.com>; Mon, 17 Jun 2013 12:54:20 -0700 (PDT)
Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) by ietfa.amsl.com (Postfix) with ESMTP id 5490D21F9CCA for <rtcweb@ietf.org>; Mon, 17 Jun 2013 12:54:16 -0700 (PDT)
Received: by mail-pd0-f180.google.com with SMTP id 10so3096328pdi.25 for <rtcweb@ietf.org>; Mon, 17 Jun 2013 12:54:16 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=LuzzBZijFelJwXW/l6RBougBrSbdEdLEmHfrrc8iFHE=; b=IYtxzj1KsTRy3zyLzu+LPBJ9chC06vAoO33IQ97KOZDTMvG/NuOOuztel7S1HJ7fDA xCOqjsePvaZcTppfl7hoCf+VDbOQS+Abg6vOw3IjhsaUd3ISPMNqAbeOc58/xF3FosTv LDEeHrFlVSImuFUTEuZ6bNWtkIrFQiUPNfWVxHAOPc10VK8ubwee5E5ft3aUxI3omUsq wDKQhhdLq5Bc9pFRZxFzB2ekWEB2WUAlfLL7wsC86wYiKB5ZmV+0zkH03GXR9ad+JjSD UkN1tFhpeu5D7tgsgog0bae7yzdvhkPleJvGPr2MsxFpZS6vQi8MaheUVy0SU2bm/CuQ eaBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=LuzzBZijFelJwXW/l6RBougBrSbdEdLEmHfrrc8iFHE=; b=YuJbYpBbTZF2adWg0tlQauCjajd6xR5rv0RNu6tQlb379OxqjDwU015Y4nhfJoVUG1 N043if7HoqDvUBk/bgJH70GZUBM2V5EXSUKPsdkHIywZx0OSP7pTQVkiLgR1idOuMflJ dpTW+KBe/w3Cfrs/llHCIFi6v5v5hqPxCsBoAyVxJD4b2Cf+kscHKk8VDTKNyYZrpv7C UAd+mX0rNGSdGRDQvWYcPgdKG5qbooTlCfik6vnfR2j4vaeRZaM410jIejNhiYXzqhFf k2fBBQ3TL+K5mu+FOs9sOnguVMNjDREVxCWyDBx2XxWrGbQwk4+Ma7bC4RzEgiBbePDM FQGA==
X-Received: by 10.66.251.202 with SMTP id zm10mr14317328pac.53.1371498856022; Mon, 17 Jun 2013 12:54:16 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.66.88.8 with HTTP; Mon, 17 Jun 2013 12:53:35 -0700 (PDT)
In-Reply-To: <CABw3bnPprVKPsFqJkDLRKeCmWDcGm-jCqezjDcBD4Y329RoROA@mail.gmail.com>
References: <CAJrXDUHdoxLTsofiwLBdwBNnCCkCBgjSdbmLaXrNEPODMrsSVA@mail.gmail.com> <1447FA0C20ED5147A1AA0EF02890A64B1C2FC071@ESESSMB209.ericsson.se> <CAJrXDUEL1ynci_bwLydYgvWD=+FQKhurzV3vC0X4LrgBjUMrkA@mail.gmail.com> <57A15FAF9E58F841B2B1651FFE16D28104C918@GENSJZMBX01.msg.int.genesyslab.com> <CAJrXDUHphQq+=vWRNjuRN1BAcUz4kfUf7DEwCPgBBX-esQh0HQ@mail.gmail.com> <CALiegfmNS=CJkUC76Lz2DW2Zuntffa9Xh1fwopqT033aRN3R_A@mail.gmail.com> <CAJrXDUG_mNmYjZJf1nJvUhZf0WkrdmrCxCSdC9WoZspCexWrng@mail.gmail.com> <CABw3bnOAWQsmNW7512Scu2W0RjnBc5otqez4fqMJvfDOKUBNxw@mail.gmail.com> <CAJrXDUFLxHpjgoNCdumD_b69D1VEgB2rFeS=X5TriBfOZQ5N8A@mail.gmail.com> <CABw3bnPprVKPsFqJkDLRKeCmWDcGm-jCqezjDcBD4Y329RoROA@mail.gmail.com>
From: Peter Thatcher <pthatcher@google.com>
Date: Mon, 17 Jun 2013 12:53:35 -0700
Message-ID: <CAJrXDUHAsjJ7KBXx=USDHR1=seNb1MeZ+E9a5=s-zj-7dWB_2w@mail.gmail.com>
To: José Luis Millán <jmillan@aliax.net>
Content-Type: multipart/alternative; boundary="047d7b15a3851f9bad04df5ef69c"
X-Gm-Message-State: ALoCoQkHg3a+fpQ0pSqdzGSBTQVLvKUAmUI8ghSApZXlHW6lZv4lYWlju8aeZat2DsMWZyfzv7QfFj2W4AMf3t71rJ6nd0S3pdfJBhUgx214IvADbz4GeDyqJ1AOR7FdPRcdG8ehjPlbhmsAXLGZOAbh9A5j8hCVVbuBFgr6xnjmN8dC4GjX5nW5pUqfgGTtPkOKLkk5XSmu
Cc: "<rtcweb@ietf.org>" <rtcweb@ietf.org>
Subject: Re: [rtcweb] Proposal for a JS API for NoPlan (adding multiple sources without encoding them in SDP)
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Jun 2013 19:54:25 -0000
X-List-Received-Date: Mon, 17 Jun 2013 19:54:25 -0000

To remove a track, stop/remove the track locally and signal the fact that
the track is stopped to the remote side (in an app specific way).

To add a track, don't modify the existing MediaStream.  Create a new
MediaStream and call createLocalStream with the new MediaStream, and then
signal the MediaStreamDescription for it to the remote side.  If you really
want two MediaStreams to be merged into one MediaStream (the previous one
and the new one), signal that desire to the remote side (in an app specific
way), and the remote JS application can handle merging them.

The key here is that that we need an API for telling the browser how to
send tracks and how to receive tracks, and this provides it.  How those
tracks get arranged into MediaStreams can be done completely  by JS and
app-specific signalling.  In fact, we could model this entirely as
PeerConnection.createLocalTrack and createRemoteTrack instead of
.createLocalStream and createRemoteStream, but we kept is stream-oriented
because it works better with the existing methods in the simple cases.


On Mon, Jun 17, 2013 at 12:42 PM, José Luis Millán <jmillan@aliax.net>wrote:

> Yes, just adding and removing tracks.
>
> Regards
>
>
> 2013/6/17 Peter Thatcher <pthatcher@google.com>
>
>> What do you mean by "change to an existing media stream"?  Do you mean
>> adding and removing tracks?  I don't know of any other changes that can be
>> made to a MediaStream.
>>
>>
>> On Mon, Jun 17, 2013 at 11:52 AM, José Luis Millán <jmillan@aliax.net>wrote:
>>
>>> Hi,
>>>
>>>
>>> 2013/6/17 Peter Thatcher <pthatcher@google.com>
>>>
>>>> In that use case, just call createLocalStream again with the new
>>>> MediaStream from GetUserMedia and get a new LocalMediaStream.  There's no
>>>> need to alter the existing one.
>>>>
>>>>
>>> On the receiver side, I would expect to represent each remote
>>> MediaStream on its own audio+video display element.
>>>
>>> If, as a sender, I create a new LocalStream to represent a change in an
>>> existent media stream source, how will the receiver notice that the NEW
>>> MediaStream does refer to the previous one?
>>>
>>>
>>>
>>>
>>>>
>>>> On Mon, Jun 17, 2013 at 9:34 AM, Iñaki Baz Castillo <ibc@aliax.net>wrote:
>>>>
>>>>> 2013/6/17 Peter Thatcher <pthatcher@google.com>:
>>>>> > But I don't really see a use case where you would need to add tracks
>>>>> to a
>>>>> > MediaStreamTrackDescription, other than when parsing signalling and
>>>>> building
>>>>> > the MediaStreamTrackDescription to send down into createRemoteStream.
>>>>> > Perhaps if we had a use case, we could define such support.  But
>>>>> otherwise,
>>>>> > I'd say it's not allowed.
>>>>>
>>>>> Use case:
>>>>>
>>>>> - Start a session/call with just audio (no video requested to the user
>>>>> via getUserMedia).
>>>>> - Latter the user wants to add video.
>>>>> - For that a new getUserMedia(video: true) is requested.
>>>>> - Extract the video track from the obtained MediaStream and add it to
>>>>> the PC.
>>>>>
>>>>> Hope this is an enough common use case.
>>>>>
>>>>>
>>>>> --
>>>>> Iñaki Baz Castillo
>>>>> <ibc@aliax.net>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> rtcweb mailing list
>>>> rtcweb@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/rtcweb
>>>>
>>>>
>>>
>>>
>>> --
>>> José Luis Millán
>>>
>>
>>
>
>
> --
> José Luis Millán
>