[rtcweb] Multiple videos in one MediaStream (Re: MediaStream Label and CNAME)

Harald Alvestrand <harald@alvestrand.no> Tue, 13 September 2011 14:29 UTC

Return-Path: <harald@alvestrand.no>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E76FE21F8B0D for <rtcweb@ietfa.amsl.com>; Tue, 13 Sep 2011 07:29:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -108.207
X-Spam-Level:
X-Spam-Status: No, score=-108.207 tagged_above=-999 required=5 tests=[AWL=2.392, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SKapsGPh4Lvi for <rtcweb@ietfa.amsl.com>; Tue, 13 Sep 2011 07:29:58 -0700 (PDT)
Received: from eikenes.alvestrand.no (eikenes.alvestrand.no [158.38.152.233]) by ietfa.amsl.com (Postfix) with ESMTP id 4378D21F8804 for <rtcweb@ietf.org>; Tue, 13 Sep 2011 07:29:58 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by eikenes.alvestrand.no (Postfix) with ESMTP id 39BCA39E0AF for <rtcweb@ietf.org>; Tue, 13 Sep 2011 16:32:04 +0200 (CEST)
X-Virus-Scanned: Debian amavisd-new at eikenes.alvestrand.no
Received: from eikenes.alvestrand.no ([127.0.0.1]) by localhost (eikenes.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NFVIbEb4yTMp for <rtcweb@ietf.org>; Tue, 13 Sep 2011 16:32:03 +0200 (CEST)
Received: from hta-dell.lul.corp.google.com (62-20-124-50.customer.telia.com [62.20.124.50]) by eikenes.alvestrand.no (Postfix) with ESMTPS id BAF4839E088 for <rtcweb@ietf.org>; Tue, 13 Sep 2011 16:32:03 +0200 (CEST)
Message-ID: <4E6F6963.9090702@alvestrand.no>
Date: Tue, 13 Sep 2011 16:32:03 +0200
From: Harald Alvestrand <harald@alvestrand.no>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110617 Thunderbird/3.1.11
MIME-Version: 1.0
To: rtcweb@ietf.org
References: <4E6F17AB.4000005@ericsson.com>
In-Reply-To: <4E6F17AB.4000005@ericsson.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
Subject: [rtcweb] Multiple videos in one MediaStream (Re: MediaStream Label and CNAME)
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Sep 2011 14:29:59 -0000

On 09/13/11 10:43, Magnus Westerlund wrote:
> WG,
> (As an individual contributor)
>
>
> There has been some discussion as result of the presentation of
> terminology in the RTCWEB Interim meeting last Thursday. The biggest
> question was why CNAME can't map to MediaStream label. Below we
> clarify why we think CNAME and label are separate entities.
>
> One part in this reasoning has to do with the current definition of
> ’media resource’
> (<http://dev.w3.org/html5/spec/Overview.html#media-resource>) and media
> elements of html5. The ‘media resource’ could be a file, or, more
> relevant to this discussion, a MediaStream. In that usage only a single
> video track can be played simultaneously and in sync with one or more
> audio tracks.
>
> Thus unless we modify an existing semantics the only way of playing
> multiple video tracks in sync with one or more audio tracks is to have
> multiple MediaStream objects.
Sorry, I think this can be handled with the existing API without any issue.
Apologies to those who know JavaScript better than me, but this is a handler
that can handle a new incoming video stream

NewStreamHandler(stream) {
    myMultiVideoStream = stream
    myFirstVideo = new MediaStream(myMultiVideoStream)
    myFirstVideo.videoTracks.select(1)
    oneVideoObject.setSource(myFirstVideo.getUrl())
    mySecondVideo = new MediaStream(myMultiVideoStream)
    mySecondVideo.videoTracks.select(2)
    anotherVideoObject.setSource(mySecondVideo.getUrl())
}

I think the API can be improved, but the improvements are unlikely to 
lose this functionality.
(Remember that the API objects are the control surfaces on the stream - 
the video data will likely go straight from the incoming buffer, through 
the codec for decoding, and blitted onto the screen; the "copying" from 
one MediaStream to another MediaStream is purely conceptual.)

I don't think it makes sense to discuss the other points before getting 
this one put to rest.

                  Harald