[rtcweb] Multiple videos in one MediaStream (Re: MediaStream Label and CNAME)

[Harald Alvestrand <[e-mail]>]

On 09/13/11 10:43, Magnus Westerlund wrote:
> WG,
> (As an individual contributor)
>
>
> There has been some discussion as result of the presentation of
> terminology in the RTCWEB Interim meeting last Thursday. The biggest
> question was why CNAME can't map to MediaStream label. Below we
> clarify why we think CNAME and label are separate entities.
>
> One part in this reasoning has to do with the current definition of
> ’media resource’
> (<http://dev.w3.org/html5/spec/Overview.html#media-resource>) and media
> elements of html5. The ‘media resource’ could be a file, or, more
> relevant to this discussion, a MediaStream. In that usage only a single
> video track can be played simultaneously and in sync with one or more
> audio tracks.
>
> Thus unless we modify an existing semantics the only way of playing
> multiple video tracks in sync with one or more audio tracks is to have
> multiple MediaStream objects.
Sorry, I think this can be handled with the existing API without any issue.
Apologies to those who know JavaScript better than me, but this is a handler
that can handle a new incoming video stream

NewStreamHandler(stream) {
    myMultiVideoStream = stream
    myFirstVideo = new MediaStream(myMultiVideoStream)
    myFirstVideo.videoTracks.select(1)
    oneVideoObject.setSource(myFirstVideo.getUrl())
    mySecondVideo = new MediaStream(myMultiVideoStream)
    mySecondVideo.videoTracks.select(2)
    anotherVideoObject.setSource(mySecondVideo.getUrl())
}

I think the API can be improved, but the improvements are unlikely to 
lose this functionality.
(Remember that the API objects are the control surfaces on the stream - 
the video data will likely go straight from the incoming buffer, through 
the codec for decoding, and blitted onto the screen; the "copying" from 
one MediaStream to another MediaStream is purely conceptual.)

I don't think it makes sense to discuss the other points before getting 
this one put to rest.

                  Harald