Re: [rtcweb] Interaction between MediaStream API and signaling

On 03/30/2012 04:26 PM, Jim Barnett wrote:
> Doesn't the basic model require that the offer be ready to receive media
> as soon as it sends the offer?  If that's the case then the offer must
> be ready and able to map RTP streams to media streams (using the mapping
> specified in the SDP for its offer) as soon as it sends the offer.

Yes, the basic SDP o/a model requires that the offerer is ready to 
receive media as soon as the offer is sent. But as I tried to explain 
(but obviously failed to) is that there is an additional constraint 
here: the JS application deals with MediaStreams, which often has more 
than one track, e.g. one video and one audio track. A track in turn 
would correspond to an RTP stream. And you can have several concurrent 
MediaStreams in action.

Say now that the offerer, before getting an ANSWER with info, receives 
three RTP streams; two video and one audio. The offerer would in this 
situation not have a clue how these belong together. One could be video 
from a front facing camera (that is supposed to be displayed on a large 
video screen), the other video RTP stream could contain the video from 
the rear facing camera and is supposed to together with the audio RTP 
stream be one MediaStream that is supposed to be played in sync in a 
thumbnail video display.

Without the offerer having the info about what these RTP streams 
represent it can't do anything sensible.

Stefan

>
> - Jim
>
> -----Original Message-----
> From: rtcweb-bounces@ietf.org [mailto:rtcweb-bounces@ietf.org] On Behalf
> Of Stefan Hakansson LK
> Sent: Friday, March 30, 2012 5:00 AM
> To: rtcweb@ietf.org
> Subject: [rtcweb] Interaction between MediaStream API and signaling
>
> The JS API has deals with MediaStreams (this is what you send and
> receive using PeerConnection from an application perspective).
>
> A browser receiving RTP streams, needs side info to be able to assemble
> those RTP streams into MediaStreams in a correct way. The current model
> is that this is signaled using SDP exchanges (where Haralds MSID
> proposal would tell which MediaStream an RTP stream belongs to).
>
> As I brought up at the mike yesterday, I think we may have a race
> condition for the responder.
>
> For the initiator side browser, this is clear: once an (PR-)ANSWER is
> received, the responder has received the SDP, and hence can map incoming
> RTP streams into MediaStreams.
>
> But for the responder side this is less clear to me. Imagine
> applications where the responder just mirrors the initiator - if one of
> the parties adds a MediaStream to PeerConnection, the other end would
> add the corresponding MediaStream.
>
> This can happen any time in the session, so ICE can very well be up and
> running. One example could be that the data channel is used for text
> chat, when one side clicks a button to start video. And the application
> can have asked for permission to use all input devices earlier, so no
> user interaction may be involved.
>
> In this situation the responder's (added) RTP streams can very well
> arrive before the ANSWER if I understand correctly. I think we need to
> find a way to handle this. One way is to add an "ACK" that indicates to
> the responder that the initiator has received the ANSWER, but I'm not
> sure that is the best way.
>
> Stefan
> _______________________________________________
> rtcweb mailing list
> rtcweb@ietf.org
> https://www.ietf.org/mailman/listinfo/rtcweb