[rtcweb] Interaction between MediaStream API and signaling

Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com> Fri, 30 March 2012 08:59 UTC

Return-Path: <stefan.lk.hakansson@ericsson.com>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id BD4EC21F882B for <rtcweb@ietfa.amsl.com>; Fri, 30 Mar 2012 01:59:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.77
X-Spam-Status: No, score=-7.77 tagged_above=-999 required=5 tests=[AWL=-1.521, BAYES_00=-2.599, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id U5Abai4XaD0g for <rtcweb@ietfa.amsl.com>; Fri, 30 Mar 2012 01:59:41 -0700 (PDT)
Received: from mailgw1.ericsson.se (mailgw1.ericsson.se []) by ietfa.amsl.com (Postfix) with ESMTP id 3C46821F87D0 for <rtcweb@ietf.org>; Fri, 30 Mar 2012 01:59:41 -0700 (PDT)
X-AuditID: c1b4fb2d-b7b76ae0000063d8-50-4f7575fb9b9c
Received: from esessmw0237.eemea.ericsson.se (Unknown_Domain []) (using TLS with cipher AES128-SHA (AES128-SHA/128 bits)) (Client did not present a certificate) by mailgw1.ericsson.se (Symantec Mail Security) with SMTP id 1D.DF.25560.BF5757F4; Fri, 30 Mar 2012 10:59:39 +0200 (CEST)
Received: from [] ( by esessmw0237.eemea.ericsson.se ( with Microsoft SMTP Server id; Fri, 30 Mar 2012 10:59:39 +0200
Message-ID: <4F7575FB.8010201@ericsson.com>
Date: Fri, 30 Mar 2012 10:59:39 +0200
From: Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120310 Thunderbird/11.0
MIME-Version: 1.0
To: "rtcweb@ietf.org" <rtcweb@ietf.org>
Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
Content-Transfer-Encoding: 7bit
X-Brightmail-Tracker: AAAAAA==
Subject: [rtcweb] Interaction between MediaStream API and signaling
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 30 Mar 2012 08:59:42 -0000

The JS API has deals with MediaStreams (this is what you send and 
receive using PeerConnection from an application perspective).

A browser receiving RTP streams, needs side info to be able to assemble 
those RTP streams into MediaStreams in a correct way. The current model 
is that this is signaled using SDP exchanges (where Haralds MSID 
proposal would tell which MediaStream an RTP stream belongs to).

As I brought up at the mike yesterday, I think we may have a race 
condition for the responder.

For the initiator side browser, this is clear: once an (PR-)ANSWER is 
received, the responder has received the SDP, and hence can map incoming 
RTP streams into MediaStreams.

But for the responder side this is less clear to me. Imagine 
applications where the responder just mirrors the initiator - if one of 
the parties adds a MediaStream to PeerConnection, the other end would 
add the corresponding MediaStream.

This can happen any time in the session, so ICE can very well be up and 
running. One example could be that the data channel is used for text 
chat, when one side clicks a button to start video. And the application 
can have asked for permission to use all input devices earlier, so no 
user interaction may be involved.

In this situation the responder's (added) RTP streams can very well 
arrive before the ANSWER if I understand correctly. I think we need to 
find a way to handle this. One way is to add an "ACK" that indicates to 
the responder that the initiator has received the ANSWER, but I'm not 
sure that is the best way.