Re: [rtcweb] Proposal for API mapping for RTP

Magnus Westerlund <magnus.westerlund@ericsson.com> Mon, 11 November 2013 09:24 UTC

Return-Path: <magnus.westerlund@ericsson.com>
X-Original-To: rtcweb@ietfa.amsl.com
Delivered-To: rtcweb@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7ED5821F9F6F for <rtcweb@ietfa.amsl.com>; Mon, 11 Nov 2013 01:24:09 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.265
X-Spam-Level:
X-Spam-Status: No, score=-103.265 tagged_above=-999 required=5 tests=[AWL=-1.627, BAYES_00=-2.599, HELO_EQ_SE=0.35, HOST_MISMATCH_NET=0.311, MIME_8BIT_HEADER=0.3, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KncNQJwEMEJF for <rtcweb@ietfa.amsl.com>; Mon, 11 Nov 2013 01:23:58 -0800 (PST)
Received: from sessmg20.mgmt.ericsson.se (sessmg20.ericsson.net [193.180.251.50]) by ietfa.amsl.com (Postfix) with ESMTP id 3A4C621E8213 for <rtcweb@ietf.org>; Mon, 11 Nov 2013 01:11:03 -0800 (PST)
X-AuditID: c1b4fb32-b7f388e0000057e0-03-52809f1d6d00
Received: from ESESSHC008.ericsson.se (Unknown_Domain [153.88.253.124]) by sessmg20.mgmt.ericsson.se (Symantec Mail Security) with SMTP id 92.A0.22496.D1F90825; Mon, 11 Nov 2013 10:10:55 +0100 (CET)
Received: from [127.0.0.1] (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.44) with Microsoft SMTP Server id 14.2.328.9; Mon, 11 Nov 2013 10:10:22 +0100
Message-ID: <52809F33.1020201@ericsson.com>
Date: Mon, 11 Nov 2013 10:11:15 +0100
From: Magnus Westerlund <magnus.westerlund@ericsson.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.1.0
MIME-Version: 1.0
To: =?ISO-8859-1?Q?Stefan_H=E5kansson_LK?= <stefan.lk.hakansson@ericsson.com>, "rtcweb@ietf.org" <rtcweb@ietf.org>
References: <527BD94C.2070900@ericsson.com> <1447FA0C20ED5147A1AA0EF02890A64B1C3D5EF5@ESESSMB209.ericsson.se>
In-Reply-To: <1447FA0C20ED5147A1AA0EF02890A64B1C3D5EF5@ESESSMB209.ericsson.se>
X-Enigmail-Version: 1.6
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupgluLIzCtJLcpLzFFi42KZGfG3Rld+fkOQwfnp+hZr/7WzOzB6LFny kymAMYrLJiU1J7MstUjfLoEr402PdsFak4pZC/wbGOdodTFyckgImEisnniBEcIWk7hwbz1b FyMXh5DACUaJkxeOs0I4yxkl5q87w97FyMHBK6At0bejDKSBRUBV4tqDi2DNbAIWEjd/NLKB 2KICwRLnXy1mB7F5BQQlTs58wgJiiwiUSFw5f4AJxBYWMJO49mcSK4gtJJArMef/F2aQ8ZwC fhJrzlqCmBIC4hI9jUEgFcwCehJTrrYwQtjyEs1bZzNDdGpLNDR1sE5gFJyFZNksJC2zkLQs YGRexShZnFpcnJtuZKCXm55bopdalJlcXJyfp1ecuokRGJoHt/w22sF4co/9IUZpDhYlcd7r rDVBQgLpiSWp2ampBalF8UWlOanFhxiZODilGhgdRdIFFm/znLE6usx8quGy8kfbePPTbCXa pc5Ec7Fn8YW9NVBgz1OTKLGf8XDGxxv/azQ/fO2+yP2g9f+Wq2x6M8/+Ybi4aao2YyPb71ed aiyFk/3mvT2wd/77FVPKqy7oh8sHixm3TE76X5mnXejBu+z6qYgQzYynGiKed8+cqD6xwfG4 1DIlluKMREMt5qLiRAAKX5X3GwIAAA==
Subject: Re: [rtcweb] Proposal for API mapping for RTP
X-BeenThere: rtcweb@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Real-Time Communication in WEB-browsers working group list <rtcweb.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/rtcweb>
List-Post: <mailto:rtcweb@ietf.org>
List-Help: <mailto:rtcweb-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtcweb>, <mailto:rtcweb-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Nov 2013 09:24:25 -0000

Stefan, WG,

Please see inline

On 2013-11-10 10:32, Stefan Håkansson LK wrote:
> On 07/11/13 19:18, Magnus Westerlund wrote:
>> WG,
>> (As draft-ietf-rtcweb-rtp-usage Editor)
> 
> Magnus,
> 
> thanks for this write up. Some comments inline.
> 
>>
>> There was discussion yesterday afternoon on the W3C MediaStream to RTP
>> mapping. This did arrive on some conclusions that I here try to word
>> into a proposal for everyones review.
>>
>> A MediaStreamTrack is sent over a source packet stream (one SSRC) and
>> can have additionally redundancy packet streams (SSRCs). These
>> redundnacy streams are RTP retranmission streams or FEC.
> 
> Can't there be enhancement packet streams as well (for layered codecs)?

So the enhancement layers if sent as individual packet streams are
called dependent layers in the current RTP taxonomy draft. Yes, they
could also exist. However, as there are no API support for these at the
current day they haven't been defined. If one uses something like SVC
MST, then clearly additions in how to interpret this must be provided.
If this is a single rooted dependency tree then I think additional SSRCs
for dependencies are quite straightforward. There will need to exist a
way of indicating that SSRCs are dependency streams for a source stream
to make handling work well.

> 
> And in some proposals a single MediaStreamTrack can be sent in more than 
> one encoding (UA - as opposed to JS handled - handled simulcast) - is 
> that covered?

So simulcast was discussed, and that would result in more than one
source stream associated with that MediaStreamTrack. This have similar
requirements that the a receiver understands that this is a simulcast of
the same MediaStreamTrack as another encoding version.

For both simulcast and scalable coding, I am happy to make it clear that
these can result in multiple source packet streams, and further have
additional dependency packet streams.

> 
>>
>> When multiple MediaStreamTracks have the same Media Source, then the
>> fact that one has creaeted multiple MediaStreamTracks and added these
>> through a MediaStream to the PeerConnection is going to result in that
>> each MediaStreamTrack will have its own source packet stream (one SSRC).
>> This will be true, even if there are no difference in the configuration
>> of the MediaStreamTrack. Thus no optimizations in regards to collapsing
>> or aggregating multiple MediaStreamTrack onto a single source packet
>> stream (SSRC). This is done for keeping things simple and straightforward.
> 
> I think this is the right way to do it.
> 
>>
>> When it comes to the use of CNAME, an RTCWEB end-point shall within the
>> context of one origin, i.e. a particular JS creating PeerConnections,
>> use the same CNAME in all these PeerConnections, for all outgoing
>> mediaStreamTracks. However, a end-point MUST be capable of receiving
>> multiple different CNAMEs both within and between different RTP sessions
>> and PeerConnections. This definition comes from the following observations.
>>
>> A MediaStream contains multiple MediaStreamTrack, and different
> 
> s/contains/can contain/

Yes

> 
>> MediaStreams can share the same MediaStreamTrack. MediaStreamTrack
>> within one MediaStream is synchronized. Thus, at any point the JS
> 
> "synchronized _at playout_"?

That is not for us in IETF to define. My aim is to enable a receiver to
perform synchronized playout. If it does it or not, I think is up to the
API definition.

>> application can create a new MediaStream that includes MediaStreamTrack
>> from different context. To avoid needing to change all SSRC and put the
>> SSRCs into a single CNAME at that point, we ensure that this can't happen.
> 
> By stating that all SSRC's originating from an endpoint shall use the 
> same CNAME? (Just for clarification)

Yes. but I would change this from "stating" to "REQUIRE".

> 
>>
>> MediaStreamTracks that are being received in one PeerConnection and then
>> forward by being added to another one will need to be re-synchronzied
>> into the endpoints outgoing. We discussed and came to the agreement on
>> Monday that forwarding would need to be done equivalent of decoding and
>> recoding when forwarding. Implementations may do other things, but must
>> function equivalent to this.
>>
>> The above have some forward interoperability properties. If one like to
>> be able to use multiple CNAMEs that can be added given that W3C API
>> ensures that you don't cause issue with what CNAME a particular SSRC
>> belongs to.
> 
> Could you expand a bit? Do you mean that if there is an API to alter 
> CNAMEs (and I guess there is: SDP mangling) it must ...do what?

I think this requires an API, and I question if any SDP mangling should
be allowed to change the CNAME. This due to the privacy concerns. If the
JS can set the CNAME, then it can enforce traceability of users or
end-points at RTP/RTCP level.

I think the API we did discuss was to consider when the
MediaStreamTracks in one MediaStream incoming in one PeerConnection gets
added as outgoing in another PeerConnection the original CNAME could be
maintained. However, that requires explicit lock down preventing one to
create a MediaStream with one of these MediaStreamTracks and any other
MediaStreamTrack comming from another CNAME context.

> 
>> In addition if one like to optimize the cases where multiple
>> MediaStreamTrack have a common media source that can be added.
> 
> This is not so simple. E.g. assuming we will support pause/resume at 
> some point, that could be quite complex if you pause one 
> MediaStreamTrack but not another - both using the same source.

Yes, that was understood in the discussion that such combinations
requires sane unions of settings across multiple MediaStreamTrack. In
your of paused or not, it is quite simple to agree that if any consumer
exists, i.e. not all paused, then it must be delivered.

But, clearly a reasons for the simple rule proposed.

> 
> The application can already optimize by only sending it once (and clone 
> - if required - at the receiving end).
> 

Good,

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: magnus.westerlund@ericsson.com
----------------------------------------------------------------------