Re: [clue] Multiple Streams - RTP muxing?

[Alex Eleftheriadis <[e-mail]>]

On Feb 1, 2011, at 3:34 PM, Stephen Botzko wrote:

>>>
As the general media type is part of the m= line one
can't have both RTP payload types of both audio and video type in the
same m= line.
>>>There might be some confusion here, I didn't hear anyone propose multiplexing both audio and video in the same session.  I believe the multiplex idea is to have two sessions, one for audio and one for video.  This is what existing video systems that employ multiplexing do.

Yes, I don't think anyone intentionally suggested that the two (audio and video) are muxed.


>>>
In general I think this discussion should up level for a while.
>>>
Probably a good idea.  This will come back, there are a lot of folks who build telepresence systems who favor SSRC multiplexing.


Certainly (and this goes beyond pure telepresence). We will need some additional signaling tools. Check out also draft-lennox-mmusic-sdp-source-selection-02, if you haven't already. For future discussion.

Incidentally, since the iPad was mentioned, you can see a demonstration of iPad running RTP (de)muxing of 3 H.264 streams in http://bit.ly/fMTP6j (posted in August 2010). This is not a "demo"; this is our regular SDK running. The complexity argument is really moot, in view of all the other things that happen in a server or endpoint.

--Alex

Stephen Botzko

On Tue, Feb 1, 2011 at 7:51 AM, Magnus Westerlund <[e-mail]<mailto:[e-mail]>> wrote:
aravind sethuraman skrev 2011-01-31 20:03:
> Hi all,
> if we chose a single RTP session for ALL video and AUDIO, how will we
> signal which SSRC conforms to which screen etc? Not via RTCP like TIP does.

I just want to point out that signalling this with a single m= line in
SDP is impossible. As the general media type is part of the m= line one
can't have both RTP payload types of both audio and video type in the
same m= line.

When it comes to RTP RFC 3550 discusses this type of multiplexing and
has the following to say on the issue (from section 5.2 of RFC 3550):

  For efficient protocol processing, the number of multiplexing points
  should be minimized, as described in the integrated layer processing
  design principle [10].  In RTP, multiplexing is provided by the
  destination transport address (network address and port number) which
  is different for each RTP session.  For example, in a teleconference
  composed of audio and video media encoded separately, each medium
  SHOULD be carried in a separate RTP session with its own destination
  transport address.

  Separate audio and video streams SHOULD NOT be carried in a single
  RTP session and demultiplexed based on the payload type or SSRC
  fields.  Interleaving packets with different RTP media types but
  using the same SSRC would introduce several problems:

  1. If, say, two audio streams shared the same RTP session and the
     same SSRC value, and one were to change encodings and thus acquire
     a different RTP payload type, there would be no general way of
     identifying which stream had changed encodings.

  2. An SSRC is defined to identify a single timing and sequence number
     space.  Interleaving multiple payload types would require
     different timing spaces if the media clock rates differ and would
     require different sequence number spaces to tell which payload
     type suffered packet loss.

  3. The RTCP sender and receiver reports (see Section 6.4) can only
     describe one timing and sequence number space per SSRC and do not
     carry a payload type field.

  4. An RTP mixer would not be able to combine interleaved streams of
     incompatible media into one stream.

  5. Carrying multiple media in one RTP session precludes: the use of
     different network paths or network resource allocations if
     appropriate; reception of a subset of the media if desired, for
     example just audio if video would exceed the available bandwidth;
     and receiver implementations that use separate processes for the
     different media, whereas using separate RTP sessions permits
     either single- or multiple-process implementations.

  Using a different SSRC for each medium but sending them in the same
  RTP session would avoid the first three problems but not the last
  two.

  On the other hand, multiplexing multiple related sources of the same
  medium in one RTP session using different SSRC values is the norm for
  multicast sessions.  The problems listed above don't apply: an RTP
  mixer can combine multiple audio sources, for example, and the same
  treatment is applicable for all of them.  It may also be appropriate
  to multiplex streams of the same medium using different SSRC values
  in other scenarios where the last two problems do not apply.


As can be seen if one puts audio and video in the same RTP session but
with unique SSRCs for each stream one still do need to take care.

Issue 4, can be solved by explicit signalling, but makes the solution
more fragile in cases with dynamic memberships. If you don't have the
inforamtion about the stream mixing decisions are more difficult.

Issue 5, I think becomes important as soon as we talk about a
heterogeneous set of session participants. A case that are part of the
charter to consider and solve.

In general I think this discussion should up level for a while. Lets
start discussing the general architecture and use cases and what
requirements that puts on the different nodes in the architecture. Then
we can return to this and see what requirements we have and how they
matches the capabilities of the protocols we intended to solve it with.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  [phone]
Färögatan 6                | Mobile [phone]
SE-164 80 Stockholm, Sweden| mailto: [e-mail]<mailto:[e-mail]>
----------------------------------------------------------------------

_______________________________________________
clue mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/clue