Re: [Dart] multiplexing different media types

[Harald Alvestrand <[e-mail]>]

Paul,

this is one of the (many) things that seem extremely simple when we look
at the RTP level, but devolves into dastardly complexity when we
consider SDP as a description protocol, and the claim is made that the
restriction is also embedded into many RTP/SDP/SIP implementations.

SDP uses a per m-line field to describe a payload type, without
indicating which SSRC these payload types are intended to belong to.
Traditionally, one m-line described one RTP session. So colliding PTs
inside one RTP session simply couldn't work.

As I've heard the arguments in the context of the BUNDLE draft, there
are implementations that expected this state of things to last forever,
and do not do any SSRC signalling - so when they see a new SSRC, they
look at its payload type to decide what kind of flow it is.

In a context where something different than SDP is used, and SSRCs are
always described explicitly, what you suggest is easy.

When using SDP, with either non-explicit SSRC signaling or multiple
media flows per m-line, it is hard.

I'll return later to why I think the problem is not so great as you
describe below, but that's enough of a topic to deserve another message.

On 06/15/2014 08:26 PM, Paul E. Jones wrote:
> Eric,
>  
> The reason is to free the endpoint from what I think might be
> an unnecessary constraints.
>  
> From the multiplexing guidelines draft, we have this:
>  
>                            |
>                            | packets
>            +--             v
>            |        +------------+
>            |        |   Socket   |
>            |        +------------+
>            |            ||  ||
>       RTP  |       RTP/ ||  |+-----> SCTP ( ...and any other protocols)
>    Session |       RTCP ||  +------> STUN (multiplexed using same port)
>            +--          ||
>            +--          ||
>            |      (split by SSRC)
>            |       ||   ||   ||
>            |       ||   ||   ||
>      Media |      +--+ +--+ +--+
>    Streams |      |PB| |PB| |PB| Jitter buffer, process RTCP, FEC, etc.
>            |      +--+ +--+ +--+
>            +--      |   |     |
>            (pick rending context based on PT)
>            +--      |  /      |
>            |        +---+     |
>            |         /  |     |
>    Payload |      +--+ +--+ +--+
>    Formats |      |CR| |CR| |CR| Codecs and rendering
>            |      +--+ +--+ +--+
>            +--
>  
> So the PT value only has practical significance within the context of
> a given SSRC (which maps directly to a specific media source), as it's
> that point in the code where some application component (e.g., the
> audio component) will look at the packets.  Given the fact that the PT
> number space is fairly limited, having PT associated only with a given
> SSRC makes it possible for any media source to have a fairly extensive
> number of values.
>  
> If there is a complex system sending a lots of different media
> formats, sending FEC information (which I assume would still use the
> same SSRC), perhaps RFC 2198 redundancy PT, etc., it's possible that
> the number of different payload types might go beyond the number
> currently allocated as dynamic PT values.  This has not been an issue
> until now, since systems would usually use separate RTP sessions for
> different media sources, which means the endpoint is less constrained
> by the numbering space for PT values.
>  
> Now that the plan is to multiplex media over the same port and using
> the SSRC to differentiate one media source from the other, each media
> source potentially using a multiplicity of payload type values, I
> think there might be benefit in not imposing the restriction of "all
> synchronisation sources sent from an single endpoint share the same
> payload types definitions."
>  
> So lifting this restriction,
> * a video source could define PT 96, 97, 98, and 99 for it's purpose
> * an audio source could define PT 96, 97, 98, 99, 100, 101 for its purpose
> * a file transfer component could use PT 96 (this might go over the
> data channel, but indulge me)
> * etc.
>  
> In terms of writing code, this could allow for further separation of
> application components.  There is no need for the audio component to
> coordinate with the whiteboarding component or video component or
> whatever when deciding what payload type values to use.  I can imagine
> how these different components might be developed by different teams,
> with all of the independent components easily plugging in together to
> share an RTP session.  Forcing the use of a "global" set of PT
> definition forces additional coordination between these application
> component teams and more complexity in the internal application
> interfaces.  If each component can be developed and maintained in
> isolation as much as possible, the easier it will be to mix/match
> components in the browser and to share the single RTP session.  It
> also make orchestration between those components a bit easier.  At
> least that's my opinion. :-)
>  
> I'm not suggesting the current text is wrong, but I am curious why we
> might not want to lift that global payload type definition restriction.
>  
> Paul
>  
> ------ Original Message ------
> From: "Eric Rescorla" <[e-mail] <mailto:[e-mail]>>
> To: "Paul E. Jones" <[e-mail] <mailto:[e-mail]>>
> Cc: "Harald Alvestrand" <[e-mail]
> <mailto:[e-mail]>>; [e-mail] <mailto:[e-mail]>
> Sent: 6/15/2014 1:01:29 PM
> Subject: Re: multiplexing different media types
>  
>>
>>
>>
>> On Sun, Jun 15, 2014 at 9:18 AM, Paul E. Jones <[e-mail]
>> <mailto:[e-mail]>> wrote:
>>
>>     Eric,
>>
>>     Sorry, I should have been clearer in preparing the question. When
>>     I was referring to media sources, I had completely different
>>     types in mind (e.g., audio and video).
>>
>>     What I am trying to understand is why there is this statement in
>>     the guidelines draft:
>>
>>     "The payload type is scoped by sending endpoint within an RTP
>>     Session. All synchronisation sources sent from an single endpoint
>>     share the same payload types definitions."
>>
>>     Clearly, that is legal. But, I'm curious why we maintain this
>>     restriction.  Since a receiver will demux incoming packets via
>>     the SSRC, it seems the PT values associated with a given SSRC
>>     could be distinct from another SSRC.
>>
>>     Presently for a given sender, there is a "global" definition of
>>     PT values (96 for some specific audio format, 97 for some
>>     specific video format, ...) and those can be used by multiple
>>     SSRCs.  So I'm asking why we don't allow {SSRC1: (96, 97), SSRC2:
>>     (96, 97), ...}, where each 96 could be different formats.
>>
>> What would be the virtue of doing that?
>>
>> -Ekr
>>  
> [snip]
>  


-- 
Surveillance is pervasive. Go Dark.