Re: [Dart] multiplexing different media types

"Paul E. Jones" <> Sun, 15 June 2014 18:25 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 38BEE1B28F7 for <>; Sun, 15 Jun 2014 11:25:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.652
X-Spam-Status: No, score=-2.652 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 3gwl_W9tr6i5 for <>; Sun, 15 Jun 2014 11:25:54 -0700 (PDT)
Received: from ( []) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 8585F1B28E8 for <>; Sun, 15 Jun 2014 11:25:54 -0700 (PDT)
Received: from [] ( []) (authenticated bits=0) by (8.14.5/8.14.5) with ESMTP id s5FIPmJd005072 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 15 Jun 2014 14:25:50 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;; s=dublin; t=1402856751; bh=5Q7p1JbQ7mD0KQB8p+HY8YXfVwgZUU+N/lzZrpw24FQ=; h=From:To:Subject:Cc:Date:Content-Type:In-Reply-To:Message-Id: Mime-Version:Reply-To; b=plXEusMc/AYvFjkO12+ed0ZpDTY0eeojF06LFwaqfNlq/DFFZJquvJrzPQUuIISlI mMziU4N74ZQPhV3ytnG+wzKpeBg7PJ9mJ3w1lP+2tF44eGPimTJuE4mcARRssyKaN9 5TpQSKhYnsYVsOV9Zrs5M4FplLY5JKPwdkWZDMwc=
From: "Paul E. Jones" <>
To: Eric Rescorla <>
Date: Sun, 15 Jun 2014 18:26:01 +0000
Content-Type: multipart/alternative; boundary="------=_MB9424C4AA-25B4-4B46-B8A6-F4D4D3FAD217"
In-Reply-To: <>
Message-Id: <em9204c90e-7b56-41fa-8f82-ed7bce563ee1@sydney>
Mime-Version: 1.0
User-Agent: eM_Client/6.0.20154.0
Cc: Harald Alvestrand <>,
Subject: Re: [Dart] multiplexing different media types
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: "Paul E. Jones" <>
List-Id: "\"DiffServ Applied to RTP Transports discussion list\"" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 15 Jun 2014 18:25:57 -0000


The reason is to free the endpoint from what I think might be an 
unnecessary constraints.

>From the multiplexing guidelines draft, we have this:

| | packets +-- v | +------------+ | | Socket | | +------------+ | || || 
RTP | RTP/ || |+-----> SCTP ( ...and any other protocols) Session | RTCP 
|| +------> STUN (multiplexed using same port) +-- || +-- || | (split by 
SSRC) | || || || | || || || Media | +--+ +--+ +--+ Streams | |PB| |PB| 
|PB| Jitter buffer, process RTCP, FEC, etc. | +--+ +--+ +--+ +-- | | | 
(pick rending context based on PT) +-- | / | | +---+ | | / | | Payload | 
+--+ +--+ +--+ Formats | |CR| |CR| |CR| Codecs and rendering | +--+ +--+ 
+--+ +--

So the PT value only has practical significance within the context of a 
given SSRC (which maps directly to a specific media source), as it's 
that point in the code where some application component (e.g., the audio 
component) will look at the packets.  Given the fact that the PT number 
space is fairly limited, having PT associated only with a given SSRC 
makes it possible for any media source to have a fairly extensive number 
of values.

If there is a complex system sending a lots of different media formats, 
sending FEC information (which I assume would still use the same SSRC), 
perhaps RFC 2198 redundancy PT, etc., it's possible that the number of 
different payload types might go beyond the number currently allocated 
as dynamic PT values.  This has not been an issue until now, since 
systems would usually use separate RTP sessions for different media 
sources, which means the endpoint is less constrained by the numbering 
space for PT values.

Now that the plan is to multiplex media over the same port and using the 
SSRC to differentiate one media source from the other, each media source 
potentially using a multiplicity of payload type values, I think there 
might be benefit in not imposing the restriction of "all synchronisation 
sources sent from an single endpoint share the same payload types 

So lifting this restriction,
* a video source could define PT 96, 97, 98, and 99 for it's purpose
* an audio source could define PT 96, 97, 98, 99, 100, 101 for its 
* a file transfer component could use PT 96 (this might go over the data 
channel, but indulge me)
* etc.

In terms of writing code, this could allow for further separation of 
application components.  There is no need for the audio component to 
coordinate with the whiteboarding component or video component or 
whatever when deciding what payload type values to use.  I can imagine 
how these different components might be developed by different teams, 
with all of the independent components easily plugging in together to 
share an RTP session.  Forcing the use of a "global" set of PT 
definition forces additional coordination between these application 
component teams and more complexity in the internal application 
interfaces.  If each component can be developed and maintained in 
isolation as much as possible, the easier it will be to mix/match 
components in the browser and to share the single RTP session.  It also 
make orchestration between those components a bit easier.  At least 
that's my opinion. :-)

I'm not suggesting the current text is wrong, but I am curious why we 
might not want to lift that global payload type definition restriction.


------ Original Message ------
From: "Eric Rescorla" <>
To: "Paul E. Jones" <>
Cc: "Harald Alvestrand" <>;
Sent: 6/15/2014 1:01:29 PM
Subject: Re: multiplexing different media types

>On Sun, Jun 15, 2014 at 9:18 AM, Paul E. Jones <> 
>>Sorry, I should have been clearer in preparing the question. When I 
>>was referring to media sources, I had completely different types in 
>>mind (e.g., audio and video).
>>What I am trying to understand is why there is this statement in the 
>>guidelines draft:
>>"The payload type is scoped by sending endpoint within an RTP Session. 
>>All synchronisation sources sent from an single endpoint share the 
>>same payload types definitions."
>>Clearly, that is legal. But, I'm curious why we maintain this 
>>restriction.  Since a receiver will demux incoming packets via the 
>>SSRC, it seems the PT values associated with a given SSRC could be 
>>distinct from another SSRC.
>>Presently for a given sender, there is a "global" definition of PT 
>>values (96 for some specific audio format, 97 for some specific video 
>>format, ...) and those can be used by multiple SSRCs.  So I'm asking 
>>why we don't allow {SSRC1: (96, 97), SSRC2: (96, 97), ...}, where each 
>>96 could be different formats.
>What would be the virtue of doing that?