Re: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast

[Bo Burman <[e-mail]>]

Should a simulcast-capable endpoint be restricted to be a simulcast-sender-only or a simulcast-receiver-only, or do we envision endpoints that are capable of both sending and receiving simulcast? One example would be if simulcast streams are to be forwarded between different SFU (using Bernard's terminology below).

If so and assuming that we define a new 5888-type simulcast grouping, do we need separate groupings for send direction simulcast and receive direction simulcast, or is a=sendonly and a=recvonly information for simulcast-grouped media blocks fully sufficient?

Also, I think the way to use this approach for simulcast in combination with multiple separate media sources (like multiple cameras) or media sinks (like multiple displays) would be pretty straightforward; just add one simulcast grouping per media source/sink.

From: mmusic [mailto:[e-mail]] On Behalf Of Bernard Aboba
Sent: den 30 november 2013 20:31
To: Christer Holmberg; [e-mail]
Subject: Re: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast


Christer asked:

"In your example, how does the answerer know that the offered m- lines are for simulcast? As far as O/A is concerned, it could just be a number of completely separate (different content etc) uni-directional streams."

[BA] The most obvious way would be grouping.  Looking at RFC 5888/5583, it doesn't appear that either FID or DDP grouping would apply to simulcast.   So perhaps a new group type would be needed.





Bernard Aboba <[e-mail]<mailto:[e-mail]>> wrote:


At IETF 88, a number of  participants (including myself) expressed the view that draft-westerlund-avtcore-rtp-simulcast was too complicated.  Below, I will attempt to explain how simulcast might be negotiated in a simplified Offer/Answer exchange based on the principles of Unified Plan.

Typically in simulcast we have an endpoint sending multiple streams in parallel to the Selective Forwarding Unit (SFU), while receiving one among a set of potential streams from the SFU, with the SFU being able to switch between the streams that it is sending to the endpoint without signaling.  RFC 3264 describes how to handle each of these situations.

With respect to sending multiple streams in parallel, RFC 3264 Section 5.1 states the following:

If the offerer wishes to only send media on a stream to its peer, it MUST mark the stream as sendonly with the "a=sendonly" attribute.  We  refer to a stream as being marked with a certain direction if a direction attribute was present as either a media stream attribute or a session attribute. For sendonly RTP streams, the payload type numbers indicate the value of the payload type field in RTP packet the offerer is planning to send for that codec...  In all cases, the formats in the "m=" line MUST be listed in order of preference, with the first format listed being preferred.  In this case, preferred means that the recipient of the offer SHOULD use the format with the highest preference that is acceptable to it. If multiple media streams of different types are present, it means that the offerer wishes to use those streams at the same time.  A typical case is an audio and a video stream as part of a videoconference.

If multiple media streams of the same type are present in an offer,  it means that the offerer wishes to send (and/or receive) multiple streams of that type at the same time...  Each stream MAY use different encodings.

With respect to receiving one among a set of potential streams, here is what RFC 3264 Section 5.1 says:

If the offerer wishes to only receive media from its peer, it MUST mark the stream as recvonly... If multiple formats are listed, it means that the offerer is capable of making use of any of those formats during the session.  In other words, the answerer MAY change formats in the middle of the session, making use of any of the formats listed, without sending a new offer.  For a sendonly stream, the offer SHOULD indicate those formats the offerer is willing to send for this stream.  For a recvonly stream,  the offer SHOULD indicate those formats the offerer is willing to receive for this stream....

For recvonly RTP streams, the payload type numbers indicate the value of the payload type field in RTP packets the offerer is expecting to  receive for that codec.

Based on the above:

1. An offer from an endpoint to an SFU would include an m=line for each potential simulcast stream to be sent, each including an a=sendonly line.  The offer would also include a single m=line (with a=recvonly) describing the streams that the endpoint is prepared to receive, each with their own payload type.

2. An answer from an SFU to an endpoint would accept or reject each of the simulcast streams that the endpoint is proposing to send (with an a=recvonly line).  In addition, the answer would include a single m=line (with a=sendonly) indicating the streams from among those proposed by the endpoint that the SFU is prepared to send.  The SFU can then switch between those streams without having to signal the endpoint.

Since it is possible for the Answerer to accept/reject each of the proposed streams the Offerer is proposing to send, an Answerer can indicate how many streams it is prepared to receive (might only be one if simulcast isn't supported).   Similarly, an Answerer can select those simulcast streams it is prepared to send (might only be one if simulcast isn't supported).

Example Offer (leaving out BUNDLE, rtp/rtcp multiplexing, and audio):

   v=0
   o=alice 2362969037 2362969040 IN IP4 192.0.2.156
   s=Simulcast Enabled Unified Plan Client
   t=0 0
   c=IN IP4 192.0.2.156
   b=AS:665
   m=video 49300 RTP/AVP 97
   a=sendonly
   b=AS:520
   a=rtpmap:97 H264/90000
   a=fmtp:97 profile-level-id=42c01e
   a=imageattr:97 send [x=640,y=360]
   m=video 49302 RTP/AVP 98
   a=sendonly
   a=rtpmap:98 H264/90000
   a=fmtp:98 profile-level-id=42c01e
   a=imageattr:98 send [x=320,y=180]
   m=video 49304 RTP/AVP 100 99
   a=recvonly
   a=rtpmap:100 H264/90000
   a=fmtp:100 profile-level-id=42c01e
   a=imageattr:100 recv [x=640,y=360]
   a=rtpmap:99 H264/90000
   a=fmtp:99 profile-level-id=42c01e
   a=imageattr:99 recv [x=320,y=180]


Example Answer:

   v=0
   o=server 823479283 1209384938 IN IP4 192.0.2.2
   s=Answer to Simulcast Enabled Unified Plan Client
   t=0 0
   c=IN IP4 192.0.2.43
   b=AS:665
   m=video 48000 RTP/AVP 97
   a=recvonly
   b=AS:520
   a=rtpmap:97 H264/90000
   a=fmtp:97 profile-level-id=42c01e
   a=imageattr:97 recv [x=640,y=360]
   m=video 48002 RTP/AVP 98
   a=recvonly
   a=rtpmap:98 H264/90000
   a=fmtp:98 profile-level-id=42c01e
   a=imageattr:98 recv [x=320,y=180]
   m=video 48004 RTP/AVP 100 99
   a=sendonly
   a=rtpmap:100 H264/90000
   a=fmtp:100 profile-level-id=42c01e
   a=imageattr:100 send [x=640,y=360]
   a=rtpmap:99 H264/90000
   a=fmtp:99 profile-level-id=42c01e
   a=imageattr:100 send [x=320,y=180]