Re: [MMUSIC] Text proposal for Bundle regarding Associating RTP/RTCP With Correct SDP Media Description

[Jonathan Lennox <[e-mail]>]

And one more question.  This one has significant interop issues, I think.

Should decoding contexts be assumed to be associated with streams independent of m= lines, or with streams within m= lines?

Specifically, if a sender moves a video stream from one m= line to another (maintaining its PT), must it send a fresh I-frame (keyframe) at the time of the move, or can it continue sending inter frames?

> On Jan 27, 2017, at 11:43 AM, Magnus Westerlund <[e-mail]> wrote:
> 
> MMUSIC and RTCWEB,
> 
> Here is now a more complete text proposal that also considers the RTCP. It also goes further than what Appendix B defines in one aspect, namely explicitly considering third party RTCP reporting and what to do with it.
> 
> Feedback is much appreciated. I plan to put this into a PR towards JSEP APPENDIX B on monday. If only to get the JSEP authors attention ;-). However, the text is really intended for the next version of BUNDLE.
> 
> 
> X.  Associating RTP/RTCP With Correct SDP Media Description
> 
>   As described in [RFC3550], RTP packets are associated with RTP
>   streams [RFC7656].  Each RTP stream is identified by an SSRC value,
>   and each RTP packet carries an SSRC value that is used to associate
>   the packet with the correct RTP stream.  RTCP packets also uses SSRCs
>   to identify on which RTP streams any report or feedback relate to.
>   Thus, an RTCP packet will commonly carry multiple SSRC values, and
>   might therefore be providing feedback or report on multiple RTP
>   streams.
> 
>   In order to be able to process received RTP/RTCP packets correctly it
>   must be possible to associate an RTP stream with the correct "m="
>   line, as the "m=" line and SDP attributes associated with the "m="
>   line contain information needed to process the packets.
> 
>   As all RTP streams associated with a BUNDLE group are part of the
>   same RTP session and using the same address:port combination for
>   sending and receiving RTP/RTCP packets, the local address:port
>   combination cannot be used to associate an RTP stream with the
>   correct "m=" line.  In addition, multiple RTP streams might be
>   associated with the same "m=" line.
> 
>   Also, as described in Section 10.1.1, the same payload type value
>   might be used by multiple RTP streams, in which case the payload type
>   value cannot be used to associate an RTP stream with the correct "m="
>   line.  However, there are cases where each "m=" line has unique
>   payload type values, and then the payload type could serve as
>   indicator to the relevant "m=" line the RTP stream is associated
>   with.
> 
>   An offerer and answerer can inform each other which SSRC values they
>   will use for an RTP stream by using the SDP 'ssrc' attribute
>   [RFC5576].  However, an offerer will not know which SSRC values the
>   answerer will use until the offerer has received the answer providing
> 
> 
> 
> Name                      Expires July 31, 2017                 [Page 2]
> 
> Internet-Draft              Abbreviated-Title               January 2017
> 
> 
>   that information.  Due to this, before the offerer has received the
>   answer, the offerer will not be able to associate an RTP stream with
>   the correct "m=" line using the SSRC value associated with the RTP
>   stream.  In addition, the offerer and answerer may start using new
>   SSRC values mid-session, without informing each other using the SDP
>   'ssrc' attribute.
> 
>   In order for an offerer and answerer to always be able to associate
>   an RTP stream with the correct "m=" line, the offerer and answerer
>   using the BUNDLE extension MUST support the mechanism defined in
>   Section 14, where the offerer and answerer includes the
>   identification-tag (provided by the remote peer) associated with an
>   "m=" line in the RTP Streams and in RTCP SDES packets part of a
>   BUNDLE group.
> 
>   The mapping from an SSRC to an identification-tag is carried in RTCP
>   SDES packets or in RTP header extensions (Section 14).  Since a
>   compound RTCP packet can contain multiple RTCP SDES packets, and each
>   RTCP SDES packet can contain multiple chunks, an RTCP packet can
>   contain several SSRC to identification-tag mappings.  The offerer and
>   answerer maintain tables mapping RTP streams identified by SSRC to
>   "m=" lines identified by the identification-tag.  These tables are
>   updated each time new information that affects how packets should be
>   processed and routed are received.
> 
>   To prepare for demultiplexing RTP streams to the correct "m=" line,
>   the following steps MUST be followed for each BUNDLE group based on
>   the SDP signalling information.
> 
>      Construct a table mapping MID to "m=" line for each "m=" line in
>      this BUNDLE group.  Note that an "m=" line may only have one MID.
> 
>      Construct a table mapping incoming RTP streams (SSRCs) to their
>      "m=" line for each "m=" line in this BUNDLE group and for each RTP
>      stream explicitly signalled for receiving in that "m=" line.
> 
>      Construct a table mapping payload types to "m=" line for each "m="
>      line in the BUNDLE group and for each payload type configured for
>      receiving in that "m=" line.  If any payload type is configured
>      for receiving in more than one "m=" line in the BUNDLE group, do
>      not it include it in the table.
> 
>   Note that for each of these tables, there can only be one mapping for
>   any given key (MID, SSRC, or PT).  In other words, the tables are not
>   multimaps.
> 
>   As "m=" lines are added or removed from the BUNDLE groups, or their
>   configurations are changed, the tables above MUST also be updated.
> 
> 
> 
> Name                      Expires July 31, 2017                 [Page 3]
> 
> Internet-Draft              Abbreviated-Title               January 2017
> 
> 
>   Received RTP packets that are syntactically correct are processed by
>   the RTP/RTCP protocol implementation for statistics etc.  After this
>   processing they need to be routed to the higher layer context
>   associated with the "m=" line within the BUNDLE group.  Somewhere in
>   the process where an received RTP packet is processed to be delivered
>   to the higher layer by RTP the matching step below MUST be performed:
> 
>   1.  Reception of an RTP packet for an RTP stream that has an existing
>       mapping in the RTP stream to m= line table.  Before proceeding in
>       delivering the packet to the higher layer context according to
>       the RTP stream to "m=" line mapping table the following checks
>       MUST be performed:
> 
>       A.  If the packet carries an RTP header extension with a SDES MID
>           value that is not in the table mapping MID to "m=" line, then
>           do not deliver the RTP packet to higher layers.
> 
>       B.  If the packet carries an RTP header extension with a SDES MID
>           value that is in the table mapping MID to "m=" line, and the
>           value indicates a different "m=" line than the current RTP
>           stream to "m=" line mapping table, then update the RTP stream
>           to "m=" line mapping.
> 
>   2.  Reception of an RTP packet for an RTP stream that has no existing
>       mapping to an m= line.  In this case the following actions MUST
>       be performed:
> 
>       A.  If the packet carries an RTP header extension with a SDES MID
>           value that is in the table mapping MID to "m=" line, then
>           create an entry in the RTP stream to "m=" line mapping table
>           for this RTP stream (SSRC).  Then deliver the RTP packet to
>           the "m=" line context of the created mapping and stop.
> 
>       B.  If the packet carries a Payload Type that is in the payload
>           type table, then create an entry in the RTP stream to "m="
>           line mapping table for this RTP stream (SSRC).  Then deliver
>           the RTP packet to the "m=" line context of the created
>           mapping and stop.
> 
>       C.  Otherwise do not deliver the RTP packet to higher layers.
>           Note, this includes unknown MID values.
> 
>   For each RTCP packet received (including each RTCP packet that is
>   part of a compound RTCP packet), the RTCP packet needs to be
>   processed by the RTP/RTCP implementation and relevant information and
>   data from the RTCP packets needs to be routed to the appropriate
>   handler for the related RTP streams.  The appropriate handler is
>   determined by using the RTP stream to "m=" line mapping table.
> 
> 
> 
> Name                      Expires July 31, 2017                 [Page 4]
> 
> Internet-Draft              Abbreviated-Title               January 2017
> 
> 
>   On reception of any compound RTCP packet prior to dispatching the
>   received information and data, if there is an RTCP SDES packet
>   included that SHOULD be processed first.  If that SDES packet
>   contains SDES MID entries, this can results in updates and additions
>   to the RTP stream to "m=" line mapping table.  Thus each of the SDES
>   MID items are processed and the current table entries are checked if
>   the corresponding MID value matches the current RTP stream to "m="
>   line mapping, else the entry is updated.  If there is no RTP stream
>   to "m=" line table mapping entry for the received SDES item's SSRC,
>   such an entry is created.  Note, that in the process of updating the
>   table entries, update flap suppression as discussed in Section 4.2.6
>   of [RFC7941] should be considered.
> 
>   The various different RTCP packets as well as their various sub
>   parts, such as the various RTCP Feedback message types, relates to
>   the RTP streams in a couple of different ways.  The currently known
>   patterns are the following:
> 
>   Reports on outgoing RTP streams:  For all RTP streams that this
>      endpoint is the source of, it can expect to receive report blocks
>      of several types identified as relating to an outgoing stream.
>      The basic pattern for these blocks are that the RTCP packet header
>      identifies the source of the reports, as identified by an SSRC,
>      and containing one or more report blocks, where each report block
>      identifies the RTP stream, using the SSRC, the report relates to.
>      For this pattern the relevant report information is provided to
>      the higher layer associated with the "m=" line the RTP Stream to
>      "m=" line table identifies.  The source SSRC as identifier of the
>      endpoint that the report originates are relevant for interpreting
>      the information, but not necessarily for routing.  Example of this
>      is pattern are:
> 
>      Sender Report (SR) and Receiver Report (RR)  The basic receiver
>         report blocks from RFC3550 start with the SSRC of the RTP
>         stream they report on.
> 
>      Extended Reports (XR):  RFC3611 is a framework that enables a
>         large number of different reports.  However, a large number of
>         these report formats are reporting on specific RTP streams and
>         thus each individual report of these types contains a SSRC
>         field to identify the RTP stream.
> 
>   RTCP Feedback Messages for outgoing RTP streams:  The RFC 4585 RTCP
>      feedback messages allow for a number of different type of feedback
>      messages.  However, the RTCP feedback message header contains the
>      SSRC identifying the source of the feedback messages as well as
>      the actual type of the feedback.  Some of the feedback messages
>      also uses the target SSRC field in the header to identify which
> 
> 
> 
> Name                      Expires July 31, 2017                 [Page 5]
> 
> Internet-Draft              Abbreviated-Title               January 2017
> 
> 
>      RTP stream the feedback is related to.  For these types all the
>      FCI entries, if multiple ones are forwarded to the identified
>      handler.  Examples of this pattern are:
> 
>      Picture Loss Indication (PLI):  RFC 4585 (PT=PSFB, FMT=1).
> 
>      Slice Loss Indication (SLI):  RFC 4585 (PT=PSFB, FMT=2).
> 
>      Reference Picture Selection Indication (RPSI):  RFC 4585 (PT=PSFB,
>         FMT=3).
> 
>      Generic NACK:  [RFC4585] (PT=RTPFB and FMT=1).
> 
>      Other feedback messages includes the target SSRC in the Feedback
>      Control Information (FCI).  Here each FCI needs to be processed
>      and the SSRC field identified.  And the individual FCI combined
>      with the RTCP packet header context needs to be forwarded to the
>      identified handler.  Example of this pattern are:
> 
>      Full Intra Request (FIR):  [RFC5104] (PT=PSFB, FMT=4).
> 
>      Temporal-Spatial Trade-off Request (TSTR):  [RFC5104] (PT=PSFB,
>         FMT=5).
> 
>      Temporal-Spatial Trade-off Notification (TSTN):  This [RFC5104]
>         defined message (PT=PSFB, FMT=5) is actually a notifciation in
>         response to a TSTR this endpoint sent using the SSRC the FCI
>         identifies as source.
> 
>      H.271 Video Back Channel Message (VBCM):  [RFC5104] (PT=PSFB,
>         FMT=7).
> 
>      Layer Refresh Request (LRR):  [I-D.ietf-avtext-lrr] (PT=PSFB,
>         FMT=TBD).
> 
>   Descriptive or Notifications for an incoming RTP stream:  There exist
>      some RTCP packet types that provides additional information or
>      notifies about events related to the RTP stream identified.  In
>      these cases the RTP stream is identified using the SSRC field
>      value, and the information is provided to the higher layer
>      associated with the "m=" line for the incoming RTP stream as
>      identified by the current RTP stream to "m=" line table.  For this
>      type of pattern it is common that the RTCP packets and information
>      is repeated, either periodically (e.g.  SDES items), or for a
>      duration (e.g.  BYE), thus suppression of repetitions can be
>      considered.  Examples of these are:
> 
> 
> 
> 
> 
> Name                      Expires July 31, 2017                 [Page 6]
> 
> Internet-Draft              Abbreviated-Title               January 2017
> 
> 
>      Source Description (SDES) RTCP Packet:  The base RTP/RTCP protocol
>         [RFC3550] defines SDES RTCP packets as a way of providing per
>         source (SSRC) specific information about the source.  An SDES
>         packet contains zero or more chunks, where a chunk contains the
>         SSRC for the source being described by the one or more items
>         included in the chunk.  Forward the SDES items in each chunk to
>         the RTP stream's handler.
> 
>      Goodbye (BYE) RTCP Packet:  This RTP/RTCP protocol [RFC3550]
>         mechanism indicates that a particular RTP stream is leaving the
>         RTP session.  Thus, a most relevant event to inform the handler
>         for this RTP steam of.
> 
>   Third Party Targeted Reports or Feedback:  There exist some
>      multiparty RTP Topologies that results in that an endpoint
>      receives third party reports (SR [RFC3550], RR [RFC3550] or XR
>      [RFC3611]) as well as Feedback Messages [RFC4585] that relates to
>      or targets an SSRC that originates from another endpoint, and
>      where the source of the RTCP packet is also another endpoint.
>      This type of packets should be forwarded to the higher layer
>      function dealing with the third party reporting.  And if none
>      exist then they can be suppressed.  As the third party handler can
>      be focused on determining the conditions for the source of the
>      reports and feedback, or focused on how the RTP stream source
>      progresses, or both recommendations can't be made.
> 
>   APP Packets:  The RTCP APP Packets [RFC3550] are a mechanism that
>      enables experimentation.  The APP packet only specifies the source
>      of the packet.  Thus this information can be related to any of the
>      "m=" lines, thus deliver a copy of the packet to each "m=" line or
>      an APP specific handler.
> 
> -- 
> Cheers
> 
> Magnus Westerlund
> 
> ----------------------------------------------------------------------
> Services, Media and Network features, Ericsson Research EAB/TXM
> ----------------------------------------------------------------------
> Ericsson AB                 | Phone  [phone]
> Färögatan 6                 | Mobile [phone]
> SE-164 80 Stockholm, Sweden | mailto: [e-mail]
> ----------------------------------------------------------------------
> 
> _______________________________________________
> mmusic mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/mmusic
>