Re: [payload] Adam Roach's Discuss on draft-ietf-payload-rtp-ancillary-10: (with DISCUSS and COMMENT)

[Thomas Edwards <[e-mail]>]

I will note that I have uploaded a new version, draft-ietf-payload-rtp-ancillary-11, to address most of the comments from the IESG evaluation.

draft-ietf-payload-rtp-ancillary-11, Section 4.1 “Grouping ANC Streams with other Media Streams” has two examples.

In the first SDP example, two “m=” lines are present (with each having a different “c=” line), one for uncompressed video, and one for ancillary data.  The two RTP sessions are identified with RFC 5888 Media Stream Identification Attributes (mid), and are grouped together using Lip Synchronization (LS), in a fashion similar to the Lip Synchronization of video and audio streams in RFC 5888 Section 7.1 “Example of LS”.

In the second SDP example, a single “m=” line is present without any grouping, implying any synchronization would be from SSRC/CNAME linkage and/or RTCP.

The most likely implementation of this RTP payload in professional video production would be where multiple endpoints are contributing elementary streams of media: video, audio channels or channel groups, and SMPTE ST 291-1 ancillary data sources.  Those endpoints (microphones, cameras, Closed Caption generators, etc.) are all likely to be different, thus with different SSRCs and CNAMEs, and transmitting to separate multicast IP addresses and UDP ports.  However, they will likely be transmitted with a commonly synchronized clock as per RFC 7273 “RTP Clock Source Signalling”, most likely IEEE 1588 PTP.  

The individual elementary media streams will be composed together by receivers as required for production.  For example, for an English language service, an English audio channel group of AES67/RFC 3190 RTP audio would be composed together with the RFC 4175 video and the ST 291-1 ancillary data flow of English language Closed Captions ancillary data and SMPTE 2016-1 AFD ancillary data.    Meanwhile the Spanish language service would have the same video and AFD ancillary data, but would have Spanish language audio and Spanish Closed Captions.  Likely it would be a distribution quality encoder (perhaps H.264 encoder) that subscribes to the different streams in order to create the final service.

Every time I read about “FID” in RFC 5888, I am struck by the sentence in Section 8 “Flow Identification (FID)” that says “there are situations where a single media instance (e.g., an audio stream or a video stream) is sent using more than one RTP session,” which I believe is not the case here.  For example, video is one media instance, and ancillary data is a second media instance.  You can’t “add” video and ancillary data together, like you can add together voice audio and DTMF tone audio (as in RFC 5888 Section 8.2).  

RFC 5888 Section 8.4 “FID Semantics” also states: “It is assumed that the application uses only one codec at a time to encode the media produced.  This codec MAY change dynamically during the session, but at any particular moment, only one codec is in use.”  Again, this does not seem relevant to the SDP description of multiple RTP streams of video, audio, and ancillary data intended for synchronized presentation.  All three “codecs” are in use at all times for their particular media stream.  And in particular regarding draft-ietf-payload-rtp-ancillary, there is only one “codec” ever used, the encapsulation of SMPTE ST 291-1 ancillary data packets into RTP.

I could really use the help of the IESG to see how we can move forward regarding the open DISCUSS item on draft-ietf-payload-rtp-ancillary.

(I will note that SMPTE is eagerly awaiting the publishing of this RFC for use as a normative reference in the upcoming ST 2110-40 standard, see: https://www.smpte.org/st-2110).

Thanks!

-Thomas Edwards
 FOX

On 10/4/17, 9:27 AM, "Roni Even" <[e-mail]> wrote:

    Hi Colin,
    
    I agree.
    
    There are two cases. The assumption is that both video and ANC have the same source and have same CNAME (Thomas - is this correct?)
    
    
    
    1. Both video and ANC are in the same RTP session (same m-line)  will be synchronized. 
    
    
    
    The second is the one we are discussing:
    
    2. The video and ANC are in separate RTP sessions (two m-lines). If both are a single flow (single RTP stream) should group using FID. If they are not a single RTP stream (flow) should use LS.
    
    
    
    I can agree that it looks like a single flow (FID) but want clarification from the authors (Thomas)
    
    
    
    
    
    
    
    Roni
    
    
    
    > -----Original Message-----
    
    > From: Colin Perkins [mailto:[e-mail]]
    
    > Sent: יום ד 04 אוקטובר 2017 01:50
    
    > To: Roni Even
    
    > Cc: Adam Roach; Thomas Edwards; The IESG; [e-mail]; draft-
    
    > [e-mail]; [e-mail]; [e-mail]
    
    > Subject: Re: [payload] Adam Roach's Discuss on draft-ietf-payload-rtp-
    
    > ancillary-10: (with DISCUSS and COMMENT)
    
    > 
    
    > Roni,
    
    > 
    
    > I said nothing about multicast vs unicast, since I don’t think that distinction is
    
    > important.
    
    > 
    
    > Sending the streams on separate m= lines or sending using a single a m= line
    
    > both work. The draft doesn’t need to discuss this, since it’s standard RTP as
    
    > would apply to any RTP session. If you use separate m= lines, then you need
    
    > a grouping mechanism. The choice of grouping mechanism is what we’re
    
    > discussing.
    
    > 
    
    > Colin
    
    > 
    
    > 
    
    > 
    
    > 
    
    > > On 3 Oct 2017, at 23:19, Roni Even <[e-mail]> wrote:
    
    > >
    
    > > Colin,
    
    > > We still need some feedback from Thomas, in a previous email response to
    
    > Adam he gave a reason for using multiple RTP sessions:
    
    > >
    
    > > "The challenge is that the use case of the Ancillary I-D within the SMPTE ST
    
    > 2110 architecture would generally require separate network destination
    
    > addresses for the different elements (video, audio, ancillary data) so they
    
    > can be flexibly composed within broadcast plants at the network level.  Your
    
    > alternative SDP example appears to place both the video and the ancillary
    
    > data onto the same destination address (https://urldefense.proofpoint.com/v2/url?u=http-3A__233.252.0.1&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=dOxHA7dn5d2eo_tU3divd4ks6mmXPhk5WmodLULHUPY&s=srzDHDG2LTcTBJexIDta4-p9142SYkQ2gDH3S8PMdZg&e=).  Is there an
    
    > appropriate SDP mechanism to communicate the intimate relationship
    
    > between the RTP streams, but also have them transmitted on separate
    
    > destination IP addresses (and synchronized)?
    
    > >
    
    > > (In production practice, this may not be a significant issue as the RTP
    
    > streams to be combined by IP receivers for conversion to SDI/HDMI or for
    
    > input to a distribution encoder may be specified by a control system directly
    
    > to the receiver using an API that isn’t just an SDP file.)"
    
    > >
    
    > > I want to assume that by different destination addresses it refers to
    
    > multicast addresses like above and not to different unicast IP addresses.
    
    > > You are right that for unicast it will make sense to use the same RTP session
    
    > with the same SSRC.
    
    > >
    
    > > Roni
    
    > >
    
    > >> -----Original Message-----
    
    > >> From: Colin Perkins [mailto:[e-mail]]
    
    > >> Sent: יום ד 04 אוקטובר 2017 00:46
    
    > >> To: Roni Even
    
    > >> Cc: Adam Roach; Thomas Edwards; The IESG; [e-mail];
    
    > >> draft- [e-mail]; [e-mail];
    
    > >> [e-mail]
    
    > >> Subject: Re: [payload] Adam Roach's Discuss on
    
    > >> draft-ietf-payload-rtp-
    
    > >> ancillary-10: (with DISCUSS and COMMENT)
    
    > >>
    
    > >>
    
    > >>> On 3 Oct 2017, at 21:40, Roni Even <[e-mail]> wrote:
    
    > >>>
    
    > >>> Hi Colin,
    
    > >>> I re-read RFC5888 and tried to understand what is the difference
    
    > >>> between FID and LS I was trying to understand what is a flow and
    
    > >>> from section 8.3
    
    > >>>
    
    > >>> "The previous examples show that the definition of a media stream in
    
    > >>> [RFC2326] does not cover some scenarios.  It cannot be assumed that
    
    > >>> a  single media instance maps into a single RTP session.  Therefore,
    
    > >>> we  introduce the definition of a media flow:
    
    > >>>
    
    > >>>     A media flow consists of a single media instance, e.g., an audio
    
    > >>>     stream or a video stream as well as a single whiteboard or shared
    
    > >>>     application group.  When using RTP, a media flow comprises one or
    
    > >>>     more RTP sessions."
    
    > >>>
    
    > >>> It looks to me that a media flow is a SINGLE media stream that spans
    
    > >>> over
    
    > >> multiple RTP sessions (the term media instance is a what we call a
    
    > >> media stream in the taxonomy). Section 8.5 (8.5.1) enhance this by
    
    > >> saying when FID cannot be used " FID semantics are useful when the
    
    > application only uses
    
    > >> one codec at   a time"  which again describe a flow as a single media
    
    > stream.
    
    > >>>
    
    > >>> I am not sure how ANC and the video relates to each other this is
    
    > >>> what I am
    
    > >> asking Thomas to be more clear. If the video and ANC are one stream
    
    > >> over multiple RTP sessions FID is correct otherwise I think the LS should be
    
    > used.
    
    > >>
    
    > >> My understanding is that the video and ancillary data are the same
    
    > >> stream. In broadcast TV the ancillary data is sent in-band, in the blanking
    
    > intervals.
    
    > >>
    
    > >>> My reading on  the ancillary draft is that ANC and the video are
    
    > >>> what we call
    
    > >> a single RTP stream but this is not what Thomas says.
    
    > >>
    
    > >>
    
    > >> I expect one could send the ancillary data using the same SSRC, and
    
    > >> the same 5-tuple, as the media data (in the same way comfort noise
    
    > >> and speech data can be sent), in which case you don’t need the
    
    > >> grouping. If they’re sent as separate RTP streams, with separate m=
    
    > >> lines, then you need grouping. I believe FID makes sense for this.
    
    > >>
    
    > >> --
    
    > >> Colin Perkins
    
    > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__csperkins.org_&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=dOxHA7dn5d2eo_tU3divd4ks6mmXPhk5WmodLULHUPY&s=s2We2es-My1JRfcPlYZEmqGCYYi9rf9Wc7dYRzxspsY&e=
    
    > >>
    
    > >>
    
    > >>
    
    > >
    
    > 
    
    > 
    
    > 
    
    > --
    
    > Colin Perkins
    
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__csperkins.org_&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=dOxHA7dn5d2eo_tU3divd4ks6mmXPhk5WmodLULHUPY&s=s2We2es-My1JRfcPlYZEmqGCYYi9rf9Wc7dYRzxspsY&e=
    
    > 
    
    > 
    
    >