Re: [payload] Adam Roach's Discuss on draft-ietf-payload-rtp-ancillary-10: (with DISCUSS and COMMENT)

["Ali C. Begen" <[e-mail]>]

On Wed, Oct 11, 2017 at 12:02 AM, Adam Roach <[e-mail]> wrote:

> Here are the salient points that pertain to my discuss:
>
>
>    1. This mechanism is broken and must not be published unless there is
>    an explicit way to correlate ancillary data with its corresponding video
>    stream.
>
>    2. LS is 100% the wrong mechanism to do so.
>
>    3. I believe that what is needed here is in the spirit of what FID is
>    intended to convey, although I take Thomas' point that there is some
>    language in the specific definition of FID that he believes makes it
>    unsuitable for this purpose.
>
>
> Given that we seem to have reached an impasse on the final point in
> particular, I would propose that the most straightforward solution here is
> to define a new group semantic (e.g., "a=group:ANC") that is explicitly
> intended to pair ancillary data with its corresponding video stream. This
> should take at most one short paragraph and an additional IANA registration.
>

Ah, did not see your email before my earlier email. I'd agree but I would
do the new grouping semantics for generic metadata not just for ANC.

-acbegen


>
>
> /a
>
>
>
> On 10/10/17 1:44 AM, Ali C. Begen wrote:
>
> Hi Thomas
>
> Please go ahead and submit a revision. Then reach out to the individual
> DISCUSS holders to clear theirs. And then lets focus on Adam's discuss.
>
> Thanks
> -acbegen (your payload wg co-chair)
>
> On Tue, Oct 10, 2017 at 3:06 AM, Thomas Edwards <[e-mail]>
> wrote:
>
>> I will note that I have uploaded a new version,
>> draft-ietf-payload-rtp-ancillary-11, to address most of the comments
>> from the IESG evaluation.
>>
>> draft-ietf-payload-rtp-ancillary-11, Section 4.1 “Grouping ANC Streams
>> with other Media Streams” has two examples.
>>
>> In the first SDP example, two “m=” lines are present (with each having a
>> different “c=” line), one for uncompressed video, and one for ancillary
>> data.  The two RTP sessions are identified with RFC 5888 Media Stream
>> Identification Attributes (mid), and are grouped together using Lip
>> Synchronization (LS), in a fashion similar to the Lip Synchronization of
>> video and audio streams in RFC 5888 Section 7.1 “Example of LS”.
>>
>> In the second SDP example, a single “m=” line is present without any
>> grouping, implying any synchronization would be from SSRC/CNAME linkage
>> and/or RTCP.
>>
>> The most likely implementation of this RTP payload in professional video
>> production would be where multiple endpoints are contributing elementary
>> streams of media: video, audio channels or channel groups, and SMPTE ST
>> 291-1 ancillary data sources.  Those endpoints (microphones, cameras,
>> Closed Caption generators, etc.) are all likely to be different, thus with
>> different SSRCs and CNAMEs, and transmitting to separate multicast IP
>> addresses and UDP ports.  However, they will likely be transmitted with a
>> commonly synchronized clock as per RFC 7273 “RTP Clock Source Signalling”,
>> most likely IEEE 1588 PTP.
>>
>> The individual elementary media streams will be composed together by
>> receivers as required for production.  For example, for an English language
>> service, an English audio channel group of AES67/RFC 3190 RTP audio would
>> be composed together with the RFC 4175 video and the ST 291-1 ancillary
>> data flow of English language Closed Captions ancillary data and SMPTE
>> 2016-1 AFD ancillary data.    Meanwhile the Spanish language service would
>> have the same video and AFD ancillary data, but would have Spanish language
>> audio and Spanish Closed Captions.  Likely it would be a distribution
>> quality encoder (perhaps H.264 encoder) that subscribes to the different
>> streams in order to create the final service.
>>
>> Every time I read about “FID” in RFC 5888, I am struck by the sentence in
>> Section 8 “Flow Identification (FID)” that says “there are situations where
>> a single media instance (e.g., an audio stream or a video stream) is sent
>> using more than one RTP session,” which I believe is not the case here.
>> For example, video is one media instance, and ancillary data is a second
>> media instance.  You can’t “add” video and ancillary data together, like
>> you can add together voice audio and DTMF tone audio (as in RFC 5888
>> Section 8.2).
>>
>> RFC 5888 Section 8.4 “FID Semantics” also states: “It is assumed that the
>> application uses only one codec at a time to encode the media produced.
>> This codec MAY change dynamically during the session, but at any particular
>> moment, only one codec is in use.”  Again, this does not seem relevant to
>> the SDP description of multiple RTP streams of video, audio, and ancillary
>> data intended for synchronized presentation.  All three “codecs” are in use
>> at all times for their particular media stream.  And in particular
>> regarding draft-ietf-payload-rtp-ancillary, there is only one “codec”
>> ever used, the encapsulation of SMPTE ST 291-1 ancillary data packets into
>> RTP.
>>
>> I could really use the help of the IESG to see how we can move forward
>> regarding the open DISCUSS item on draft-ietf-payload-rtp-ancillary.
>>
>> (I will note that SMPTE is eagerly awaiting the publishing of this RFC
>> for use as a normative reference in the upcoming ST 2110-40 standard, see:
>> https://www.smpte.org/st-2110).
>>
>> Thanks!
>>
>> -Thomas Edwards
>>  FOX
>>
>> On 10/4/17, 9:27 AM, "Roni Even" <[e-mail]> wrote:
>>
>>     Hi Colin,
>>
>>     I agree.
>>
>>     There are two cases. The assumption is that both video and ANC have
>> the same source and have same CNAME (Thomas - is this correct?)
>>
>>
>>
>>     1. Both video and ANC are in the same RTP session (same m-line)  will
>> be synchronized.
>>
>>
>>
>>     The second is the one we are discussing:
>>
>>     2. The video and ANC are in separate RTP sessions (two m-lines). If
>> both are a single flow (single RTP stream) should group using FID. If they
>> are not a single RTP stream (flow) should use LS.
>>
>>
>>
>>     I can agree that it looks like a single flow (FID) but want
>> clarification from the authors (Thomas)
>>
>>
>>
>>
>>
>>
>>
>>     Roni
>>
>>
>>
>>     > -----Original Message-----
>>
>>     > From: Colin Perkins [mailto:[e-mail]]
>>
>>     > Sent: יום ד 04 אוקטובר 2017 01:50
>>
>>     > To: Roni Even
>>
>>     > Cc: Adam Roach; Thomas Edwards; The IESG; [e-mail];
>> draft-
>>
>>     > [e-mail]; [e-mail];
>> [e-mail]
>>
>>     > Subject: Re: [payload] Adam Roach's Discuss on
>> draft-ietf-payload-rtp-
>>
>>     > ancillary-10: (with DISCUSS and COMMENT)
>>
>>     >
>>
>>     > Roni,
>>
>>     >
>>
>>     > I said nothing about multicast vs unicast, since I don’t think that
>> distinction is
>>
>>     > important.
>>
>>     >
>>
>>     > Sending the streams on separate m= lines or sending using a single
>> a m= line
>>
>>     > both work. The draft doesn’t need to discuss this, since it’s
>> standard RTP as
>>
>>     > would apply to any RTP session. If you use separate m= lines, then
>> you need
>>
>>     > a grouping mechanism. The choice of grouping mechanism is what we’re
>>
>>     > discussing.
>>
>>     >
>>
>>     > Colin
>>
>>     >
>>
>>     >
>>
>>     >
>>
>>     >
>>
>>     > > On 3 Oct 2017, at 23:19, Roni Even <[e-mail]> wrote:
>>
>>     > >
>>
>>     > > Colin,
>>
>>     > > We still need some feedback from Thomas, in a previous email
>> response to
>>
>>     > Adam he gave a reason for using multiple RTP sessions:
>>
>>     > >
>>
>>     > > "The challenge is that the use case of the Ancillary I-D within
>> the SMPTE ST
>>
>>     > 2110 architecture would generally require separate network
>> destination
>>
>>     > addresses for the different elements (video, audio, ancillary data)
>> so they
>>
>>     > can be flexibly composed within broadcast plants at the network
>> level.  Your
>>
>>     > alternative SDP example appears to place both the video and the
>> ancillary
>>
>>     > data onto the same destination address (
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__233.
>> 252.0.1&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-
>> EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=
>> dOxHA7dn5d2eo_tU3divd4ks6mmXPhk5WmodLULHUPY&s=
>> srzDHDG2LTcTBJexIDta4-p9142SYkQ2gDH3S8PMdZg&e=).  Is there an
>>
>>     > appropriate SDP mechanism to communicate the intimate relationship
>>
>>     > between the RTP streams, but also have them transmitted on separate
>>
>>     > destination IP addresses (and synchronized)?
>>
>>     > >
>>
>>     > > (In production practice, this may not be a significant issue as
>> the RTP
>>
>>     > streams to be combined by IP receivers for conversion to SDI/HDMI
>> or for
>>
>>     > input to a distribution encoder may be specified by a control
>> system directly
>>
>>     > to the receiver using an API that isn’t just an SDP file.)"
>>
>>     > >
>>
>>     > > I want to assume that by different destination addresses it
>> refers to
>>
>>     > multicast addresses like above and not to different unicast IP
>> addresses.
>>
>>     > > You are right that for unicast it will make sense to use the same
>> RTP session
>>
>>     > with the same SSRC.
>>
>>     > >
>>
>>     > > Roni
>>
>>     > >
>>
>>     > >> -----Original Message-----
>>
>>     > >> From: Colin Perkins [mailto:[e-mail]]
>>
>>     > >> Sent: יום ד 04 אוקטובר 2017 00:46
>>
>>     > >> To: Roni Even
>>
>>     > >> Cc: Adam Roach; Thomas Edwards; The IESG;
>> [e-mail];
>>
>>     > >> draft- [e-mail]; [e-mail];
>>
>>     > >> [e-mail]
>>
>>     > >> Subject: Re: [payload] Adam Roach's Discuss on
>>
>>     > >> draft-ietf-payload-rtp-
>>
>>     > >> ancillary-10: (with DISCUSS and COMMENT)
>>
>>     > >>
>>
>>     > >>
>>
>>     > >>> On 3 Oct 2017, at 21:40, Roni Even <[e-mail]>
>> wrote:
>>
>>     > >>>
>>
>>     > >>> Hi Colin,
>>
>>     > >>> I re-read RFC5888 and tried to understand what is the difference
>>
>>     > >>> between FID and LS I was trying to understand what is a flow and
>>
>>     > >>> from section 8.3
>>
>>     > >>>
>>
>>     > >>> "The previous examples show that the definition of a media
>> stream in
>>
>>     > >>> [RFC2326] does not cover some scenarios.  It cannot be assumed
>> that
>>
>>     > >>> a  single media instance maps into a single RTP session.
>> Therefore,
>>
>>     > >>> we  introduce the definition of a media flow:
>>
>>     > >>>
>>
>>     > >>>     A media flow consists of a single media instance, e.g., an
>> audio
>>
>>     > >>>     stream or a video stream as well as a single whiteboard or
>> shared
>>
>>     > >>>     application group.  When using RTP, a media flow comprises
>> one or
>>
>>     > >>>     more RTP sessions."
>>
>>     > >>>
>>
>>     > >>> It looks to me that a media flow is a SINGLE media stream that
>> spans
>>
>>     > >>> over
>>
>>     > >> multiple RTP sessions (the term media instance is a what we call
>> a
>>
>>     > >> media stream in the taxonomy). Section 8.5 (8.5.1) enhance this
>> by
>>
>>     > >> saying when FID cannot be used " FID semantics are useful when
>> the
>>
>>     > application only uses
>>
>>     > >> one codec at   a time"  which again describe a flow as a single
>> media
>>
>>     > stream.
>>
>>     > >>>
>>
>>     > >>> I am not sure how ANC and the video relates to each other this
>> is
>>
>>     > >>> what I am
>>
>>     > >> asking Thomas to be more clear. If the video and ANC are one
>> stream
>>
>>     > >> over multiple RTP sessions FID is correct otherwise I think the
>> LS should be
>>
>>     > used.
>>
>>     > >>
>>
>>     > >> My understanding is that the video and ancillary data are the
>> same
>>
>>     > >> stream. In broadcast TV the ancillary data is sent in-band, in
>> the blanking
>>
>>     > intervals.
>>
>>     > >>
>>
>>     > >>> My reading on  the ancillary draft is that ANC and the video are
>>
>>     > >>> what we call
>>
>>     > >> a single RTP stream but this is not what Thomas says.
>>
>>     > >>
>>
>>     > >>
>>
>>     > >> I expect one could send the ancillary data using the same SSRC,
>> and
>>
>>     > >> the same 5-tuple, as the media data (in the same way comfort
>> noise
>>
>>     > >> and speech data can be sent), in which case you don’t need the
>>
>>     > >> grouping. If they’re sent as separate RTP streams, with separate
>> m=
>>
>>     > >> lines, then you need grouping. I believe FID makes sense for
>> this.
>>
>>     > >>
>>
>>     > >> --
>>
>>     > >> Colin Perkins
>>
>>     > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__csperki
>> ns.org_&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-
>> EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=
>> dOxHA7dn5d2eo_tU3divd4ks6mmXPhk5WmodLULHUPY&s=s2We2es-My1JRf
>> cPlYZEmqGCYYi9rf9Wc7dYRzxspsY&e=
>>
>>     > >>
>>
>>     > >>
>>
>>     > >>
>>
>>     > >
>>
>>     >
>>
>>     >
>>
>>     >
>>
>>     > --
>>
>>     > Colin Perkins
>>
>>     > https://urldefense.proofpoint.com/v2/url?u=https-3A__csperki
>> ns.org_&d=DwIGaQ&c=uw6TLu4hwhHdiGJOgwcWD4AjKQx6zvFcGEsbfiY9-
>> EI&r=lekNOOM5noV61zrPH3rwPyhtNnLLWoLEHgd0quQxly8&m=
>> dOxHA7dn5d2eo_tU3divd4ks6mmXPhk5WmodLULHUPY&s=s2We2es-My1JRf
>> cPlYZEmqGCYYi9rf9Wc7dYRzxspsY&e=
>>
>>     >
>>
>>     >
>>
>>     >
>>
>>
>>
>>
>>
>>
>
>