Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

Hi YK,

Good point on trailing non-VCL NAL units.

That brings up another point on non-VCL NAL units. I assume the non-VCL NAL units defined in this draft (AP/FU) can’t appear in the bitstream processed by the RTP sender using this informative note. That is, the RTP sender must be processing an elementary stream that only contains NAL units specified in the H.265 standard itself [0-47], and can’t be processing a bitstream that came from another RTP sender which may have inserted unspecified NAL units [48-63] like AP/FU, since that could yield improper results for this note.

Or do we actually need to split AP and FU each into two non-VCL units, one in the leading non-VCL range [48-55] and another in the trailing non-VCL range [56-63]? I would like to avoid that complexity, but that seems to be what the rules in H.265 7.4.2.4.4 require when using the unspecified range [48-63]. Do you recall the motivation for rules on subranges of the unspecified range?

Also, it may be useful to indicate in the informative note how to differentiate VCL NAL units [0-31] from non-VCL NAL units [32-63], i.e. the MSB of the 6-bit NAL unit type is 1 in non-VCL NAL units.

Regards,
Mo

From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com]
Sent: Tuesday, August 27, 2013 9:36 PM
To: Mo Zanaty (mzanaty)
Cc: Ross Finlayson; avt@ietf.org; payload@ietf.org
Subject: RE: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

Hi Mo,

"The content of a NAL unit" can tell whether the NAL unit is the last slice of a picture, but the last NAL unit of an AU can also be a non-VCL NAL unit. Considering this, "the content of a NAL unit" does not tell whether a NAL unit is the last NAL unit of an AU.

BR, YK

From: Mo Zanaty (mzanaty) [mailto:mzanaty@cisco.com]
Sent: Tuesday, August 27, 2013 3:58 PM
To: Wang, Ye-Kui
Cc: Ross Finlayson; avt@ietf.org; payload@ietf.org
Subject: Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

I suggest replacing "The content of a NAL unit" with "The NAL unit header". The content does reveal end of access unit boundaries if you're willing to parse/decode it all.

Mo

On Aug 27, 2013, at 12:51 AM, "Wang, Ye-Kui" <yekuiw@qti.qualcomm.com<mailto:yekuiw@qti.qualcomm.com>> wrote:
Hi Ross,

Sorry for a slow response. I started to edit this draft today, and have included the following note after the semantics of the M bit:

Informative note: The content of a NAL unit does not tell whether or not the NAL unit is the last NAL unit, in decoding order, of an access unit.  An RTP sender implementation may obtain this information from the video encoder.  If, however, the implementation cannot obtain this information directly from the encoder, e.g., when the stream was pre-encoded, and also there is no timestamp allocated for each NAL unit, then the sender implementation can inspect subsequent NAL units in decoding order to determine whether or not the NAL unit is the last NAL unit of an access unit as follows.  A NAL unit naluX is the last NAL unit of an access unit if the next VCL NAL unit naluY in decoding order has the high-order bit of the first byte after its NAL unit header equal to 1, and all NAL units between naluX and naluY, when present, have nal_unit_type in the range of 32 to 35, inclusive, equal to 39, in the range of 41 to 44, inclusive, or in the range of 48 to 55, inclusive.

Would you be fine with this wording?

BR, YK

From: avt-bounces@ietf.org<mailto:avt-bounces@ietf.org> [mailto:avt-bounces@ietf.org] On Behalf Of Ross Finlayson
Sent: Monday, August 12, 2013 12:25 PM
To: avt@ietf.org<mailto:avt@ietf.org>; payload@ietf.org<mailto:payload@ietf.org>
Subject: Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

You just need one more little step to get everything right here, instead of thinking of compromising the implementation ☺

That is, read subclause 7.4.2.4.4 of the HEVC spec. The most relevant paragraphs are copied below for convenience (the first paragraph itself contains the answer to your question, but the second paragraph may also provide helpful information)

Thanks for the info; that was useful.

Here, then is the updated text of a paragraph - to be added after the existing 'M bit' text - that clarifies when/how to set it:
----------
Unfortunately the contents of a NAL unit, alone, do not tell a RTP sender implementation whether or not the NAL unit ends an access unit.  Instead, the implementation may be able to obtain this information separately, from the encoder.  If, however, the implementation cannot obtain this information directly from the encoder (e.g., because the implementation is sending data that consists solely of a sequence of pre-encoded NAL units), then it must instead inspect subsequent NAL units, to determine whether or not the NAL unit ends an access unit.  If the NAL units have already been timestamped, then these timestamps can be used to determine the access unit boundaries.  Otherwise, the access unit boundaries (and times) must be determined by inspecting the NAL units' contents - specifically, the "nal_unit_type" and (for VCL NAL units) the "first_slice_segment_in_pic_flag".  The following rule can be used:
           A NAL unit (X) ends an access unit if the next-occurring VCL NAL unit (Y) has the high-order bit of the first byte after its NAL unit header equal to 1, and all NAL units, if any, between (X) and (Y) have a "nal_unit_type" in one of the following ranges: [32,35], 39, [41,44], or [48,55].
----------

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

_______________________________________________
Audio/Video Transport Core Maintenance
avt@ietf.org<mailto:avt@ietf.org>
https://www.ietf.org/mailman/listinfo/avt