Re: [AVTCORE] Clarifying the H.264 RTP payload format specification's text on when to set the RTP "M" bit

["Wang, Ye-Kui" <[e-mail]>]

I share Mo's opinion here that there is no need to add additional text to the H.264 or the H.265 draft in this aspect, rather the semantics of the M bit is sufficient.

First of all, implementers are expected to have basic knowledge of the video coding specification itself. If we try to explain all the coding-level details, there can be a lot.

Secondly, as I mentioned in the avtcore session in Berlin, the M bit for H.264 payload format is not reliable because RFC 6184 supports multi-time aggregation packets (MTAPs). This is not an issue any more in the H.265 payload as MTAPs are not supported anymore, thus the M bit can be a reliable indication. Of course, bad packetization/sender implementations can make that unreliable - but bad packetization/sender implementations can make anything unreliable - thus they don't count.

Lastly, the suggested text is too vague and even incorrect. For example, in the following copied part: what is "the current NAL unit"? Also, it is possible for non-VCL NAL unit to be between two VCL NAL units in an AU, the following language would suggest that the first of the two VCL NAL unit ends the AU - this is definitely incorrect.
    The current NAL unit ends an access unit if it is a VCL NAL unit, and if either:
            - the next NAL unit is not a VCL NAL unit, or
            - the next NAL unit is a VCL NAL unit, and the high-order bit of the first byte after its NALU header is 1.

BR, YK

From: [e-mail] [mailto:[e-mail]] On Behalf Of Ross Finlayson
Sent: Sunday, August 04, 2013 2:52 PM
To: [e-mail]
Subject: Re: [AVTCORE] Clarifying the H.264 RTP payload format specification's text on when to set the RTP "M" bit

If you are willing to detect the last NALU of a frame by waiting for the first VCL NALU of the next frame, that is very simple in both H.264 and H.265. The first bit after the VCL NALU header is set if it's the first VCL NALU of a frame. This works in H.265 because the slice header explicitly has such a flag.

Thanks.  This is exactly the kind of information that would be useful for implementers.



Note that RTP receivers are technically forbidden to rely on the marker bit. But many implementations detect whether it is reliable, then use it as an optimization to minimize latency if reliable.

Yes, it's unfortunate that there may be implementations out there that do not set the RTP 'M' bit correctly.  But to reduce the likelihood of this happening, we should try to make our RTP payload format specifications as clear as is reasonable about when to set the 'M' bit.  (It is common - in non-interactive environments - for RTP transmitting applications to obtain their encoded data as a byte stream.)



 I don't think the draft needs any more text beyond what it currently says.

On the contrary - I think you've explained that some additional text would be both useful and reasonable.  (Once again, because we're not MPEG, we should not feel the need to fully-specify only the behavior that's required by RTP *receivers*.)

So I suggest adding the following paragraph to the H.265 RTP payload format specification, just after the existing 'M bit' text:

----------
Unfortunately the contents of a NAL unit, alone, does not tell a RTP sender implementation whether or not the NAL unit ends an access unit.  Instead, the implementation can obtain this information separately, from the encoder.  If, however, this information is not directly available from the encoder (e.g., because the implementation is sending data that consists solely of a sequence of pre-encoded NAL units), then it must instead inspect the next NAL unit, to determine whether or not the current NAL unit ends an access unit.  The following rule can be used:
    The current NAL unit ends an access unit if it is a VCL NAL unit, and if either:
          - the next NAL unit is not a VCL NAL unit, or
          - the next NAL unit is a VCL NAL unit, and the high-order bit of the first byte after its NALU header is 1.
----------

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/