Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

[Ross Finlayson <[e-mail]>]

> Now I understand your intention better – did not see that the note was intended for sender implementers. 

OK, thanks.


>  For this purpose, indeed (as Mo pointed out earlier?) that first of all, the sender needs make sure the RTP timestamp for each RTP packet is correct. As long as RTP timestamp is correct, it is also easy to know when the M bit should be set. Or are you saying that there is no problem with RTP timestamp setting but there is a problem for the M bit setting? If yes, please clarify why.

Figuring out the RTP timestamp isn't (nearly as much of) a problem, because this can be done by knowing the stream's frame rate.  But in this email thread, I'm focusing only on the RTP 'M' bit, because for it (unlike the timestamp) there's less of an incentive for a sender implementation to 'get it right'.  I'm worried that if the RTP payload format specification doesn't make it crystal clear when a NAL unit ends an 'access unit', then an implementer might be tempted to just not bother setting the bit at all.  I don't want to see that happen.  Hence this email thread.


> I have no problem to putting some information in the line as you suggested, if the suggested text is good. However, the suggested wording below still has a few problems (or shortcomings).
>  
> ----------
> Unfortunately the contents of a NAL unit, alone, does not tell a RTP sender implementation whether or not the NAL unit ends an access unit.  Instead, the implementation can obtain this information separately, from the encoder.  If, however, this information is not available directly from the encoder (e.g., because the implementation is sending data that consists solely of a sequence of pre-encoded NAL units), then it must instead inspect subsequent NAL units, to determine whether or not the NAL unit ends an access unit.  The following rule can be used:
>     A NAL unit ends an access unit if it is a VCL NAL unit, and the next-occurring VCL NAL unit has the high-order bit of the first byte after its NALU header equal to 1.
> ----------
>  
> The problems (or shortcomings) are:
> -          I don’t think it is good to say that the fact that a NAL unit itself does not indicate whether it is the last one of an AU is unfortunate.

Why not?  It *is* "unfortunate".  I'm not intending to 'insult' H.265 here :-)


> -          Saying that the information can be obtained from the encoder is vague to me. I guess you meant the bitstream instead, as the encoder may be absent e.g. when dealing pre-encoded video.

I think you may have been confused by my use of the word "can" in the 2nd sentence of my proposed paragraph.  Here is a rewording of the 2nd (& 3rd) sentences; I hope this will make things clearer:

	"Instead, the implementation may be able to obtain this information separately, from the encoder.  If, however, the implementation cannot obtain this information directly from the encoder (e.g., because the implementation is sending data that consists solely of a sequence of pre-encoded NAL units), then it must instead inspect subsequent NAL units, to determine whether or not the NAL unit ends an access unit."


> -          Last but not least, an AU may also end with a non-VCL NAL unit, e.g. a filler data NAL unit, an end of sequence NAL unit, an end of bitstream NAL unit, a suffix SEI NAL unit, etc.

Arggh!  Now, I hope, you see the problem :-)  Can you then help me figure out a more accurate wording for the rule?  In particular:
	1/ Suppose we have a VCL NAL unit, followed immediately by another VCL NAL unit with the first slice header bit set.  In this case, there's no doubt: The first VCL NAL unit definitely ends an 'access unit' - correct?
	2/ Suppose we have a VCL NAL unit, followed by a sequence of one or more non-VCL NAL units, followed immediately by a VCL NAL unit with the first slice header bit set.  In this case, which (if any) of the non-VCL NAL units ends an 'access unit'?  Can we figure out a rule (ideally, an easy rule, based solely on the "nal_unit_type"s) for this?


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/