[AVTCORE] Clarifying the H.264 RTP payload format specification's text on when to set the RTP "M" bit

[Ross Finlayson <[e-mail]>]

As I noted in Wednesday morning's AVTCORE session, the proposed H.265 RTP payload format specification is not as clear as it could be about how a RTP sender knows when to set the "M" bit on an outgoing RTP packet.

Currently, the text says:

   Marker bit (M): 1 bit

      Set for the last packet of the access unit indicated by the RTP
      timestamp, in line with the normal use of the M bit in video
      formats, to allow an efficient playout buffer handling.  Decoders
      can use this bit as an early indication of the last packet of an
      access unit.

There is nothing inherently wrong with this.  However (because we're not in the MPEG world here :-) it's not sufficient to fully specify only how the receivers of RTP packets should behave; we should also try to specify, as clearly as possible, how the *senders* of RTP packets should behave.  We should try to give at least some guidance about when the NAL unit that's being sent ends an "access unit" because - for many implementations - it is not immediately obvious.

If I were amending the H.264 payload format specification - which of course I'm not - then I would, hypothetically, add a paragraph something like the following:

----------
Unfortunately the contents of a NAL unit, alone, does not tell a RTP sender implementation whether or not the NAL unit ends an access unit.  Instead, the implementation can obtain this information separately, from the encoder.  If, however, this information is not directly available from the encoder (e.g., because the implementation is sending data that consists solely of a sequence of pre-encoded NAL units), then it must instead inspect the next NAL unit, to determine whether or not the current NAL unit ends an access unit.  The following rule can be used:
    The current NAL unit ends an access unit if it is a VCL NAL unit (i.e., if its "nal_unit_type" is in the range [1..5]), and if either:
	- the next NAL unit is not a VCL NAL unit, or
	- either the current NAL unit's "nal_unit_type" or the next NAL unit's "nal_unit_type" - but not both - is 5 (i.e., "Coded slice of an IDR picture"), or
	- the next NAL unit's "nal_ref_idc" field differs from the current NAL unit's "nal_ref_idc" field, with one of them being equal to zero, or
	- both the current and next NAL units begin with slice headers, and their "frame_num" fields differ, or
	- both the current and next NAL units begin with slice headers, and their "pic_parameter_set_id" fields differ, or
	- both the current and next NAL units begin with slice headers, and their "field_pic_flag" fields differ, or
	- both the current and next NAL units begin with slice headers, and their "bottom_field_flag" fields differ, or
	- both the current and next NAL units' "nal_ref_idc" field is 5, and the "idr_pic_id" fields differ.
----------

(I derived this rule from Section 7.4.1.2.4 - "Detection of the first VCL NAL unit of a primary coded picture" - of the H.264 specification.  Perhaps it could be simplified?)

I suggest that an analogous paragraph be added to the H.265 RTP payload format specification.  (Or alternatively, instead of writing a detailed rule like this, perhaps a reference to an appropriate part of the H.265 specification might be made instead (can we do this in a IETF RFC?).)

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/