Re: [AVTCORE] Some changes to rfc3984bis, SVC, and RCDO payload drafts

[Ye-Kui Wang <[e-mail]>]

I think the statement "The proposed changes to max-dpb, max-cpb and max-br sacrifice some backward compatibility with RFC 3984 and forward compatibility with H.264." is incorrect.

On backward compatibility with RFC 3984: The semantics in RFC 3984 does not work with color formats other than YUV 4:2:0 and bit depth other than 8 bits, and the new profiles defined in H.264 after RFC 3984's publication. The proposed changes fix this bug in a way that is backward compatible to RFC 3984, i.e., the fix now covers all color formats and bit depths, while for the video formats and profiles supported in RFC 3984, the bits on the wire remain unchanged. Therefore, I don't think the changes sacrifice backward compatibility with RFC 3984.

On the so-called forward compatibility with H.264: "NOT changing the units of max-cpb and max-br for the high profiles (to include the scaling factors 1.25/3/4)" means that the numbers in the referred H.264 spec must be scaled before they are put into the payload format signaling as specified in the new RFC. Note that whether the scaling factor is considered or not, there is no problem with RFC 3984 backward compatibility. Are you saying you prefer to have the scaled units for the semantics of max-cpb and max-br? If yes, could you please explain why that is better?

BR, YK

From: Mo Zanaty (mzanaty) [mailto:[e-mail]]
Sent: Sunday, March 27, 2011 12:23 AM
To: Ye-Kui Wang; [e-mail]; [e-mail]
Cc: Roni Even; Stephen Botzko; Gary Sullivan
Subject: RE: [AVTCORE] Some changes to rfc3984bis, SVC,and RCDO payload drafts

The proposed changes to max-dpb, max-cpb and max-br sacrifice some backward compatibility with RFC 3984 and forward compatibility with H.264. The latter seems intentional but the former seems unintentional. Further changes are needed to fix or clearly convey these incompatibilities.

What was discussed and decided regarding max-dpb, max-cpb and max-br in the ITU/IETF F2F meetings? Based on the original suggested changes (at the end without color coding), forward compatibility with H.264 appears to be the focus. But after "further discussions", the newest suggested changes (with color coding) appear focused on backward compatibility with RFC 3984, but don't fully achieve this.

The problem with RFC 3984 backward compatibility is that changing the units of max-dpb (from 1024 bytes to 8/3 macroblocks) only works for the 4:2:0 chroma format, which is the primary format used in most profiles and applications. However, the high profiles support other chroma formats, and this change breaks backward compatibility for them.

For example, consider an RFC 3984 implementation that declares max-dpb=8100 (KB) to support 3 reference frames of 720p at 4:4:4. The proposed change would misinterpret this declaration to mean 6 reference frames of 720p at any chroma format including 4:4:4, hence potentially overflowing the old RFC 3984 receiver's DPB by a factor of 2.

(Note that H.264 got away with changing MaxDPB since the original 2003 version only supported 4:2:0. When the high profiles which support other chroma formats were added in 2005, MaxDPB was explicitly redefined in terms of 4:2:0 so it remains constant across all chroma formats. For further clarity, MaxDpbMbs later replaced MaxDPB. But RFC 3984 never kept up with these changes, so implementations are stuck with the 2003 definitions even for later features such as the high profiles and other chroma formats.)

The problem with H.264 forward compatibility is that NOT changing the units of max-cpb and max-br for the high profiles (to include the scaling factors 1.25/3/4) obviously creates confusion and incompatibility between H.264 and this bis draft. This seems like an intentional change after "further discussions" decided to focus on RFC 3984 backward compatibility at the expense of H.264 forward compatibility.

Before deciding on specific new text, the WG should decide on whether the focus of this draft should be backward compatibility with RFC 3984 or forward compatibility with H.264. Or if the intent is to strike some balance between the two, then clearly convey the intentional incompatibilities in the text.

Regards,
Mo Zanaty

From: [e-mail] [mailto:[e-mail]] On Behalf Of Ye-Kui Wang
Sent: Monday, March 21, 2011 4:11 AM
To: [e-mail]; [e-mail]
Subject: Re: [AVTCORE] Some changes to rfc3984bis, SVC,and RCDO payload drafts

Further discussions with the same group of persons led to a decision to stay with the unscaled units for max-br and max-cpb, thus fewer changes are needed to the three payload formats listed in the title and H.241. With this, the changes needed are listed below (the originally suggested changes are dropped from this email). This time I have highlighted the changes, and I have also described the nature the changes below. Hope these may help understand better what have been changed, and can lead to a quicker decision by the group, including WG chairs, and our AD.

BR, YK


------------------------------------Start of suggested changes --------------------------------------


Section 8.1:
OLD:
      max-dpb: The value of max-dpb is an integer indicating the maximum
         decoded picture buffer size in units of 1024 bytes.  The max-
         dpb parameter signals that the receiver has more memory than
         the minimum amount of decoded picture buffer memory required by
         the signaled highest level conveyed in the value of the
         profile-level-id parameter or the max-recv-level parameter.
         When max-dpb is signaled, the receiver MUST be able to decode
         NAL unit streams that conform to the signaled highest level,
         with the exception that the MaxDPB value in Table A-1 of [1]
         for the signaled highest level is replaced with the value of
         max-dpb.  Consequently, a receiver that signals max-dpb MUST be
         capable of storing the following number of decoded frames,
         complementary field pairs, and non-paired fields in its decoded
         picture buffer:

            Min(1024 * max-dpb / ( PicWidthInMbs * FrameHeightInMbs *
            256 * ChromaFormatFactor ), 16)

         PicWidthInMbs, FrameHeightInMbs, and ChromaFormatFactor are
         defined in [1].

         The value of max-dpb MUST be greater than or equal to the value
         of MaxDPB given in Table A-1 of [1] for the highest level.
         Senders MAY use this knowledge to construct coded video streams
         with improved compression.
NEW: (When this change can be considered as editorial can be discussed, but the nature of this change as follows. On the other hand, if not changed, then the semantics of max-dpb is simply equivalent to unspecified, as MaxDPB and ChromaFormatFactor are not found in the latest H.264 spec any more. Note that compared to RFC 3984, the bits on the wire do not change.)
      max-dpb: The value of max-dpb is an integer indicating the maximum
         decoded picture buffer size in units of 8/3 macroblocks.  The max-
         dpb parameter signals that the receiver has more memory than
         the minimum amount of decoded picture buffer memory required by
         the signaled highest level conveyed in the value of the
         profile-level-id parameter or the max-recv-level parameter.
         When max-dpb is signaled, the receiver MUST be able to decode
         NAL unit streams that conform to the signaled highest level,
         with the exception that the MaxDpbMbs value in Table A-1 of [1]
         for the signaled highest level is replaced with the value of
         max-dpb * 3 / 8.  Consequently, a receiver that signals max-dpb MUST be
         capable of storing the following number of decoded frames,
         complementary field pairs, and non-paired fields in its decoded
         picture buffer:

            Min(max-dpb * 3 / 8 / ( PicWidthInMbs * FrameHeightInMbs), 16)

         Wherein PicWidthInMbs and FrameHeightInMbs are defined in [1].

         The value of max-dpb MUST be greater than or equal to the value
         of MaxDpbMbs * 3 / 8, wherein the value of MaxDpbMbs is given in
Table A-1 of [1] for the highest level.
         Senders MAY use this knowledge to construct coded video streams
         with improved compression.



------------------------------------End of suggested changes --------------------------------------

From: [e-mail] [mailto:[e-mail]] On Behalf Of Ye-Kui Wang
Sent: Sunday, March 20, 2011 4:21 AM
To: [e-mail]; [e-mail]
Subject: [AVTCORE] Some changes to rfc3984bis, SVC, and RCDO payload drafts

Folks,

The three H.264/AVC related payload formats, namely, draft-ietf-avt-rtp-rfc3984bis-12, draft-ietf-avt-rtp-svc-27, and draft-ietf-avt-rtp-h264-rcdo-08, are all at the AUTH48 stage.

The RFC-Editor has found the following problem: In draft-ietf-avt-rtp-rfc3984bis-12, the definition of the max-dpb media parameter refers to the MaxDPB that was defined the first version of the H.264/AVC spec, but not any more in the latest version (the 2010 version). The parameter in the latest H.264/AVC version corresponding to MaxDPB is MaxDpbMbs, and the unit of the new parameter (i.e., macroblocks) is different from the original one (i.e. 1024 bytes).

The problem applies also to the SVC payload format, the RCDO payload format, and H.241.

A solution has been found and agreed, involving rfc3984bis authors and some key people related to H.264/AVC (e.g., Gary Sullivan and Heiko Schwarz) and H.241 (e.g., Stephen Botzko and Patrick Luthi). Furthermore, we have found that there are also a couple of places that need fixes due to similar changes from the initial version of H.264/AVC to the latest version.

Per Roni's suggestion, I am sending in below the changes to draft-ietf-avt-rtp-rfc3984bis-12 for review by the Payload and AVTcore WGs. It seems that exactly the same changes are needed to draft-ietf-avt-rtp-h264-rcdo-08 (co-authors of this draft may confirm), and similar but slightly different changes are needed to draft-ietf-avt-rtp-svc-27.

Since the drafts are at the AUTH48 stage, please provide comments by Monday, March 21, if any. Many thanks!

BR, YK



------------------------------------Start of suggested changes --------------------------------------

Section 8.1:
OLD:
      max-cpb: The value of max-cpb is an integer indicating the maximum
         coded picture buffer size in units of 1000 bits for the VCL HRD
         parameters (see A.3.1, item i of [1]) and in units of 1200 bits
         for the NAL HRD parameters (see A.3.1, item j of [1]).
NEW:
      max-cpb: The value of max-cpb is an integer indicating the maximum
         coded picture buffer size. For Constrained Baseline, Baseline, Main
and Extended profiles, the unit is 1000 bits for the VCL HRD
         parameters and 1200 bits for the NAL HRD parameters. For High, High 10,
High 4:2:2, High 4:4:4 Predictive, High 10 Intra, High 4:2:2 Intra,
High 4:4:4 Intra and CAVLC 4:4:4 Intra profiles, the unit is cpbBrVclFactor
bits for the VCL HRD and cpbBrNalFactor bits for the NAL HRD parameters.

OLD:

      max-br: The value of max-br is an integer indicating the maximum

         video bitrate in units of 1000 bits per second for the VCL HRD

         parameters (see A.3.1, item i of [1]) and in units of 1200 bits

         per second for the NAL HRD parameters (see A.3.1, item j of

         [1]).
NEW:

      max-br: The value of max-br is an integer indicating the maximum
         video bitrate.  For Constrained Baseline, Baseline, Main
and Extended profiles, the unit is 1000 bits per second for the VCL HRD
         parameters and 1200 bits per second for the NAL HRD parameters. For High, High 10,
High 4:2:2, High 4:4:4 Predictive, High 10 Intra, High 4:2:2 Intra,
High 4:4:4 Intra and CAVLC 4:4:4 Intra profiles, the unit is cpbBrVclFactor
bits per second for the VCL HRD and cpbBrNalFactor bits per second for the NAL HRD parameters.


------------------------------------End of suggested changes --------------------------------------