Re: [payload] [AVTCORE] Some changes to rfc3984bis, SVC, and RCDO payload drafts

["Mo Zanaty (mzanaty)" <[e-mail]>]

The proposed changes to max-dpb, max-cpb and max-br sacrifice some
backward compatibility with RFC 3984 and forward compatibility with
H.264. The latter seems intentional but the former seems unintentional.
Further changes are needed to fix or clearly convey these
incompatibilities.

 

What was discussed and decided regarding max-dpb, max-cpb and max-br in
the ITU/IETF F2F meetings? Based on the original suggested changes (at
the end without color coding), forward compatibility with H.264 appears
to be the focus. But after "further discussions", the newest suggested
changes (with color coding) appear focused on backward compatibility
with RFC 3984, but don't fully achieve this.

 

The problem with RFC 3984 backward compatibility is that changing the
units of max-dpb (from 1024 bytes to 8/3 macroblocks) only works for the
4:2:0 chroma format, which is the primary format used in most profiles
and applications. However, the high profiles support other chroma
formats, and this change breaks backward compatibility for them.

 

For example, consider an RFC 3984 implementation that declares
max-dpb=8100 (KB) to support 3 reference frames of 720p at 4:4:4. The
proposed change would misinterpret this declaration to mean 6 reference
frames of 720p at any chroma format including 4:4:4, hence potentially
overflowing the old RFC 3984 receiver's DPB by a factor of 2.

 

(Note that H.264 got away with changing MaxDPB since the original 2003
version only supported 4:2:0. When the high profiles which support other
chroma formats were added in 2005, MaxDPB was explicitly redefined in
terms of 4:2:0 so it remains constant across all chroma formats. For
further clarity, MaxDpbMbs later replaced MaxDPB. But RFC 3984 never
kept up with these changes, so implementations are stuck with the 2003
definitions even for later features such as the high profiles and other
chroma formats.)

 

The problem with H.264 forward compatibility is that NOT changing the
units of max-cpb and max-br for the high profiles (to include the
scaling factors 1.25/3/4) obviously creates confusion and
incompatibility between H.264 and this bis draft. This seems like an
intentional change after "further discussions" decided to focus on RFC
3984 backward compatibility at the expense of H.264 forward
compatibility.

 

Before deciding on specific new text, the WG should decide on whether
the focus of this draft should be backward compatibility with RFC 3984
or forward compatibility with H.264. Or if the intent is to strike some
balance between the two, then clearly convey the intentional
incompatibilities in the text.

 

Regards,

Mo Zanaty

 

From: [e-mail] [mailto:[e-mail]] On Behalf Of
Ye-Kui Wang
Sent: Monday, March 21, 2011 4:11 AM
To: [e-mail]; [e-mail]
Subject: Re: [AVTCORE] Some changes to rfc3984bis, SVC,and RCDO payload
drafts

 

Further discussions with the same group of persons led to a decision to
stay with the unscaled units for max-br and max-cpb, thus fewer changes
are needed to the three payload formats listed in the title and H.241.
With this, the changes needed are listed below (the originally suggested
changes are dropped from this email). This time I have highlighted the
changes, and I have also described the nature the changes below. Hope
these may help understand better what have been changed, and can lead to
a quicker decision by the group, including WG chairs, and our AD.

 

BR, YK

 

------------------------------------Start of suggested changes
--------------------------------------

 

Section 8.1:

OLD:

      profile-level-id:
         A base16 [7] (hexadecimal) representation of the following
         three bytes in the sequence parameter set NAL unit is specified
         in [1]: 1) profile_idc, 2) a byte herein referred to as
         profile-iop, composed of the values of constraint_set0_flag,
         constraint_set1_flag,constraint_set2_flag,
         constraint_set3_flag, and reserved_zero_4bits in bit-
         significance order, starting from the most-significant bit, and
         3) level_idc.  Note that reserved_zero_4bits is required to be
         equal to 0 in [1], but other values for it may be specified in
         the future by ITU-T or ISO/IEC.

 NEW: (note that the change here is purely editorial)

      profile-level-id:

         A base16 [7] (hexadecimal) representation of the following

         three bytes in the sequence parameter set NAL unit is specified

         in [1]: 1) profile_idc, 2) a byte herein referred to as

         profile-iop, composed of the values of constraint_set0_flag,

         constraint_set1_flag,constraint_set2_flag,

         constraint_set3_flag, constraint_set4_flag,
constraint_set5_flag,

 and reserved_zero_2bits in bit-significance order, starting from the
most-significant bit, and

         3) level_idc.  Note that reserved_zero_2bits is required to be

         equal to 0 in [1], but other values for it may be specified in

         the future by ITU-T or ISO/IEC.

 

OLD:

         For example, in the table above, profile_idc equal to 58
         (Extended) with profile-iop equal to 11xx0000 indicates the
         same sub-profile corresponding to profile_idc equal to 42
         (Baseline) with profile-iop equal to x1xx0000.  Note that other
         combinations of profile_idc and profile-iop (not listed in
         Table 5) may represent a sub-profile equivalent to the common
         subset of coding tools for more than one profile.  Note also
         that a decoder conforming to a certain profile may be able to
         decode bitstreams conforming to other profiles.  For example, a
         decoder conforming to the High 4:4:4 profile, at a certain
         level, must be able to decode bitstreams conforming to the
         Constrained Baseline, Main, High, High 10, or High 4:2:2
         profile at the same or a lower level.

 NEW: (note that the change here is purely editorial)

         For example, in the table above, profile_idc equal to 58

         (Extended) with profile-iop equal to 11xx0000 indicates the

         same sub-profile corresponding to profile_idc equal to 42

         (Baseline) with profile-iop equal to x1xx0000.  Note that other

         combinations of profile_idc and profile-iop (not listed in

         Table 5) may represent a sub-profile equivalent to the common

         subset of coding tools for more than one profile.  Note also

         that a decoder conforming to a certain profile may be able to

         decode bitstreams conforming to other profiles.

 

OLD:

         If the profile-level-id parameter is used for capability
         exchange or session setup procedure, it indicates the subset of
         coding tools, which is equal to the default sub-profile, that
         the codec supports for both receiving and sending.

 NEW: (note that the change here is purely editorial)

         If the profile-level-id parameter is used for capability

         exchange or session setup, it indicates the subset of

         coding tools, which is equal to the default sub-profile, that

         the codec supports for both receiving and sending.

 

OLD: 

      max-cpb: The value of max-cpb is an integer indicating the maximum

         coded picture buffer size in units of 1000 bits for the VCL HRD

         parameters (see A.3.1, item i of [1]) and in units of 1200 bits

         for the NAL HRD parameters (see A.3.1, item j of [1]).

NEW: (note that the change here is purely editorial)

      max-cpb: The value of max-cpb is an integer indicating the maximum

         coded picture buffer size in units of 1000 bits for the VCL HRD

         parameters and in units of 1200 bits

         for the NAL HRD parameters.

 

OLD:

      max-dpb: The value of max-dpb is an integer indicating the maximum

         decoded picture buffer size in units of 1024 bytes.  The max-

         dpb parameter signals that the receiver has more memory than

         the minimum amount of decoded picture buffer memory required by

         the signaled highest level conveyed in the value of the

         profile-level-id parameter or the max-recv-level parameter.

         When max-dpb is signaled, the receiver MUST be able to decode

         NAL unit streams that conform to the signaled highest level,

         with the exception that the MaxDPB value in Table A-1 of [1]

         for the signaled highest level is replaced with the value of

         max-dpb.  Consequently, a receiver that signals max-dpb MUST be

         capable of storing the following number of decoded frames,

         complementary field pairs, and non-paired fields in its decoded

         picture buffer:

 

            Min(1024 * max-dpb / ( PicWidthInMbs * FrameHeightInMbs *

            256 * ChromaFormatFactor ), 16)

 

         PicWidthInMbs, FrameHeightInMbs, and ChromaFormatFactor are

         defined in [1].

 

         The value of max-dpb MUST be greater than or equal to the value

         of MaxDPB given in Table A-1 of [1] for the highest level.

         Senders MAY use this knowledge to construct coded video streams

         with improved compression.

NEW: (When this change can be considered as editorial can be discussed,
but the nature of this change as follows. On the other hand, if not
changed, then the semantics of max-dpb is simply equivalent to
unspecified, as MaxDPB and ChromaFormatFactor are not found in the
latest H.264 spec any more. Note that compared to RFC 3984, the bits on
the wire do not change.)

      max-dpb: The value of max-dpb is an integer indicating the maximum

         decoded picture buffer size in units of 8/3 macroblocks.  The
max-

         dpb parameter signals that the receiver has more memory than

         the minimum amount of decoded picture buffer memory required by

         the signaled highest level conveyed in the value of the

         profile-level-id parameter or the max-recv-level parameter.

         When max-dpb is signaled, the receiver MUST be able to decode

         NAL unit streams that conform to the signaled highest level,

         with the exception that the MaxDpbMbs value in Table A-1 of [1]

         for the signaled highest level is replaced with the value of

         max-dpb * 3 / 8.  Consequently, a receiver that signals max-dpb
MUST be

         capable of storing the following number of decoded frames,

         complementary field pairs, and non-paired fields in its decoded

         picture buffer:

 

            Min(max-dpb * 3 / 8 / ( PicWidthInMbs * FrameHeightInMbs),
16)

 

         Wherein PicWidthInMbs and FrameHeightInMbs are defined in [1].

 

         The value of max-dpb MUST be greater than or equal to the value

         of MaxDpbMbs * 3 / 8, wherein the value of MaxDpbMbs is given
in 

Table A-1 of [1] for the highest level.

         Senders MAY use this knowledge to construct coded video streams

         with improved compression.

 

OLD:

      max-br: The value of max-br is an integer indicating the maximum
         video bitrate in units of 1000 bits per second for the VCL HRD
         parameters (see A.3.1, item i of [1]) and in units of 1200 bits
         per second for the NAL HRD parameters (see A.3.1, item j of
         [1]).
         ...

         For example, if a receiver signals capability for Level 1.2

         with max-br equal to 1550, this indicates a maximum video

         bitrate of 1550 kbits/sec for VCL HRD parameters, a maximum

         video bitrate of 1860 kbits/sec for NAL HRD parameters, and a

         CPB size of 4036458 bits (1550000 / 384000 * 1000 * 1000).

NEW: (note that the change here is purely editorial) 

      max-br: The value of max-br is an integer indicating the maximum
         video bitrate in units of 1000 bits per second for the VCL HRD
         parameters and in units of 1200 bits
         per second for the NAL HRD parameters.
         ...

         For example, if a receiver signals capability for Main profile
Level 1.2

         with max-br equal to 1550, this indicates a maximum video

         bitrate of 1550 kbits/sec for VCL HRD parameters, a maximum

         video bitrate of 1860 kbits/sec for NAL HRD parameters, and a

         CPB size of 4036458 bits (1550000 / 384000 * 1000 * 1000).

 

------------------------------------End of suggested changes
--------------------------------------

 

From: [e-mail] [mailto:[e-mail]] On Behalf Of
Ye-Kui Wang
Sent: Sunday, March 20, 2011 4:21 AM
To: [e-mail]; [e-mail]
Subject: [AVTCORE] Some changes to rfc3984bis, SVC, and RCDO payload
drafts

 

Folks,

 

The three H.264/AVC related payload formats, namely,
draft-ietf-avt-rtp-rfc3984bis-12, draft-ietf-avt-rtp-svc-27, and
draft-ietf-avt-rtp-h264-rcdo-08, are all at the AUTH48 stage.

 

The RFC-Editor has found the following problem: In
draft-ietf-avt-rtp-rfc3984bis-12, the definition of the max-dpb media
parameter refers to the MaxDPB that was defined the first version of the
H.264/AVC spec, but not any more in the latest version (the 2010
version). The parameter in the latest H.264/AVC version corresponding to
MaxDPB is MaxDpbMbs, and the unit of the new parameter (i.e.,
macroblocks) is different from the original one (i.e. 1024 bytes).

 

The problem applies also to the SVC payload format, the RCDO payload
format, and H.241.

 

A solution has been found and agreed, involving rfc3984bis authors and
some key people related to H.264/AVC (e.g., Gary Sullivan and Heiko
Schwarz) and H.241 (e.g., Stephen Botzko and Patrick Luthi).
Furthermore, we have found that there are also a couple of places that
need fixes due to similar changes from the initial version of H.264/AVC
to the latest version.

 

Per Roni's suggestion, I am sending in below the changes to
draft-ietf-avt-rtp-rfc3984bis-12 for review by the Payload and AVTcore
WGs. It seems that exactly the same changes are needed to
draft-ietf-avt-rtp-h264-rcdo-08 (co-authors of this draft may confirm),
and similar but slightly different changes are needed to
draft-ietf-avt-rtp-svc-27.

 

Since the drafts are at the AUTH48 stage, please provide comments by
Monday, March 21, if any. Many thanks!

 

BR, YK

 

 

------------------------------------Start of suggested changes
--------------------------------------

 

Section 8.1:

OLD:

      profile-level-id:
         A base16 [7] (hexadecimal) representation of the following
         three bytes in the sequence parameter set NAL unit is specified
         in [1]: 1) profile_idc, 2) a byte herein referred to as
         profile-iop, composed of the values of constraint_set0_flag,
         constraint_set1_flag,constraint_set2_flag,
         constraint_set3_flag, and reserved_zero_4bits in bit-
         significance order, starting from the most-significant bit, and
         3) level_idc.  Note that reserved_zero_4bits is required to be
         equal to 0 in [1], but other values for it may be specified in
         the future by ITU-T or ISO/IEC.

 NEW:

      profile-level-id:

         A base16 [7] (hexadecimal) representation of the following

         three bytes in the sequence parameter set NAL unit is specified

         in [1]: 1) profile_idc, 2) a byte herein referred to as

         profile-iop, composed of the values of constraint_set0_flag,

         constraint_set1_flag,constraint_set2_flag,

         constraint_set3_flag, constraint_set4_flag,
constraint_set5_flag,

 and reserved_zero_2bits in bit-significance order, starting from the
most-significant bit, and

         3) level_idc.  Note that reserved_zero_2bits is required to be

         equal to 0 in [1], but other values for it may be specified in

         the future by ITU-T or ISO/IEC.

 

OLD:

         For example, in the table above, profile_idc equal to 58
         (Extended) with profile-iop equal to 11xx0000 indicates the
         same sub-profile corresponding to profile_idc equal to 42
         (Baseline) with profile-iop equal to x1xx0000.  Note that other
         combinations of profile_idc and profile-iop (not listed in
         Table 5) may represent a sub-profile equivalent to the common
         subset of coding tools for more than one profile.  Note also
         that a decoder conforming to a certain profile may be able to
         decode bitstreams conforming to other profiles.  For example, a
         decoder conforming to the High 4:4:4 profile, at a certain
         level, must be able to decode bitstreams conforming to the
         Constrained Baseline, Main, High, High 10, or High 4:2:2
         profile at the same or a lower level.

 NEW:

         For example, in the table above, profile_idc equal to 58

         (Extended) with profile-iop equal to 11xx0000 indicates the

         same sub-profile corresponding to profile_idc equal to 42

         (Baseline) with profile-iop equal to x1xx0000.  Note that other

         combinations of profile_idc and profile-iop (not listed in

         Table 5) may represent a sub-profile equivalent to the common

         subset of coding tools for more than one profile.  Note also

         that a decoder conforming to a certain profile may be able to

         decode bitstreams conforming to other profiles.

 

OLD:

         If the profile-level-id parameter is used for capability
         exchange or session setup procedure, it indicates the subset of
         coding tools, which is equal to the default sub-profile, that
         the codec supports for both receiving and sending.

 NEW:

         If the profile-level-id parameter is used for capability

         exchange or session setup, it indicates the subset of

         coding tools, which is equal to the default sub-profile, that

         the codec supports for both receiving and sending.

 

OLD:

      max-cpb: The value of max-cpb is an integer indicating the maximum

         coded picture buffer size in units of 1000 bits for the VCL HRD

         parameters (see A.3.1, item i of [1]) and in units of 1200 bits

         for the NAL HRD parameters (see A.3.1, item j of [1]).

NEW:

      max-cpb: The value of max-cpb is an integer indicating the maximum

         coded picture buffer size. For Constrained Baseline, Baseline,
Main 

and Extended profiles, the unit is 1000 bits for the VCL HRD

         parameters and 1200 bits for the NAL HRD parameters. For High,
High 10,

High 4:2:2, High 4:4:4 Predictive, High 10 Intra, High 4:2:2 Intra, 

High 4:4:4 Intra and CAVLC 4:4:4 Intra profiles, the unit is
cpbBrVclFactor

bits for the VCL HRD and cpbBrNalFactor bits for the NAL HRD parameters.

 

OLD:

      max-dpb: The value of max-dpb is an integer indicating the maximum

         decoded picture buffer size in units of 1024 bytes.  The max-

         dpb parameter signals that the receiver has more memory than

         the minimum amount of decoded picture buffer memory required by

         the signaled highest level conveyed in the value of the

         profile-level-id parameter or the max-recv-level parameter.

         When max-dpb is signaled, the receiver MUST be able to decode

         NAL unit streams that conform to the signaled highest level,

         with the exception that the MaxDPB value in Table A-1 of [1]

         for the signaled highest level is replaced with the value of

         max-dpb.  Consequently, a receiver that signals max-dpb MUST be

         capable of storing the following number of decoded frames,

         complementary field pairs, and non-paired fields in its decoded

         picture buffer:

 

            Min(1024 * max-dpb / ( PicWidthInMbs * FrameHeightInMbs *

            256 * ChromaFormatFactor ), 16)

 

         PicWidthInMbs, FrameHeightInMbs, and ChromaFormatFactor are

         defined in [1].

 

         The value of max-dpb MUST be greater than or equal to the value

         of MaxDPB given in Table A-1 of [1] for the highest level.

         Senders MAY use this knowledge to construct coded video streams

         with improved compression.

NEW:

      max-dpb: The value of max-dpb is an integer indicating the maximum

         decoded picture buffer size in units of 8/3 macroblocks.  The
max-

         dpb parameter signals that the receiver has more memory than

         the minimum amount of decoded picture buffer memory required by

         the signaled highest level conveyed in the value of the

         profile-level-id parameter or the max-recv-level parameter.

         When max-dpb is signaled, the receiver MUST be able to decode

         NAL unit streams that conform to the signaled highest level,

         with the exception that the MaxDpbMbs value in Table A-1 of [1]

         for the signaled highest level is replaced with the value of

         max-dpb * 3 / 8.  Consequently, a receiver that signals max-dpb
MUST be

         capable of storing the following number of decoded frames,

         complementary field pairs, and non-paired fields in its decoded

         picture buffer:

 

            Min(max-dpb * 3 / 8 / ( PicWidthInMbs * FrameHeightInMbs),
16)

 

         Wherein PicWidthInMbs and FrameHeightInMbs are defined in [1].

 

         The value of max-dpb MUST be greater than or equal to the value

         of MaxDpbMbs * 3 / 8, wherein the value of MaxDpbMbs is given
in 

Table A-1 of [1] for the highest level.

         Senders MAY use this knowledge to construct coded video streams

         with improved compression.

 

OLD:

      max-br: The value of max-br is an integer indicating the maximum
         video bitrate in units of 1000 bits per second for the VCL HRD
         parameters (see A.3.1, item i of [1]) and in units of 1200 bits
         per second for the NAL HRD parameters (see A.3.1, item j of
         [1]).
         ...

         For example, if a receiver signals capability for Level 1.2

         with max-br equal to 1550, this indicates a maximum video

         bitrate of 1550 kbits/sec for VCL HRD parameters, a maximum

         video bitrate of 1860 kbits/sec for NAL HRD parameters, and a

         CPB size of 4036458 bits (1550000 / 384000 * 1000 * 1000).

NEW: 

      max-br: The value of max-br is an integer indicating the maximum

         video bitrate.  For Constrained Baseline, Baseline, Main 

and Extended profiles, the unit is 1000 bits per second for the VCL HRD

         parameters and 1200 bits per second for the NAL HRD parameters.
For High, High 10,

High 4:2:2, High 4:4:4 Predictive, High 10 Intra, High 4:2:2 Intra, 

High 4:4:4 Intra and CAVLC 4:4:4 Intra profiles, the unit is
cpbBrVclFactor

bits per second for the VCL HRD and cpbBrNalFactor bits per second for
the NAL HRD parameters.

         ...

         For example, if a receiver signals capability for Main profile
Level 1.2

         with max-br equal to 1550, this indicates a maximum video

         bitrate of 1550 kbits/sec for VCL HRD parameters, a maximum

         video bitrate of 1860 kbits/sec for NAL HRD parameters, and a

         CPB size of 4036458 bits (1550000 / 384000 * 1000 * 1000).

 

------------------------------------End of suggested changes
--------------------------------------