Re: [AVT] Clarification on Offer Answer usage of H.264 payload format (RFC 3984)

[Randell Jesup <[e-mail]>]

This is a response to a VERY old message (and there's been one long
discussion of this since then (last summer)).  

Magnus Westerlund <[e-mail]> writes:
>We authors was made aware of an issue in the Offer/Answer usage for the
>H.264 RTP payload format by Tang Yongliang.
>
>The issue is that section 8.2.2 says:
>
>    o  The parameters identifying a media format configuration for H.264
>       are "profile-level-id", "packetization-mode", and, if required by
>       "packetization-mode", "sprop-deint-buf-req".  These three
>       parameters MUST be used symmetrically; i.e., the answerer MUST
>       either maintain all configuration parameters or remove the media
>       format (payload type) completely, if one or more of the parameter
>       values are not supported.
>
>And later
>
>"  o  Parameters declaring a configuration point are not downgradable,
>       with the exception of the level part of the "profile-level-id"
>       parameter."
>
>That is contradicting for the "profile-level-id" parameter. The intention
>of the authors was that the profile part (profile_idc and profile_iop)
>would be fixed but the level (level_idc) can be downgraded. That way you as
>an receiver state the profiles and highest level for that profile you are
>willing to accept. The answerer can then respond with the same profile but
>with a possibly lower level.
>
>So for clarification: An answerer MUST only maintain the profile_idc and
>profile_iop bytes of the profile-level-id value and MAY downgrade the level
>part.
>
>Please note: This may require the offering party to make a new offer to
>provide its "sprop" parameters correctly due to the reduction in level.
>
>However without this clarification the only way of getting a successful
>offer/answer for H.264 when not fully aware of the counter-parts
>capabilities would be to write one payload type for each profile-level
>combination that one could downgrade to.
>
>Any comments on this clarification?

Yes:

There is another side-effect on the text of RFC 3984 of this clarification.

In particular, this text (in description of Offer/Answer):

  o  The "sprop-parameter-sets" parameter is used as described above.
      In addition, an answerer MUST maintain all parameter sets received
      in the offer in its answer.  Depending on the value of the
      "parameter-add" parameter, different rules apply: If "parameter-
      add" is false (0), the answer MUST NOT add any additional
      parameter sets.  If "parameter-add" is true (1), the answerer, in
      its answer, MAY add additional parameter sets to the "sprop-
      parameter-sets" parameter.  The answerer MUST also, independent of
      the value of "parameter-add", accept to receive a video stream
      using the sprop-parameter-sets it declared in the answer.

         Informative note: care must be taken when parameter sets are
         added not to cause overwriting of already transmitted parameter
         sets by using conflicting parameter set identifiers.

is a real problem.

If you offer level 3.0, and include a level 3.0 sprop-paramater-set (which
you should), then per the above, I MUST maintain the 3.0 parameter-set in
my response.  If I want to respond with level 1.0, Magnus says above
that I can, but I'm going to have to include a level 3.0 parameter set.
I can also (if allowed by parameter-add) add a level 1.0 set.  When you add
this:

 o  The parameters "sprop-parameter-sets", "sprop-deint-buf-req",
      "sprop-interleaving-depth", "sprop-max-don-diff", and "sprop-
      init-buf-time" describe the properties of the NAL unit stream that
      the offerer or answerer is sending for this media format
      configuration.  This differs from the normal usage of the
      Offer/Answer parameters: normally such parameters declare the
      properties of the stream that the offerer or the answerer is able
      to receive.  When dealing with H.264, the offerer assumes that the
      answerer will be able to receive media encoded using the
      configuration being offered.

That says the offerer should assume the answerer can receive it, and the
previous quote says that the answerer MUST be willing to receive any
parameter-set in the answer, and it also says the answerer MUST maintain
the original parameter sets in the answer.  Since in most cases, the reason
for downgrading the level in a response is the inability to handle the
level of the offer, the answerer won't be able to handle video encoded
using the parameter sets from the offer.  

Thus, I see no way an answerer can validly answer with a downgrade unless
it internally decides to lie, and drop all video packets until the offerer
renegotiates to remove the unusable parameter sets.

Also, to verify: does this mean that offer-matching is done using the first
4 digits of profile-level-id (and packetization-mode and
sprop-deint-buf-req as needed)?  Should "is the same" (when applied to
profile-level-id) always mean "first 4 digits the same, and last two digits
be the same or lower"?


This really, really, really needs a rewrite.  It's also so complex, it
needs more examples, and if possible an informational(?) algorithm on
how to respond. Right now, I'm not certain *anyone* has come even close to
100% compliance.  (Maybe one or two have, but...)

Existing examples of H.264 traces I've seen show huge variations:

*) Many respond without a matching packetization-mode (i.e. respond to an
   offer of packetization-mode=1 with no packetization-mode)
*) Some respond with a level *higher* than the offer.
*) None I've seen include the sprop-parameter-sets from the offerer.
*) I haven't seen any that respect parameter-add, though given the previous
   point it may not matter.
*) Some don't even include profile-level-id at all.
*) Some accept packetization-mode=1 in the SDP, but send packetization-mode=0.
and there are more problems, that's just the start.  :-(

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[e-mail]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

_______________________________________________
Audio/Video Transport Working Group
[e-mail]
https://www.ietf.org/mailman/listinfo/avt