Re: [MMUSIC] [AVTCORE] RSDP directorate review of draft-ietf-avtcore-rtp-vvc-14 - SDP examples

[Stephan Wenger <[e-mail]>]

All:

I don’t think I have seen a reply on this “disposition of comments” (to use ISO jargon).  To summarize my suggestions, I would prefer not to add examples except a simple O/A exchange.  And that example can be found below.  Is that agreeable?

If we don’t hear objections within a week, we will move forward under the assumption that it is.

Thanks,
Stephan



From: avt <[e-mail]> on behalf of Stephan Wenger <[e-mail]>
Date: Monday, April 4, 2022 at 08:23
To: Christer Holmberg <[e-mail]>, Bernard Aboba <[e-mail]>
Cc: "[e-mail]" <[e-mail]>, Miska Hannuksela <[e-mail]>, IETF AVTCore WG <[e-mail]>, mmusic <[e-mail]>
Subject: Re: [AVTCORE] [MMUSIC] RSDP directorate review of draft-ietf-avtcore-rtp-vvc-14 - SDP examples

Hi Christer,
Thanks for your comments, and please see inline.
S.

From: Christer Holmberg <[e-mail]>
Date: Monday, April 4, 2022 at 07:05
To: Stephan Wenger <[e-mail]>, Bernard Aboba <[e-mail]>
Cc: IETF AVTCore WG <[e-mail]>, Miska Hannuksela <[e-mail]>, "[e-mail]" <[e-mail]>, mmusic <[e-mail]>
Subject: RE: [MMUSIC] RSDP directorate review of draft-ietf-avtcore-rtp-vvc-14 - SDP examples

Hi,

Note that I am not asking for a long list of example – I think there should be at least *ONE* example of an offer and the associated answer :)

Adding one example is not a major burden for us, but I don’t see much value here for the reasons already stated.  If the decision were to add one example, I would suggest the following (please see also the anecdote below regarding possibly more complex examples):

The perhaps most common outcome of a capability negotiation in video is down-dialing the “level” (pixel over time processing capability, corresponds to maximum resolution/framerate) based on receiver capabilities.  We could include a simple example where the offeror offers level 5.1 symmetric (hence no max-recv-level-id), and the answerer dials down to level 4.1.  That would probably one example that you may find in the wild.  It would look something like this:

Offer: main profile with level 5.1 (represented as 83 per table A.1 of [H.266]).  Informally speaking, level 5.1 would allow, among other resolution/frame rate combinations, 4Kp60.

          m=video 49170 RTP/AVP 98
           a=rtpmap:98 H266/90000
           a=fmtp:98 profile-id=1; level_id=83;

Answer: main profile with level 4.1 (represented as 67 per table A.1 of [H.266]).  Informally speaking, level 4.1 would allow, among other resolution/frame rate combinations, 1080p60.

          m=video 49170 RTP/AVP 98
           a=rtpmap:98 H266/90000
           a=fmtp:98 profile-id=1; level_id=67

As a result, both sides would send H.266 compliant with main profile, and following the level constraints of level 4.1.


The only SDP is the following in Section 7.2.1.:

          m=video 49170 RTP/AVP 98
           a=rtpmap:98 H266/90000
           a=fmtp:98 profile-id=1;
             sprop-vps=<video parameter sets data>;
             sprop-sps=<sequence parameter set data>;
             sprop-pps=<picture parameter set data>;

But, there is no example of any video/sequence/picture parameter sets data.


Yes, there isn’t.  However, these codepoints are base64 representations of H.266 coded bitstream fragments.  We could grab the binaries from a real-world H.266 bitstream, encode them BASE64, and put them into the draft instead of the textual description in angle brackets.  We did something like that when writing RFC3984 back in the day.  RFC3984 (payload format for H.264) contains an example as follows: sprop-parameter-sets=Z0IACpZTBYmI,aMljiA==. We could do the same.  But what would it give us?



The parameter set related syntax tables in H.266 are 10+ pages, the related semantics even longer, and its super-dense MPEG-style text.  We can’t explain how we arrive at the BASE64 string for the parameter sets beyond what is already in the text, without doubling the word count of the draft.  Therefore, if we were replacing the placeholders indicated by the angle brackets with real BASE64 strings, we would simply replace self-explanatory placeholders with some “magic” characters.



Here’s an anecdote from 15 or so years ago, when people were still using SDP in a SIP environment for video conferencing.  Some bloke literally copied that string into his implementation, despite using a different parameter set in-band later.  In RFC3984, it’s fine to have inband parameter sets different than the ones set up during O/A so, arguable, he was in compliance.  Still, with his prototype as offeror and sender, under test, our receiving system occasionally threw exceptions indicating a corrupt bitstream.  Why?  When a packet loss affected the inband parameter set (which is typically the first RTP packet in a session), our decoder, in full compliance with RFC3984 and H.264, used the parameter sets it received out of band in that magic string above.  The parameters in those parameter sets, however, indicated a different resolution than what the encoder used.  It took as a few engineering hours to hunt down his bug.  When confronted, his reaction was “but I only implemented RFC3984”.  Well, he didn’t.  He blindly copy-pasted an example into his implementation.  That’s but one example for why examples could (dare I say: should) be considered harmful :-). This episode was fresh in my mind when we wrote RFC7798, which is why there’s no magic BASE64 string in that spec, but the angle bracket descriptive text.  I’m not in favor of changing that.

Stephan

Regards,

Christer


From: Stephan Wenger <[e-mail]>
Sent: lauantai 2. huhtikuuta 2022 5.44
To: Bernard Aboba <[e-mail]>
Cc: Christer Holmberg <[e-mail]>; IETF AVTCore WG <[e-mail]>; Hannuksela, Miska (Nokia - FI/Tampere) <[e-mail]>; [e-mail]; mmusic <[e-mail]>
Subject: Re: [MMUSIC] RSDP directorate review of draft-ietf-avtcore-rtp-vvc-14

Bernard, folks,
Please see inline below in blue.
S.

From: Bernard Aboba <[e-mail]<mailto:[e-mail]>>
Date: Friday, April 1, 2022 at 19:00
To: Stephan Wenger <[e-mail]<mailto:[e-mail]>>
Cc: Christer Holmberg <christer.holmberg=[e-mail]<mailto:christer.holmberg=[e-mail]>>, IETF AVTCore WG <[e-mail]<mailto:[e-mail]>>, Miska Hannuksela <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, mmusic <[e-mail]<mailto:[e-mail]>>
Subject: Re: [MMUSIC] RSDP directorate review of draft-ietf-avtcore-rtp-vvc-14

Stefan said:

"This has come up before in the AVTCORE WG.  There are upsides and downsides to examples.  One key downside is that no few implementers have taken the presence of examples as an excuse of not reading/understanding the spec, leading to botched implementations.  I’m not an implementer myself—never really was—but that’s a story I have heard often.  Therefore, we authors consciously minimized the number of examples in this spec.  As said, we discussed that in AVTCORE, last during IETF112.
- Bernard Aboba: Is an additional profile document needed to enable interoperability? Without this, we are seeing problems with HEVC interop.
- Mo Zanaty: examples for all complex cases are not needed, but some basic ones might help.
My recollection is that I went out of the virtual room with a clear mandate not to include more and more complex examples.  I don’t have time right now to verify that from the audio record.

[BA] The specific situation that I was referring to in the notes is the situation with WebRTC support for HEVC. It is currently supported in Safari behind a flag, but interop has been tricky, both at the SDP and RTP levels.  RFC 7742 covers H.264 but not HEVC, and the lack of SDP examples in RFC 7798 has added to the vacuum.   One might cover HEVC and VVC in a followon to RFC 7798 and do the profiling and examples there.  But having some basic examples to draw upon would provide a foundation for that.
[StW]: It has been a long time that I looked at webrtc in the wild.  Can someone please tell us how much of RFC7798 the Safari folks are using, and anyone else who’s using that RFC for that matter?  I note that RFC7798 has one trivial example, at the end of section 7.2.1, which basically describes a main profile H.265/RFC7798 stream whose further H.265 characteristics are encoded in binary/base64 form in the VPS, and which is not using optional RFC7798 parameters beyond the VPS.  In the distant past (H.264-times), that was how most of the SDP blobs looked like that I came across.  The subject VVC draft has something similar in section 7.2.1., with more parameter sets conveyed in the SDP.  What is needed beyond that those trivial examples, and what has led to those interop problems?  We authors need help here.  AFAIK, no one of us uses or is interested much in SDP—those of us who do product work at all (not many AFAIK) sit on top of our own call stack.  As for a fix/follow up of RFC7798, I personally am not particularly interested in driving this—it should really be driven by someone who has felt the pain.  I’m willing to help, though.
Thanks,
Stephan

On Thu, Mar 31, 2022 at 3:24 PM Stephan Wenger <[e-mail]<mailto:[e-mail]>> wrote:
Hi Christer,

Thanks very much for this detailed and timely review.  Initial reactions inline.
Miska: please take a look at Q5 below and my answer.
AVTCore: please take a look at Q7 below, and my answer.  Is there interest, and are there volunteers?

Stephan

From: mmusic <[e-mail]<mailto:[e-mail]>> on behalf of Christer Holmberg <christer.holmberg=[e-mail]<mailto:[e-mail]>>
Date: Wednesday, March 30, 2022 at 11:39
To: IETF AVTCore WG <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, mmusic <[e-mail]<mailto:[e-mail]>>
Subject: [MMUSIC] RSDP directorate review of draft-ietf-avtcore-rtp-vvc-14

Hi,

I have performed the SDP directorate review of draft-ietf-avtcore-rtp-vvc-14.

GEN:

In general, it is quite difficult to read and parse the SDP sections. Also, for the offer/answer procedures, we often have dedicated sub-sections for creating the offer, creating the answer, etc.

However, I will focus on things which I think are unclear.

---

Q1:

Section 7.2.1 says:

   *  The OPTIONAL parameters profile-id, tier-flag, sub-profile-id,
      interop-constraints, level-id, sprop-sublayer-id, sprop-ols-id,
      recv-sublayer-id, recv-ols-id, max-recv-level-id, max-lsr, max-
      fps, sprop-max-don-diff, sprop-depack-buf-bytes and depack-buf-
      cap, when present, MUST be included in the "a=fmtp" line of SDP.
      This parameter is expressed as a media type string, in the form of
      a semicolon-separated list of parameter=value pairs.

What does “this parameter” refer to? Do you mean “these parameters”?

---

I believe “this parameter” refers to the a=fmtp line.  Does that make sense?  If yes, we can rephrase by replacing “This parameter” with “The fmtp line”

Q2:

Section 7.2.1 says:


   *  The OPTIONAL parameter sprop-vps, sprop-sps, sprop-pps, sprop-sei,

      and sprop-dci, when present, MUST be included in the "a=fmtp" line

      of SDP or conveyed using the "fmtp" source attribute as specified

      in Section 6.3 of [RFC5576]<https://datatracker.ietf.org/doc/html/rfc5576#section-6.3>.

Why are you using the “fmtp” source attribute?

---


This was inherited from the comparable section of the H.265 payload format, RFC7798 (which most of the SDP stuff was copied from and adapted).  RFC7798 includes this informative note:

         Informative note: Conveyance of sprop-vps, sprop-sps, and
         sprop-pps using the "fmtp" source attribute allows for out-of-
         band transport of parameter sets in topologies like Topo-Video-
         switch-MCU as specified in [RFC7667].
We deleted the informative note for brevity’s sake but can reinsert it if you see a value (and if the note makes sense to you).


Q3:

Section 7.2.1 says:


      When included in

      the "a=fmtp" line of SDP, those parameters are expressed as a

      media type string, in the form of a semicolon-separated list of

      parameter=value pairs.  When conveyed in the "a=fmtp" line of SDP

      for a particular payload type, the parameters sprop-vps, sprop-

      sps, sprop-pps, sprop-sei, and sprop-dci MUST be applied to each

      SSRC with the payload type.

First, what is a “media type string”?

It’s not a term of art that I’m aware of.  Absent that, I think its definition is in the same sentence.  If “media type string” is confusing (and I think it may well be, considering that it sounds like a term of art without being one), we could simply remove “a media type string, in the form of”.  The result would read

      When included in

      the "a=fmtp" line of SDP, those parameters are expressed as

      a semicolon-separated list of

      parameter=value pairs.  When conveyed in the "a=fmtp" line of SDP

      for a particular payload type, the parameters sprop-vps, sprop-

      sps, sprop-pps, sprop-sei, and sprop-dci MUST be applied to each

      SSRC with the payload type.


Second, the “fmtp” attribute is ALWAYS associated with a particular payload type, isn’t it?

I believe so, yes.

---

Q4:

It would be very useful to have some SDP offer/answer examples.

This has come up before in the AVTCORE WG.  There are upsides and downsides to examples.  One key downside is that no few implementers have taken the presence of example as an excuse of not reading/understanding the spec, leading to botched implementations.  I’m not an implementer myself—never really was—but that’s a story I have heard often.  Therefore, we authors consciously minimized the number of examples in this spec.  As said, we discussed that in AVTCORE, last during IETF112.  Here are the relevant notes:

  5. RTP Payloads for VVC/EVC (Stefan Wenger, 15 min)
https://datatracker.ietf.org/doc/html/draft-ietf-avtcore-rtp-vvc
https://datatracker.ietf.org/doc/html/draft-ietf-avtcore-rtp-evc
- Stephan Wenger asked whether more complete examples of SDP Offer/Asnwer are needed, and recommends no. Simple cases already have examples.
- Bernard Aboba: Is an additional profile document needed to enable interoperability? Without this, we are seeing problems with HEVC interop.
- Mo Zanaty: examples for all complex cases are not needed, but some basic ones might help.
My recollection is that I went out of the virtual room with a clear mandate not to include more and more complex examples.  I don’t have time right now to verify that from the audio record.

In the light of this discussion, would you suggest to add more examples?

---

Q5:

Section 7.2.2.3 says:


      When dealing with VVC, the offerer assumes that the answerer will

      be able to receive media encoded using the configuration being

      offered.

And what if the answerer is not able to?

---


Once more, this is adapted from the H.265 payload format, RFC 7798, page 65 towards the bottom.  I have only a weak recollection of the reasoning.  The context of this sentence is the negotiation of sprop-max-don-diff and sprop-depack-buf-bytes.  For both of these parameters, there are reasonably low maximums that would prevent the use of the maximum range to become a serious implementation burden for a receiver.  Insofar, we require that the receiver is able to handle anything the sender could throw at him, within those hard limits of course.  Instead, these parameters are now packet stream properties that can be used to optimize/minimize playout delay in the receiver (by knowing the maximum interleaving and the maximum depacketization buffer that the sender is ever using).

Co-authors, especially Miska: I think you were deeper into this than me.  Is above about right?

Q6:

Is there a reason why Section 7.2.2.3 is not placed before Sections 7.2.2.1 and 7.2.2.2, as it applies to both?

---

Again, a copy-paste leftover and ancient RFC 7798 history.  No particular reason that I recall.

Q7:

The text says nothing (as far as I can tell) sending subsequent offers and answers within a session, and whether there are restrictions etc regarding which parameters can be modified etc.

Uh.  That’s a hard one.  Once more, our starting point RFC 7798 didn’t say anything either AFAICT, and no one complained about it.  Old payload formats for ITU codecs had somewhat similar SDP sections, and I also do not recall anyone complaining about the absence of that aspect.  Perhaps that is because most folks who use complex H.26x bitstreams negotiate them using something other than SDP.  Whatever.

I’m not against adding language, but, as it should be obvious by now, I’m far from an SDP expert or enthusiast, and AFAIK so are my co-authors.  If such modifications of existing sessions were desirable to anyone using SDP, we would need to find the right people to work towards it.  So, folks in AVTCore, is that something we should address, and who is willing to help?

---

Thanks again, Christer,

Stephan

Regards,

Christer