Re: [AVTCORE] Finishing up framemarking....

[Bernard Aboba <[e-mail]>]

An update. The VP9 RTP payload no longer references framemarking, so
framemarking is not holding up publication (LRR is the MISREF).

The text that Mo references is now in Section 9 of the VP9 spec (
https://datatracker.ietf.org/doc/html/draft-ietf-payload-vp9-16#section9)

9 <https://datatracker.ietf.org/doc/html/draft-ietf-payload-vp9-16#section-9>.
Congestion Control

   Congestion control for RTP SHALL be used in accordance with RFC
3550 <https://datatracker.ietf.org/doc/html/rfc3550>
   [RFC3550 <https://datatracker.ietf.org/doc/html/rfc3550>], and with
any applicable RTP profile; e.g., RFC 3551
<https://datatracker.ietf.org/doc/html/rfc3551>
   [RFC3551 <https://datatracker.ietf.org/doc/html/rfc3551>].  The
congestion control mechanism can, in a real-time
   encoding scenario, adapt the transmission rate by instructing the
   encoder to encode at a certain target rate.  Media aware network
   elements MAY use the information in the VP9 payload descriptor in
   Section 4.2 <https://datatracker.ietf.org/doc/html/draft-ietf-payload-vp9-16#section-4.2>
to identify non-reference frames and discard them in
   order to reduce network congestion.  Note that discarding of non-
   reference frames cannot be done if the stream is encrypted (because
   the non-reference marker is encrypted).



Section 4.2 now includes definition of an N bit:


   Reference indices:  When P and F are both set to one, indicating a
      non-key frame in flexible mode, then at least one reference index
      MUST be specified as below.  Additional reference indices (total
      of up to 3 reference indices are allowed) may be specified using
      the N bit below.  When either P or F is set to zero, then no
      reference index is specified.

      P_DIFF:  The reference index (in 7 bits) specified as the relative
         PID from the current picture.  For example, when P_DIFF=3 on a
         packet containing the picture with PID 112 means that the
         picture refers back to the picture with PID 109.  This
         calculation is done modulo the size of the PID field, i.e.,
         either 7 or 15 bits.  A P_DIFF value of 0 is invalid.

      N:  1 if there is additional P_DIFF following the current P_DIFF.


On Sun, Jul 11, 2021 at 8:04 AM Bernard Aboba <[e-mail]>
wrote:

> On Thursday, March 11, 2021, Mo posted a question relating to VP9:
> https://mailarchive.ietf.org/arch/msg/avt/JA4b0XAllPtrSiTRxr8JIeGWR7E/
>
> I believe this is the only remaining issue on the framemarking document.
> Since framemarking is holding up publication of the VP9 RTP payload as an
> RFC, it would be good to move this toward resolution at IETF 111 (or
> ideally, before).
>
> --------------
>
> VP9 experts,
>
> While updating frame marking to add back VP9, I had difficulty finding an indicator for discardable non-reference frames in the VP9 RTP payload draft to map to the frame marking “D” bit. VP8 has the “N” bit for this in the first payload byte (payload descriptor). But VP9 lacks anything similar, although section 8 thinks this should be in section 4.2 (but there is nothing there or elsewhere).
> https://tools.ietf.org/html/draft-ietf-payload-vp9-11#section-8
>    ...Media aware network elements MAY use the information in the VP9 payload descriptor in Section 4.2<https://tools.ietf.org/html/draft-ietf-payload-vp9-11#section-4.2> <http://4.2%3Chttps//tools.ietf.org/html/draft-ietf-payload-vp9-11#section-4.2%3E> to identify non-reference frames and discard them in order to reduce network congestion.  Note that discarding of non-reference frames cannot be done if the stream is encrypted (because the non-reference marker is encrypted).
>
> Is this an oversight? Or was the non-ref indicator intentionally removed? If so, why?
>
> From my work on AV1, I know how the 8 reference frames work in AV1/VP9, so I devised a solution for frame marking. But it seems like a hack to require this knowledge and go deep into the bitstream uncompressed header (not RTP payload descriptor) to extract this information. Is this really what we must do for VP9? Here is what I put in frame marking version -12, which seems like a hack:
> https://tools.ietf.org/html/draft-ietf-avtext-framemarking-12#section-3.3.1
>    The D bit MUST be 1 if the refresh_frame_flags in the VP9 payload uncompressed header are all 0, otherwise it MUST be 0.
>
> Reading the refresh_frame_flags from the payload requires some payload parsing and checks since it’s not a fixed offset, and requires knowing subtle nuances like key frames implicitly force all 8 flags to 1. If implementers have been confused by something as simple as TL0PICIDX++, they will likely stumble on this as well.
>
> Is it possible to revive the VP8 N bit? This would simplify frame marking, AV1 DD, GCC, L2/L3 QoS mappings, or anything else that wants to know about discardable data.
>
> Regards,
> Mo
>
>