Re: [payload] VP9 frame markings: typo and questions

[Jonathan Lennox <[e-mail]>]

Hi, Sergio — sorry for not responding sooner.

> On May 8, 2018, at 12:12 PM, Sergio Garcia Murillo <[e-mail]> wrote:
> 
> Hi,
> 
> I have some comments regarding the latest vp9 draft regarding the framemarking header information.
> 
> I think it is confusing to have two fields on the text named "S"
>        0                   1                   2                   3
>        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |  ID=2 |  L=2  |S|E|I|D|B|  T  |0|0|0|0|0|  S  |    TL0PICIDX  |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 
> IMHO it would be better to rename S/T fields for spatial and temporal ids to TID/SID to be consistent with framemarking draft.
> 
>        0                   1                   2                   3
>        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |  ID=2 |  L=2  |S|E|I|D|B| TID |0|0|0|0|0| SID |    TL0PICIDX  |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Good idea.  I’ll convert them throughout.

> 
> Also,  I have doubts about how to fill the I/D/B fields of the vp9 framemarking :
> 
>    o  I: Independent Frame (1 bit) - MUST be 1 for frames that can be
>       decoded independent of temporally prior frames, e.g. intra-frame,
>       VPX keyframe, H.264 IDR [RFC6184 <https://tools.ietf.org/html/rfc6184>], H.265 IDR/CRA/BLA/RAP
>       [RFC7798 <https://tools.ietf.org/html/rfc7798>]; otherwise MUST be 0.  Note that this bit only signals
>       temporal independence, so it can be 1 in spatial or quality
>       enhancement layers that depend on temporally co-located layers but
>       not temporally prior frames.
>    o  D: Discardable Frame (1 bit) - MUST be 1 for frames that can be
>       discarded, and still provide a decodable media stream; otherwise
>       MUST be 0.
>    o  B: Base Layer Sync (1 bit) - MUST be 1 if this frame only depends
>       on the base layer; otherwise MUST be 0.  If no scalability is
>       used, this MUST be 0.
> 
> from the information of the VP9 payload description (and viceversa).
> 
>    P: Inter-picture predicted frame.  When set to zero, the frame does
>       not utilize inter-picture prediction.  In this case, up-switching
>       to a current spatial layer's frame is possible from directly lower
>       spatial layer frame.  P SHOULD also be set to zero when encoding a
>       layer synchronization frame in response to an LRR
>       [I-D.ietf-avtext-lrr <https://tools.ietf.org/html/draft-ietf-payload-vp9-05#ref-I-D.ietf-avtext-lrr>] message (see Section 5.4 <https://tools.ietf.org/html/draft-ietf-payload-vp9-05#section-5.4>).  When P is set to
>       zero, the T field (described below) MUST also be set to 0 (if
>       present).  Note that the P bit does not forbid intra-picture,
>       inter-layer prediction from earlier frames of the same picture, if
>       any.
>    U: Switching up point.  If this bit is set to 1 for the current
>       picture with temporal layer ID equal to T, then "switch up" to
>       a higher frame rate is possible as subsequent higher temporal
>       layer pictures will not depend on any picture before the
>       current picture (in coding order) with temporal layer ID
>       greater than T.
>    D: Inter-layer dependency used.  MUST be set to one if current
>       spatial layer S frame depends on spatial layer S-1 frame of the
>       same picture.  MUST only be set to zero if current spatial
>       layer S frame does not depend on spatial layer S-1 frame of the
>       same picture.  For the base layer frame (with S equal to 0),
>       this D bit MUST be set to zero. 
> 
> It seems that fm.I = !desc.P but I am not sure if fm.B=!desc.D (the definition doesn't seem to match). I guess that the frame marking B discardable flag could be retrieved from the scalability structure or frame dependencies (although I don't use it at the SFU). Could you confirm if those assumptions are correct, please? If so, couldn't this info be added to the draft for clarity?

fm.I == !desc.P, yes.

The fm.B flag doesn’t directly correspond to anything in the VP9 header, but can indeed be inferred from the dependencies.

fm.D also doesn’t directly correspond to anything in the VP9 header.  It can be inferred from the scalability structure for non-flexible mode, but for flexible mode you either need to know future frames’ dependencies, or to parse codec headers to see whether the frame is stored as a reference.

> May main problem is with the description U: witching up point which I can't find a field in the framemarking information that could be usable to provide the information to the SFU.
> 
> Am I missing something or should we add the U bit also to the vp9 frame marking info on the LID field?

This was discussed in the context of frame marking.  The consensus was that frame marking can only be used for streams for which the U bit would always be set to “true” — it doesn’t support more complex temporal structures at all.  Thus, the value of the U bit can be inferred from the mere presence of the frame marking header extension.