Re: [AVTCORE] Review of draft-ietf-avtext-framemarking-05

["Mo Zanaty (mzanaty)" <[e-mail]>]

Sergio,
You originally raised the issues with VP9 U/P bits. Are you ok with the restriction that frame marking requires nested temporal layers? That would resolve the U bit issue.

For multiple spatial enhancement layers, I propose tweaking the definition of the frame marking I (Independent) bit to mean no temporal dependencies while allowing dependencies on lower spatial layers that are temporally co-located. So S2* below would set the I bit to indicate a spatial layer up switching point.

This change would mean frame marking could not be used to differentiate between Intra frames of simulcast spatial layers vs Intra frames of scalable spatial layers (with inter-layer prediction dependencies). I'm fine with that, since such a distinction really belongs at a higher level (media negotiation / signaling, overall stream or layer properties, etc.) rather than per RTP packet.

Does this work for all?

Mo

On Sep 7, 2017, at 6:30 PM, Bernard Aboba <[e-mail]<mailto:[e-mail]>> wrote:

Jonathan said:

" If this is the case, then VP9’s U bit wouldn’t need mapping — any VP9 stream which could validly have frame marking applied to it would always have the U bit set to true, and MANEs could start sending temporal layers at any time."

[BA] I would be ok with a temporal nesting restriction, since that restriction is so commonly applied anyway.

"However, as I understand it, it would *not* have the frame marking B or I bits set, nor any other fields in the frame marking header extension which distinguish it."

[BA] That was my reading as well.  So it seemed to me that another bit would be needed.

On Thu, Sep 7, 2017 at 1:29 PM, Jonathan Lennox <[e-mail]<mailto:[e-mail]>> wrote:
I think the issue isn’t specifically with VP9 mapping as such, but rather with the semantics that the VP9 payload’s U and P bits express.

Essentially, VP9’s U indicates that this is a point at which a MANE can validly start sending a higher temporal layer, and P indicates a point at which a MANE can start sending a higher spatial layer.

It’s not clear to me that frame marking makes it possible to extract either of these semantics at the moment.

Possibly, the (vastly underspecified) statement in section 3.4.2, requiring that frame marking not be used for “complex or irregular” scalability structures, means that (among other things) any stream using frame marking needs to be temporally nested.  (See the LRR draft for a definition of temporal nesting).  If this is the case, then VP9’s U bit wouldn’t need mapping — any VP9 stream which could validly have frame marking applied to it would always have the U bit set to true, and MANEs could start sending temporal layers at any time.  I’d be fine with this restriction, but it would need to be written down explicitly; and I don’t understand everyone else’s use cases well enough to be sure that no one else would have a problem with it.

For the P bit, the issue is indeed about spatial layer switching-up points.  Consider the following stream encoded with three spatial layers:

        ... <--  S2  <--  S2       S2* <--  S2  <-- ...
                  |        |        |        |
                 \/       \/       \/       \/
        ... <--  S1  <--  S1  <--  S1  <--  S1  <-- ...
                  |        |        |        |
                 \/       \/       \/       \/
... <--  S0  <--  S0  <--  S0  <--  S0  <-- ...

The frame marked with an asterisk is a layer refresh point for layer S2, and is a point at which a MANE which was previously forwarding just layer S1 could start sending S2 as well.  This frame would have the VP9 P bit set.  However, as I understand it, it would *not* have the frame marking B or I bits set, nor any other fields in the frame marking header extension which distinguish it.

Possibly the definition of the B bit could be changed somehow to provide this information, but I feel like the current B bit semantics provides other functionality (about the decodability of non base-temporal-layer frames following loss) which we don’t want to break; I’m not sure how best to square this circle.


On Sep 1, 2017, at 6:15 PM, Mo Zanaty (mzanaty) <[e-mail]<mailto:[e-mail]>> wrote:

Thanks for bumping this, Cullen.

At IETF 99, it was decided that priority marking beyond the Discardable
bit should go in a separate draft. Roni submitted
draft-even-avtcore-priority-markings for this. So that is closed.

The only other open issue was the mapping of the VP9 U/P bits to the frame
marking B/I bits. In version 05 of frame marking, we already moved VP9
LID/TID mappings to the VP9 RTP payload draft. If the frame marking B/I
bits are sufficient for the needs of VP9 SVC applications, then the
mapping of VP9 U/P bits to frame marking B/I bits should also be
documented in the VP9 draft rather than frame marking, and we can request
WGLC for frame marking without waiting for the VP9 draft updates. If the
frame marking B/I bits are insufficient, or if their definition needs to
be altered to support VP9 SVC applications, and the WG feels those
applications are important, then we need to finalize any changes needed
for frame marking B/I bits ASAP then start WGLC.

Jonathan, Sergio, Bernard,
You expressed interest in VP9 SVC applications. Are the frame marking B/I
bits sufficient for your needs? If not, what are the specific issues we
need to address? Is it only switching up to a higher spatial layer when
there are multiple spatial enhancement layers?

Thanks,
Mo


On Sep 1, 2017, at 8:23 AM, Cullen Jennings <[e-mail]<mailto:[e-mail]>> wrote:

I read this draft and did not see any issues. Where are we with this
draft and what happens next ?