Re: [AVTCORE] Review of draft-ietf-avtext-framemarking-05

[Sergio Garcia Murillo <[e-mail]>]

Hi guys,

Sorry for the late reply.

It doesn't seem reasonable to me to modify the vp9 svc behavior just
because we are using an rtp header extension or not, specially when we have
4 unused bits on the codec specific info. Frame marking should adapt to
match codec, not the other way round (if not we are doing it wrong IMHO).

Regarding the other bits, I think that the frame marking info on each draft
should not only define what are they, but also how to extract it from the
encoder/rtp data. In case of VP9 I think there should be an explicit
mapping between the frame marking bits and the vp9 payload description
bits. So in this regards, changing the meaning of the I and B bits, I could
agree with it, as long as we say exactly what values of the vp9 payload
description should they be equal too.

Best regards
Sergio


2017-09-08 1:28 GMT+02:00 Mo Zanaty (mzanaty) <[e-mail]>:

> Sergio,
> You originally raised the issues with VP9 U/P bits. Are you ok with the
> restriction that frame marking requires nested temporal layers? That would
> resolve the U bit issue.
>
> For multiple spatial enhancement layers, I propose tweaking the definition
> of the frame marking I (Independent) bit to mean no temporal dependencies
> while allowing dependencies on lower spatial layers that are temporally
> co-located. So S2* below would set the I bit to indicate a spatial layer up
> switching point.
>
> This change would mean frame marking could not be used to differentiate
> between Intra frames of simulcast spatial layers vs Intra frames of
> scalable spatial layers (with inter-layer prediction dependencies). I'm
> fine with that, since such a distinction really belongs at a higher level
> (media negotiation / signaling, overall stream or layer properties, etc.)
> rather than per RTP packet.
>
> Does this work for all?
>
> Mo
>
> On Sep 7, 2017, at 6:30 PM, Bernard Aboba <[e-mail]> wrote:
>
> Jonathan said:
>
> " If this is the case, then VP9’s U bit wouldn’t need mapping — any VP9
> stream which could validly have frame marking applied to it would always
> have the U bit set to true, and MANEs could start sending temporal layers
> at any time."
>
> [BA] I would be ok with a temporal nesting restriction, since that
> restriction is so commonly applied anyway.
>
> "However, as I understand it, it would *not* have the frame marking B or
> I bits set, nor any other fields in the frame marking header extension
> which distinguish it."
>
> [BA] That was my reading as well.  So it seemed to me that another bit
> would be needed.
>
> On Thu, Sep 7, 2017 at 1:29 PM, Jonathan Lennox <[e-mail]>
> wrote:
>
>> I think the issue isn’t specifically with VP9 mapping as such, but rather
>> with the semantics that the VP9 payload’s U and P bits express.
>>
>> Essentially, VP9’s U indicates that this is a point at which a MANE can
>> validly start sending a higher temporal layer, and P indicates a point at
>> which a MANE can start sending a higher spatial layer.
>>
>> It’s not clear to me that frame marking makes it possible to extract
>> either of these semantics at the moment.
>>
>> Possibly, the (vastly underspecified) statement in section 3.4.2,
>> requiring that frame marking not be used for “complex or irregular”
>> scalability structures, means that (among other things) any stream using
>> frame marking needs to be temporally nested.  (See the LRR draft for a
>> definition of temporal nesting).  If this is the case, then VP9’s U bit
>> wouldn’t need mapping — any VP9 stream which could validly have frame
>> marking applied to it would always have the U bit set to true, and MANEs
>> could start sending temporal layers at any time.  I’d be fine with this
>> restriction, but it would need to be written down explicitly; and I don’t
>> understand everyone else’s use cases well enough to be sure that no one
>> else would have a problem with it.
>>
>> For the P bit, the issue is indeed about spatial layer switching-up
>> points.  Consider the following stream encoded with three spatial layers:
>>
>>         ... <--  S2  <--  S2       S2* <--  S2  <-- ...
>>                   |        |        |        |
>>                  \/       \/       \/       \/
>>         ... <--  S1  <--  S1  <--  S1  <--  S1  <-- ...
>>                   |        |        |        |
>>                  \/       \/       \/       \/
>> ... <--  S0  <--  S0  <--  S0  <--  S0  <-- ...
>>
>> The frame marked with an asterisk is a layer refresh point for layer S2,
>> and is a point at which a MANE which was previously forwarding just layer
>> S1 could start sending S2 as well.  This frame would have the VP9 P bit
>> set.  However, as I understand it, it would *not* have the frame marking B
>> or I bits set, nor any other fields in the frame marking header extension
>> which distinguish it.
>>
>> Possibly the definition of the B bit could be changed somehow to provide
>> this information, but I feel like the current B bit semantics provides
>> other functionality (about the decodability of non base-temporal-layer
>> frames following loss) which we don’t want to break; I’m not sure how best
>> to square this circle.
>>
>>
>> On Sep 1, 2017, at 6:15 PM, Mo Zanaty (mzanaty) <[e-mail]>
>> wrote:
>>
>> Thanks for bumping this, Cullen.
>>
>> At IETF 99, it was decided that priority marking beyond the Discardable
>> bit should go in a separate draft. Roni submitted
>> draft-even-avtcore-priority-markings for this. So that is closed.
>>
>> The only other open issue was the mapping of the VP9 U/P bits to the frame
>> marking B/I bits. In version 05 of frame marking, we already moved VP9
>> LID/TID mappings to the VP9 RTP payload draft. If the frame marking B/I
>> bits are sufficient for the needs of VP9 SVC applications, then the
>> mapping of VP9 U/P bits to frame marking B/I bits should also be
>> documented in the VP9 draft rather than frame marking, and we can request
>> WGLC for frame marking without waiting for the VP9 draft updates. If the
>> frame marking B/I bits are insufficient, or if their definition needs to
>> be altered to support VP9 SVC applications, and the WG feels those
>> applications are important, then we need to finalize any changes needed
>> for frame marking B/I bits ASAP then start WGLC.
>>
>> Jonathan, Sergio, Bernard,
>> You expressed interest in VP9 SVC applications. Are the frame marking B/I
>> bits sufficient for your needs? If not, what are the specific issues we
>> need to address? Is it only switching up to a higher spatial layer when
>> there are multiple spatial enhancement layers?
>>
>> Thanks,
>> Mo
>>
>>
>> On Sep 1, 2017, at 8:23 AM, Cullen Jennings <[e-mail]> wrote:
>>
>> I read this draft and did not see any issues. Where are we with this
>> draft and what happens next ?
>>
>>
>>
>>
>>
>