Re: [AVTCORE] [External] VVC payload format, open issue cluster #1 (SDP for scalability)

[Alexandre GOUAILLARD <[e-mail]>]

Hi stefan,

Sure.  How does a week later (13th) sound for the next version?
>

I think it should be ok, worse case scenario we can discuss it at the jan
28th interim meeting.


> Some of us will be busy with MPEG, but unless there is a deluge of useful
> comments during that week, I think we will cope.
>

Yes I do remember our discussion during the AV1 codec group weekly meeting
about being courteous to other standardization bodies by not having AV1
meetings when other bodies had some. Ironically that discussion happened
during an IETF meeting week, and you explained that MPEG was important, but
IETF was not, or not enough to extend the courtesy to IETF.


> No, regarding MMUSIC.  The design of the offer-answer part of an RTP
> payload spec is AVT territory.
>

If you are just defining the Mapping of Media Subtype Parameters to SDP and
not extending any SDP feature, or SDP O/A feature, you are good. This is
not always the case, and specifically in the case of simulcast and SVC
modes, work had to be done when writing the AV1 RTP payload specification.
I just wanted to make sure you were aware this could be the case for you
too since you are touching scalability modes.

Regards,


> S.
>
>
>
>
>
> *From: *Alexandre GOUAILLARD <[e-mail]>
> *Date: *Friday, December 18, 2020 at 03:36
> *To: *"Sanchez de la Fuente, Yago" <[e-mail]>
> *Cc: *Stephan Wenger <[e-mail]>, "[e-mail]" <[e-mail]>
> *Subject: *Re: [AVTCORE] [External] VVC payload format, open issue
> cluster #1 (SDP for scalability)
>
>
>
> Hi stefan,
>
>
>
> Some of the AVTCORE members who have worked on the design of RTP
> payload for AV1, VP9 and other SVC / Simulcast features in webrtc, included
> but not limited to SDP O/A, RID and others, are on vacation until january
> 7th.
>
>
>
> I think your proposal would benefit a lot from their spending time and
> providing feedback from an informed position, and I would humbly request
> that you push back your deadline for them to participate in the discussion.
> I do believe that the discussion can go for as long as the chairs want, and
> there is usually no deadline per say anyway, but it's alway nicer to ask.
>
>
>
> Also, I believe that anything related to SDP O/A and its modifications
> needs to be proposed to the MMUSIC group.
>
>
>
> Regards.
>
>
>
>
>
>
>
> On Fri, Dec 18, 2020 at 4:48 PM Sanchez de la Fuente, Yago <
> [e-mail]> wrote:
>
> Thanks Stephan,
>
>
>
> Your suggestion looks good to me as well.
>
>
>
> Best regards,
>
> Yago Sánchez
>
>
>
> ---
>
> Department Video Coding & Analytics
>
> Group Multimedia Communications
>
> Fraunhofer HHI - Heinrich Hertz Institute
> Einsteinufer 37, 10587 Berlin, Germany
> http://www.hhi.fraunhofer.de/ip/mc
>
> Tel.: [phone]
>
> [e-mail]
>
>
>
>
>
>
>
> On 17. Dec 2020, at 18:49, Ye-Kui Wang <[e-mail]> wrote:
>
>
>
> Thanks Stephan for the long email (which is sort of usual for you anyway
> 😊).
>
>
>
> The suggestion looks good to me.
>
>
>
> BR, YK
>
>
>
> *From:* avt <[e-mail]> *On Behalf Of *Stephan Wenger
> *Sent:* Thursday, December 17, 2020 8:31
> *To:* [e-mail]
> *Subject:* [External] [AVTCORE] VVC payload format, open issue cluster #1
> (SDP for scalability)
>
>
>
> Hi,
>
> This is the first in a series of emails in which we authors will try to
> propose ways to solve open issues indicated in the draft (
> https://tools.ietf.org/html/draft-ietf-avtcore-rtp-vvc-06).  As we have
> many of those, we will try to organize those open issues into meaningful
> clusters get piecemeal agreement on each cluster.  Once received, we will
> update the draft and go for the next cluster.  Sometimes, we will overlap
> unrelated clusters.
>
>
>
> Many of these emails are going to be long, sorry for that.  Many also
> require quite detailed knowledge of H.266.  Occasionally, we will include
> some explanatory language on the H.266 feature in question.  If you want to
> dive deeper, H.266 is now available for free download from
> https://www.itu.int/itu-t/recommendations/rec.aspx?rec=14336 –have fun
> reading the 500 pages of super-dense (and small font) text.
>
>
>
> The first cluster concerns section 7.1, Media Type Registration, and
> therein specifically the codepoints and language related to non-temporal
> scalability support.  This is a hard problem, quite possibly the hardest we
> need to address in this exercise.  I apologize in advance for the word
> count.
>
>
>
> As a very brief and oversimplified recap, H.266/VVC supports temporal
> scalability in all of its profiles except the still image profiles.  In
> that, H.266 is similar to H.265/HEVC.  However, H.266 also has two scalable
> profiles (one for 4:2:0 and the another for 4:4:4) that allow
>
> for a powerful spatial/SNR scalability mode.  Many of us believe that
> these profiles have higher chances of success compared to previous video
> coding standards, because they are a) not requiring much in terms of
> decoder implementation complexity beyond non-scalable implementations, and
> b) are conceptually similar to what VP9 and AV1 use, and hence familiar to
> those using modern scalable codecs, like the webrtc folks.  Insofar, we
> want to support scalability in the payload format from the outset.
>
>
>
> If only temporal scalability is supported, the selection of an appropriate
> sub-layer in the onion-shaped multi-sublayered stream by an SDP-based
> offer-answer is a one-dimensional problem, and has been solved in places
> like the HEVC payload format by an offer-answer exchange involving a single
> numeric parameter indicative of the highest supported sub-layer.  However,
> when having all temporal, spatial, and SNR scalability, the optimal
> selection of an appropriate output layer set (OLS) becomes a complex
> multi-dimensional problem, that may have more than one solution for
> different values of “optimal”, and also requires certain context.  We try
> to solve this multi-dimensional problem by a combination of a requirement
> for conditional presence of certain otherwise optional SDP codepoints when
> scalability is in use, and the introduction of an additional few code
> points.  Alternatives we rejected include not supporting scalability, or
> relying exclusively on the exchange of possibly many variants of parameter
> sets, as that could lead to excessively large SDP blobs (easily larger than
> the 1500 bytes still common as MTU).  Note that the latter option can still
> be implemented for cases where our restrictions are considered
> inappropriate.
>
>
>
> The design philosophy is as follows:
>
>    1. When using scalability and OLS-based negotiation, the sender must
>    include a Video Parameter Set (VPS) as a sprop-vps or have otherwise
>    pre-arranged knowledge that the VPS is known by the receiver.  For those
>    unfamiliar with H.266, the VPS includes a “table of content” of the
>    layering structure, and without it, the values of new codepoints
>    (sprop-ols-id and recv-ols-id) could not be interpreted.
>    2. sprop-ols-id will typically point to the “best” output layer
>    set—the combination of layers providing the most pleasing reconstructed
>    result—that the sender can support taking into account all other parameters
>    known at this point of the O/A process.
>    3. The value of sprop-ols-id selects one of possibly several OLSs
>    branches in the VPS.  In the reply, using recv-ols-id, the answerer can
>    select that OLS or other OLSs from the same branch of scalable layers in
>    the VPS to which the sprop-ols-id belongs, but not any other OLSs that may
>    exist in the VPS.  The worst-case fallback will be base-layer only, which
>    is always supported, assuming H.266 and the relevant profiles tiers, and
>    levels are supported.
>    4. Sublayers may be negotiated as in the HEVC payload format and in
>    parallel.
>    5. The interaction between what’s going on for the (temporal)
>    sub-layers and for the (spatial/SNR) layers is an application logic problem
>    and does not need to be specified, nor documented.  It’s very hard to grok
>    for anyone not intimately familiar with H.266.
>    6. With these design principles, sprop-opi is not needed anymore and
>    will be removed.
>
>
>
> Based on this design, the following changes can be made in the draft:
>
> -Remove edt note #5, and remove the mentioning of sprop-opi in the
> sentence above note 5.
>
> -Remove edt note #6, and remove the mentioning of sprop-opi in the
> sentence above note 6.
>
> -sprop-ols-id: add a sentence “if this optional parameter is present,
> sprop-vps must also be present or its content must be known a priori at the
> receiver”
>
> -sprop-recv-id: add “sprop-recv-id, when present, must be included only
> when sprop-ols-id was received and must refer to an output layer set in the
> VPS that is in the same dependency tree as the OLS referred to by
> sprop-ols-id.  If this optional parameter is present, sprop-vps must have
> been received or its content must be known a priori at the receiver”
>
> -Remove the sprop-opi language and edt note #17
>
>
>
> Please provide comments to this design choice by Jan 6th.  Absent
> comments, we will implement the changes as described above, also any
> changes required for consistency, and we will add corresponding language in
> the currently empty offer-answer section of the draft.
>
>
>
> Thanks,
>
> Stephan
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [e-mail]
> https://www.ietf.org/mailman/listinfo/avt
>
>
>
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [e-mail]
> https://www.ietf.org/mailman/listinfo/avt
>
>
>
>
> --
>
> Alex. Gouaillard, PhD, PhD, MBA
>
>
> ------------------------------------------------------------------------------------
>
> President - CoSMo Software Consulting, Singapore
>
>
> ------------------------------------------------------------------------------------
>
> sg.linkedin.com/agouaillard
>
>    -
>
>

-- 
Alex. Gouaillard, PhD, PhD, MBA
------------------------------------------------------------------------------------
President - CoSMo Software Consulting, Singapore
------------------------------------------------------------------------------------
sg.linkedin.com/agouaillard

   -