Re: [AVTCORE] [External] VVC payload format, open issue cluster #1 (SDP for scalability)

[Alexandre GOUAILLARD <[e-mail]>]

you seem to know what yo uare doing, we cand discuss further point at the
meeting on january 28th. or later.

On Tue, Dec 22, 2020 at 1:37 AM Stephan Wenger <[e-mail]> wrote:

> Dr. Alex,
>
> Pls see inline in blue.
>
> S.
>
>
>
>
>
> *From: *Alexandre GOUAILLARD <[e-mail]>
> *Date: *Monday, December 21, 2020 at 02:03
> *To: *Stephan Wenger <[e-mail]>
> *Cc: *"Sanchez de la Fuente, Yago" <[e-mail]>, "
> [e-mail]" <[e-mail]>
> *Subject: *Re: [AVTCORE] [External] VVC payload format, open issue
> cluster #1 (SDP for scalability)
>
>
>
> Hi stefan,
>
>
>
> Sure.  How does a week later (13th) sound for the next version?
>
>
>
> I think it should be ok, worse case scenario we can discuss it at the jan
> 28th interim meeting.
>
>
>
> Some of us will be busy with MPEG, but unless there is a deluge of useful
> comments during that week, I think we will cope.
>
>
>
> Yes I do remember our discussion during the AV1 codec group weekly meeting
> about being courteous to other standardization bodies by not having AV1
> meetings when other bodies had some. Ironically that discussion happened
> during an IETF meeting week, and you explained that MPEG was important, but
> IETF was not, or not enough to extend the courtesy to IETF.
>
>
>
> Being cited out of context is all the more fun when the citation is
> incorrect :-)
>
>
>
> No, regarding MMUSIC.  The design of the offer-answer part of an RTP
> payload spec is AVT territory.
>
>
>
> If you are just defining the Mapping of Media Subtype Parameters to SDP
> and not extending any SDP feature, or SDP O/A feature, you are good. This
> is not always the case, and specifically in the case of simulcast and SVC
> modes, work had to be done when writing the AV1 RTP payload specification.
> I just wanted to make sure you were aware this could be the case for you
> too since you are touching scalability modes.
>
>
>
> For those not active in the Alliance for Open Media (AOM) codec WG and RTC
> subgroup (which, I fear, is the majority here): AOM decided a long time ago
> to design the payload format for AV1 outside of the IETF.  While the
> deliberations of the subgroup are AFAIK open only to AOM members, the
> github with the draft spec is in the open, secured only by obscurity :-).
> The current (I hope) and I believe close to final AV1 payload spec can be
> found here:
> https://aomediacodec.github.io/av1-rtp-spec/#723-usage-with-the-sdp-offeranswer-model
>
>
>
> @Dr. Alex: I do not see anything in 7.2.3 (or, in fact, anywhere else in
> the AV1 payload spec), that would not normally be done here in AVTcore.
> Can you please elaborate on your previous para?
>
>
>
> Thanks,
> S.
>
>
>
>
>
> Regards,
>
>
>
> S.
>
>
>
>
>
> *From: *Alexandre GOUAILLARD <[e-mail]>
> *Date: *Friday, December 18, 2020 at 03:36
> *To: *"Sanchez de la Fuente, Yago" <[e-mail]>
> *Cc: *Stephan Wenger <[e-mail]>, "[e-mail]" <[e-mail]>
> *Subject: *Re: [AVTCORE] [External] VVC payload format, open issue
> cluster #1 (SDP for scalability)
>
>
>
> Hi stefan,
>
>
>
> Some of the AVTCORE members who have worked on the design of RTP
> payload for AV1, VP9 and other SVC / Simulcast features in webrtc, included
> but not limited to SDP O/A, RID and others, are on vacation until january
> 7th.
>
>
>
> I think your proposal would benefit a lot from their spending time and
> providing feedback from an informed position, and I would humbly request
> that you push back your deadline for them to participate in the discussion.
> I do believe that the discussion can go for as long as the chairs want, and
> there is usually no deadline per say anyway, but it's alway nicer to ask.
>
>
>
> Also, I believe that anything related to SDP O/A and its modifications
> needs to be proposed to the MMUSIC group.
>
>
>
> Regards.
>
>
>
>
>
>
>
> On Fri, Dec 18, 2020 at 4:48 PM Sanchez de la Fuente, Yago <
> [e-mail]> wrote:
>
> Thanks Stephan,
>
>
>
> Your suggestion looks good to me as well.
>
>
>
> Best regards,
>
> Yago Sánchez
>
>
>
> ---
>
> Department Video Coding & Analytics
>
> Group Multimedia Communications
>
> Fraunhofer HHI - Heinrich Hertz Institute
> Einsteinufer 37, 10587 Berlin, Germany
> http://www.hhi.fraunhofer.de/ip/mc
>
> Tel.: [phone]
>
> [e-mail]
>
>
>
>
>
>
>
> On 17. Dec 2020, at 18:49, Ye-Kui Wang <[e-mail]> wrote:
>
>
>
> Thanks Stephan for the long email (which is sort of usual for you anyway
> 😊).
>
>
>
> The suggestion looks good to me.
>
>
>
> BR, YK
>
>
>
> *From:* avt <[e-mail]> *On Behalf Of *Stephan Wenger
> *Sent:* Thursday, December 17, 2020 8:31
> *To:* [e-mail]
> *Subject:* [External] [AVTCORE] VVC payload format, open issue cluster #1
> (SDP for scalability)
>
>
>
> Hi,
>
> This is the first in a series of emails in which we authors will try to
> propose ways to solve open issues indicated in the draft (
> https://tools.ietf.org/html/draft-ietf-avtcore-rtp-vvc-06).  As we have
> many of those, we will try to organize those open issues into meaningful
> clusters get piecemeal agreement on each cluster.  Once received, we will
> update the draft and go for the next cluster.  Sometimes, we will overlap
> unrelated clusters.
>
>
>
> Many of these emails are going to be long, sorry for that.  Many also
> require quite detailed knowledge of H.266.  Occasionally, we will include
> some explanatory language on the H.266 feature in question.  If you want to
> dive deeper, H.266 is now available for free download from
> https://www.itu.int/itu-t/recommendations/rec.aspx?rec=14336 –have fun
> reading the 500 pages of super-dense (and small font) text.
>
>
>
> The first cluster concerns section 7.1, Media Type Registration, and
> therein specifically the codepoints and language related to non-temporal
> scalability support.  This is a hard problem, quite possibly the hardest we
> need to address in this exercise.  I apologize in advance for the word
> count.
>
>
>
> As a very brief and oversimplified recap, H.266/VVC supports temporal
> scalability in all of its profiles except the still image profiles.  In
> that, H.266 is similar to H.265/HEVC.  However, H.266 also has two scalable
> profiles (one for 4:2:0 and the another for 4:4:4) that allow
>
> for a powerful spatial/SNR scalability mode.  Many of us believe that
> these profiles have higher chances of success compared to previous video
> coding standards, because they are a) not requiring much in terms of
> decoder implementation complexity beyond non-scalable implementations, and
> b) are conceptually similar to what VP9 and AV1 use, and hence familiar to
> those using modern scalable codecs, like the webrtc folks.  Insofar, we
> want to support scalability in the payload format from the outset.
>
>
>
> If only temporal scalability is supported, the selection of an appropriate
> sub-layer in the onion-shaped multi-sublayered stream by an SDP-based
> offer-answer is a one-dimensional problem, and has been solved in places
> like the HEVC payload format by an offer-answer exchange involving a single
> numeric parameter indicative of the highest supported sub-layer.  However,
> when having all temporal, spatial, and SNR scalability, the optimal
> selection of an appropriate output layer set (OLS) becomes a complex
> multi-dimensional problem, that may have more than one solution for
> different values of “optimal”, and also requires certain context.  We try
> to solve this multi-dimensional problem by a combination of a requirement
> for conditional presence of certain otherwise optional SDP codepoints when
> scalability is in use, and the introduction of an additional few code
> points.  Alternatives we rejected include not supporting scalability, or
> relying exclusively on the exchange of possibly many variants of parameter
> sets, as that could lead to excessively large SDP blobs (easily larger than
> the 1500 bytes still common as MTU).  Note that the latter option can still
> be implemented for cases where our restrictions are considered
> inappropriate.
>
>
>
> The design philosophy is as follows:
>
>    1. When using scalability and OLS-based negotiation, the sender must
>    include a Video Parameter Set (VPS) as a sprop-vps or have otherwise
>    pre-arranged knowledge that the VPS is known by the receiver.  For those
>    unfamiliar with H.266, the VPS includes a “table of content” of the
>    layering structure, and without it, the values of new codepoints
>    (sprop-ols-id and recv-ols-id) could not be interpreted.
>    2. sprop-ols-id will typically point to the “best” output layer
>    set—the combination of layers providing the most pleasing reconstructed
>    result—that the sender can support taking into account all other parameters
>    known at this point of the O/A process.
>    3. The value of sprop-ols-id selects one of possibly several OLSs
>    branches in the VPS.  In the reply, using recv-ols-id, the answerer can
>    select that OLS or other OLSs from the same branch of scalable layers in
>    the VPS to which the sprop-ols-id belongs, but not any other OLSs that may
>    exist in the VPS.  The worst-case fallback will be base-layer only, which
>    is always supported, assuming H.266 and the relevant profiles tiers, and
>    levels are supported.
>    4. Sublayers may be negotiated as in the HEVC payload format and in
>    parallel.
>    5. The interaction between what’s going on for the (temporal)
>    sub-layers and for the (spatial/SNR) layers is an application logic problem
>    and does not need to be specified, nor documented.  It’s very hard to grok
>    for anyone not intimately familiar with H.266.
>    6. With these design principles, sprop-opi is not needed anymore and
>    will be removed.
>
>
>
> Based on this design, the following changes can be made in the draft:
>
> -Remove edt note #5, and remove the mentioning of sprop-opi in the
> sentence above note 5.
>
> -Remove edt note #6, and remove the mentioning of sprop-opi in the
> sentence above note 6.
>
> -sprop-ols-id: add a sentence “if this optional parameter is present,
> sprop-vps must also be present or its content must be known a priori at the
> receiver”
>
> -sprop-recv-id: add “sprop-recv-id, when present, must be included only
> when sprop-ols-id was received and must refer to an output layer set in the
> VPS that is in the same dependency tree as the OLS referred to by
> sprop-ols-id.  If this optional parameter is present, sprop-vps must have
> been received or its content must be known a priori at the receiver”
>
> -Remove the sprop-opi language and edt note #17
>
>
>
> Please provide comments to this design choice by Jan 6th.  Absent
> comments, we will implement the changes as described above, also any
> changes required for consistency, and we will add corresponding language in
> the currently empty offer-answer section of the draft.
>
>
>
> Thanks,
>
> Stephan
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [e-mail]
> https://www.ietf.org/mailman/listinfo/avt
>
>
>
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [e-mail]
> https://www.ietf.org/mailman/listinfo/avt
>
>
>
>
> --
>
> Alex. Gouaillard, PhD, PhD, MBA
>
>
> ------------------------------------------------------------------------------------
>
> President - CoSMo Software Consulting, Singapore
>
>
> ------------------------------------------------------------------------------------
>
> sg.linkedin.com/agouaillard
>
>    -
>
>
>
>
> --
>
> Alex. Gouaillard, PhD, PhD, MBA
>
>
> ------------------------------------------------------------------------------------
>
> President - CoSMo Software Consulting, Singapore
>
>
> ------------------------------------------------------------------------------------
>
> sg.linkedin.com/agouaillard
>
>    -
>
>

-- 
Alex. Gouaillard, PhD, PhD, MBA
------------------------------------------------------------------------------------
President - CoSMo Software Consulting, Singapore
------------------------------------------------------------------------------------
sg.linkedin.com/agouaillard

   -