Re: [AVTCORE] Comments on draft-ietf-avtcore-rtp-vvc-05.txt

["Sanchez de la Fuente, Yago" <[e-mail]>]

Hi Martin, Stephan,

I think that one thing that we can definitely do is to include GDR pictures with ph_recovery_poc_cnt equal to 0, as this is basically the same as a CRA and an IDR, just using a different NAL unit type. I think that would be more aligned to VVC than what we have currently in the text.

However, for GDR pictures with ph_recovery_poc_cnt greater than 0, I agree with Stephan’s assessment and I am not sure what’s the better way forward. If GDRs were useful and were to be used in that context then we could allow GDRs with ph_recovery_poc_cnt greater than 0 as a response to FIR, but this would come with an associated complexity as Stephan pointed out.

I am also not sure whether allowing to set the I bit to 1 for  GDR pictures with ph_recovery_poc_cnt greater than 0 is also correct. Such a GDR picture can be used for random access but it is not a picture that can be decoded independent of temporally prior frames as it is typically inter coded (at least not correctly), while https://tools.ietf.org/id/draft-ietf-avtext-framemarking-09.html states that the I bit is used to mark frames that can be decoded independent of temporally prior frames and must be 0 otherwise. Should we relax that constraint and use the I bit just to mark pictures that can be used as for random access or should we keep the property that they are also independent frames from previous pictures?

For both aspects we could go either way, but it would be nice to hear what other people think.

Best regards,
Yago Sánchez

---
Department Video Coding & Analytics
Group Multimedia Communications

Fraunhofer HHI - Heinrich Hertz Institute
Einsteinufer 37, 10587 Berlin, Germany
http://www.hhi.fraunhofer.de/ip/mc

Tel.: [phone]
[e-mail]<mailto:[e-mail]>



On 5. Nov 2020, at 19:49, Stephan Wenger <[e-mail]<mailto:[e-mail]>> wrote:

Hi Martin,

Thanks for those suggested changes, which I think would consistently implement the option to react to a “Full Intra Request” (FIR) with a gradual decoder refresh (GDR) series of pictures.

As to whether we should allow GDR as a reaction to a FIR: I’m a bit torn here.  Arguments can be made either way, please see below.  Would others in the WG please weigh in?

On one hand, the argument (somewhat rephrased here) that VVC’s GDR pictures are a fully specified replacement for traditional “all intra” pictures is a good one.  I concur that this is somewhat new in VVC, compared to older video coding standards.  Pretty much all of those could do some form of GDR (even good old H.261 and MPEG-2), but things were clumsy, results were not guaranteed, or one had to rely on SEI messages and similar exotics for implementation.  In the environments where FIR matters—video conferencing mostly—no one ever used GDR in any context except in those ca. 1990 H.261-based systems which didn’t implement full intra pictures at all, and relied on intra macroblock walk-around during the initial communication setup.

On the other hand, there’s a reason why FIR until now was consistently interpreted as a requirement of sending a single “all intra” picture (whatever that translates to in the various video coding standards and technologies).  That reason was related to the architecture of the MCUs that were around when RFC 5104 was written, back in the 2005-2008 timeframe.  What people requested then was that the internal architecture of an MCU should stay as independent of the codec in use as possible.  For FIR, that means that means: if an MCU sends out a FIR to a sending endpoint, it expects exactly one intra picture at the earliest opportunity that can be used to sync in added decoders of unknown state.  That logic would now have to change to receive either a single IDR picture or a series of pictures that make up a GDR.  A transcoding MCU would have to go further and include the decoding of those multiple pictures with all the tricky (though now fully specified!) stuff that goes on in VVC, before transcoding.

Stephan


From: avt <[e-mail]<mailto:[e-mail]>> on behalf of Martin Pettersson M <martin.m.pettersson=[e-mail]<mailto:martin.m.pettersson=[e-mail]>>
Date: Tuesday, November 3, 2020 at 08:38
To: "shuaiizhao(Shuai Zhao)" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Subject: [AVTCORE] Comments on draft-ietf-avtcore-rtp-vvc-05.txt

Hi,

Thanks for the good progress on the VVC RTP payload format. Below are some suggested modifications for your consideration:


  1.  In section 3.2, add “GDR                       Gradual Decoding Refresh”



  1.  In section 8.4, change “Upon reception of a FIR, a sender must send an IDR picture.” to “Upon reception of a FIR, a sender must send an IDR, a CRA or a GDR picture.”


Motivation:
One of the versatile features in VVC is its support for low-latency coding where the GDR picture is a key component to achieve low latency. Compared to AVC and HEVC where GDR is signaled in an SEI message with optional support by the decoder, the GDR picture in VVC is a normative part of the specification and the decoder must be able to tune in at a GDR picture. Therefore it makes sense to allow a sender to respond with a GDR picture upon receiving a FIR. Note also that a gradual decoding refresh point is mentioned as a possible Decoder Refresh Point in response to the FIR command in https://tools.ietf.org/html/rfc5104.

Sending a CRA picture as a response to FIR would be fine as well in my opinion. I don’t see the reason to exclude that.



  1.  In section 9.1, change “The I bit MUST be 1 when the NAL unit type is 7-9 (inclusive), otherwise it MUST be 0.” to “The I bit MUST be 1 when the NAL unit type is 7-10 (inclusive), otherwise it MUST be 0.”


In section 9.2, change “The I bit MUST be 1 when the NAL unit type is 7-9 (inclusive), otherwise it MUST be 0.” to “The I bit MUST be 1 when the NAL unit type is 7-10 (inclusive), otherwise it MUST be 0.”

Motivation:
NAL unit type 10 is GDR_NUT.

In https://tools.ietf.org/id/draft-ietf-avtext-framemarking-09.html the I bit is specified as:
I: Independent Frame (1 bit) - MUST be 1 for frames that can be decoded independent of temporally prior frames, e.g. intra-frame, VPX keyframe, H.264 IDR [RFC6184], H.265 IDR/CRA/BLA/RAP [RFC7798]; otherwise MUST be 0.

The GDR picture is typically not fully refreshed in one frame, but it does not need prior temporal pictures to start the decoding process, i.e. a bitstream that starts with a GDR picture in VVC is a valid bitstream.

Best regards,
Martin Pettersson


From: avt <[e-mail]<mailto:[e-mail]>> On Behalf Of shuaiizhao(Shuai Zhao)
Sent: den 2 november 2020 23:11
To: [e-mail]<mailto:[e-mail]>
Subject: [AVTCORE] FW: I-D Action: draft-ietf-avtcore-rtp-vvc-05.txt(Internet mail)

In this revision, Yago’s proposal for SDP parameters were implemented in section 7.2.1.

Editor’s notes were added for things we will provide clearfication in next revision.  So do review and critisize lightly. ☺

Best
SZ

From: avt <[e-mail]<mailto:[e-mail]>> on behalf of "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Reply-To: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Date: Monday, November 2, 2020 at 14:07
To: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Subject: [AVTCORE] I-D Action: draft-ietf-avtcore-rtp-vvc-05.txt(Internet mail)


A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Audio/Video Transport Core Maintenance WG of the IETF.

        Title           : RTP Payload Format for Versatile Video Coding (VVC)
        Authors         : Shuai Zhao
                          Stephan Wenger
                          Yago Sanchez
                          Ye-Kui Wang
                Filename        : draft-ietf-avtcore-rtp-vvc-05.txt
                Pages           : 61
                Date            : 2020-11-02

Abstract:
   This memo describes an RTP payload format for the video coding
   standard ITU-T Recommendation H.266 and ISO/IEC International
   Standard 23090-3, both also known as Versatile Video Coding (VVC) and
   developed by the Joint Video Experts Team (JVET).  The RTP payload
   format allows for packetization of one or more Network Abstraction
   Layer (NAL) units in each RTP packet payload as well as fragmentation
   of a NAL unit into multiple RTP packets.  The payload format has wide
   applicability in videoconferencing, Internet video streaming, and
   high-bitrate entertainment-quality video, among other applications.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-avtcore-rtp-vvc/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-avtcore-rtp-vvc-05
https://datatracker.ietf.org/doc/html/draft-ietf-avtcore-rtp-vvc-05

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-avtcore-rtp-vvc-05


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


_______________________________________________
Audio/Video Transport Core Maintenance
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/avt


_______________________________________________
Audio/Video Transport Core Maintenance
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/avt