Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

"Wang, Ye-Kui" <yekuiw@qti.qualcomm.com> Wed, 28 August 2013 01:36 UTC

Return-Path: <yekuiw@qti.qualcomm.com>
X-Original-To: avt@ietfa.amsl.com
Delivered-To: avt@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3563711E810F; Tue, 27 Aug 2013 18:36:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.598
X-Spam-Level:
X-Spam-Status: No, score=-102.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7dzONskcQer1; Tue, 27 Aug 2013 18:36:21 -0700 (PDT)
Received: from sabertooth02.qualcomm.com (sabertooth02.qualcomm.com [65.197.215.38]) by ietfa.amsl.com (Postfix) with ESMTP id 733F011E80F6; Tue, 27 Aug 2013 18:36:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1377653781; x=1409189781; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=UPKwrPw2jtO0RxxUtEZpC1yOb4GNbcEqmbXFYfWerJM=; b=vlyKitEclIAwfcUWbQ1jfGoxiv4HI+2AT0t/6zbGO7lwtqh9GqJ/4Pyx +cdzhRqX5ZqDdC+/ine6NGYvGs50iBMZJF9ODhXLyf7MPCYtYbb5o7mxs OSyxU57UfnwoncWdZIUPj8cKv9hrIon8FUua8/dmL+8I3mnCQt444ZmX3 o=;
X-IronPort-AV: E=McAfee;i="5400,1158,7180"; a="50444729"
Received: from ironmsg01-lv.qualcomm.com ([10.47.202.180]) by sabertooth02.qualcomm.com with ESMTP; 27 Aug 2013 18:36:20 -0700
X-IronPort-AV: E=McAfee;i="5400,1158,7180"; a="18566149"
Received: from nasanexhc11.na.qualcomm.com ([172.30.39.6]) by ironmsg01-lv.qualcomm.com with ESMTP/TLS/RC4-SHA; 27 Aug 2013 18:36:20 -0700
Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.196]) by nasanexhc11.na.qualcomm.com ([172.30.39.6]) with mapi id 14.03.0146.002; Tue, 27 Aug 2013 18:36:20 -0700
From: "Wang, Ye-Kui" <yekuiw@qti.qualcomm.com>
To: "Mo Zanaty (mzanaty)" <mzanaty@cisco.com>
Thread-Topic: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit
Thread-Index: AQHOo3jm1moEaDpZa0yx3vgEUVhvCZmp1oPg
Date: Wed, 28 Aug 2013 01:36:20 +0000
Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A8338512D57@nasanexd02f.na.qualcomm.com>
References: <C1F36850-2B72-4A98-97B7-8847C9C90CB0@live555.com> <3879D71E758A7E4AA99A35DD8D41D3D91D4C7AA7@xmb-rcd-x14.cisco.com> <A0D9B971-CDE0-4AF7-9A70-C280514AEF10@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83384D4C47@nasanexd02f.na.qualcomm.com> <5D781561-69CD-4E0D-9377-B4B26036E691@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83384D5FB3@nasanexd02f.na.qualcomm.com> <69F25DB2-6926-4ACA-935D-90376A2A7BFA@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83384D690F@nasanexd02f.na.qualcomm.com> <A6FDCB43-F221-4BE0-BEDA-1F1E5394DDEC@live555.com> <3879D71E758A7E4AA99A35DD8D41D3D91D4CC65C@xmb-rcd-x14.cisco.com> <F5C111A5-64A6-48AB-BBA8-8812EE49F7ED@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83384D6FD4@nasanexd02f.na.qualcomm.com> <B995CF20-66A4-47B9-B141-B650290F8DC7@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83385120E1@nasanexd02f.na.qualcomm.com> <75CC2B0E-28F2-452C-9CE5-060DB8DFEE6B@cisco.com>
In-Reply-To: <75CC2B0E-28F2-452C-9CE5-060DB8DFEE6B@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [172.30.48.1]
Content-Type: multipart/alternative; boundary="_000_8BA7D4CEACFFE04BA2D902BF11719A8338512D57nasanexd02fnaqu_"
MIME-Version: 1.0
Cc: Ross Finlayson <finlayson@live555.com>, "avt@ietf.org" <avt@ietf.org>, "payload@ietf.org" <payload@ietf.org>
Subject: Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit
X-BeenThere: avt@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Audio/Video Transport Core Maintenance <avt.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/avt>, <mailto:avt-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/avt>
List-Post: <mailto:avt@ietf.org>
List-Help: <mailto:avt-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/avt>, <mailto:avt-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 28 Aug 2013 01:36:45 -0000

Hi Mo,

"The content of a NAL unit" can tell whether the NAL unit is the last slice of a picture, but the last NAL unit of an AU can also be a non-VCL NAL unit. Considering this, "the content of a NAL unit" does not tell whether a NAL unit is the last NAL unit of an AU.

BR, YK

From: Mo Zanaty (mzanaty) [mailto:mzanaty@cisco.com]
Sent: Tuesday, August 27, 2013 3:58 PM
To: Wang, Ye-Kui
Cc: Ross Finlayson; avt@ietf.org; payload@ietf.org
Subject: Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

I suggest replacing "The content of a NAL unit" with "The NAL unit header". The content does reveal end of access unit boundaries if you're willing to parse/decode it all.

Mo


On Aug 27, 2013, at 12:51 AM, "Wang, Ye-Kui" <yekuiw@qti.qualcomm.com<mailto:yekuiw@qti.qualcomm.com>> wrote:
Hi Ross,

Sorry for a slow response. I started to edit this draft today, and have included the following note after the semantics of the M bit:

Informative note: The content of a NAL unit does not tell whether or not the NAL unit is the last NAL unit, in decoding order, of an access unit.  An RTP sender implementation may obtain this information from the video encoder.  If, however, the implementation cannot obtain this information directly from the encoder, e.g., when the stream was pre-encoded, and also there is no timestamp allocated for each NAL unit, then the sender implementation can inspect subsequent NAL units in decoding order to determine whether or not the NAL unit is the last NAL unit of an access unit as follows.  A NAL unit naluX is the last NAL unit of an access unit if the next VCL NAL unit naluY in decoding order has the high-order bit of the first byte after its NAL unit header equal to 1, and all NAL units between naluX and naluY, when present, have nal_unit_type in the range of 32 to 35, inclusive, equal to 39, in the range of 41 to 44, inclusive, or in the range of 48 to 55, inclusive.

Would you be fine with this wording?

BR, YK


From: avt-bounces@ietf.org<mailto:avt-bounces@ietf.org> [mailto:avt-bounces@ietf.org] On Behalf Of Ross Finlayson
Sent: Monday, August 12, 2013 12:25 PM
To: avt@ietf.org<mailto:avt@ietf.org>; payload@ietf.org<mailto:payload@ietf.org>
Subject: Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

You just need one more little step to get everything right here, instead of thinking of compromising the implementation ☺

That is, read subclause 7.4.2.4.4 of the HEVC spec. The most relevant paragraphs are copied below for convenience (the first paragraph itself contains the answer to your question, but the second paragraph may also provide helpful information)

Thanks for the info; that was useful.

Here, then is the updated text of a paragraph - to be added after the existing 'M bit' text - that clarifies when/how to set it:
----------
Unfortunately the contents of a NAL unit, alone, do not tell a RTP sender implementation whether or not the NAL unit ends an access unit.  Instead, the implementation may be able to obtain this information separately, from the encoder.  If, however, the implementation cannot obtain this information directly from the encoder (e.g., because the implementation is sending data that consists solely of a sequence of pre-encoded NAL units), then it must instead inspect subsequent NAL units, to determine whether or not the NAL unit ends an access unit.  If the NAL units have already been timestamped, then these timestamps can be used to determine the access unit boundaries.  Otherwise, the access unit boundaries (and times) must be determined by inspecting the NAL units' contents - specifically, the "nal_unit_type" and (for VCL NAL units) the "first_slice_segment_in_pic_flag".  The following rule can be used:
           A NAL unit (X) ends an access unit if the next-occurring VCL NAL unit (Y) has the high-order bit of the first byte after its NAL unit header equal to 1, and all NAL units, if any, between (X) and (Y) have a "nal_unit_type" in one of the following ranges: [32,35], 39, [41,44], or [48,55].
----------

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

_______________________________________________
Audio/Video Transport Core Maintenance
avt@ietf.org<mailto:avt@ietf.org>
https://www.ietf.org/mailman/listinfo/avt