Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit

Ross Finlayson <finlayson@live555.com> Sat, 10 August 2013 04:55 UTC

Return-Path: <finlayson@live555.com>
X-Original-To: avt@ietfa.amsl.com
Delivered-To: avt@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2978D11E8131; Fri, 9 Aug 2013 21:55:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.598
X-Spam-Level:
X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[AWL=-0.000, BAYES_00=-2.599, HTML_MESSAGE=0.001]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qg3n6t+LvOuc; Fri, 9 Aug 2013 21:55:21 -0700 (PDT)
Received: from ns.live555.com (ns.live555.com [4.79.217.242]) by ietfa.amsl.com (Postfix) with ESMTP id 6677611E80F6; Fri, 9 Aug 2013 21:49:53 -0700 (PDT)
Received: from [127.0.0.1] (localhost.live555.com [127.0.0.1]) by ns.live555.com (8.14.4/8.14.4) with ESMTP id r7A4nkJF057217; Fri, 9 Aug 2013 21:49:50 -0700 (PDT) (envelope-from finlayson@live555.com)
From: Ross Finlayson <finlayson@live555.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_41DC87E7-EDDD-43ED-AB0D-08FD287188A7"
Message-Id: <F5C111A5-64A6-48AB-BBA8-8812EE49F7ED@live555.com>
Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\))
Date: Fri, 9 Aug 2013 21:49:45 -0700
References: <C1F36850-2B72-4A98-97B7-8847C9C90CB0@live555.com> <3879D71E758A7E4AA99A35DD8D41D3D91D4C7AA7@xmb-rcd-x14.cisco.com> <A0D9B971-CDE0-4AF7-9A70-C280514AEF10@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83384D4C47@nasanexd02f.na.qualcomm.com> <5D781561-69CD-4E0D-9377-B4B26036E691@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83384D5FB3@nasanexd02f.na.qualcomm.com> <69F25DB2-6926-4ACA-935D-90376A2A7BFA@live555.com> <8BA7D4CEACFFE04BA2D902BF11719A83384D690F@nasanexd02f.na.qualcomm.com> <A6FDCB43-F221-4BE0-BEDA-1F1E5394DDEC@live555.com> <3879D71E758A7E4AA99A35DD8D41D3D91D4CC65C@xmb-rcd-x14.cisco.com>
To: "avt@ietf.org" <avt@ietf.org>, "payload@ietf.org" <payload@ietf.org>
In-Reply-To: <3879D71E758A7E4AA99A35DD8D41D3D91D4CC65C@xmb-rcd-x14.cisco.com>
X-Mailer: Apple Mail (2.1508)
Subject: Re: [AVTCORE] [payload] Clarifying the H.265 RTP payload format specification's text on when to set the RTP "M" bit
X-BeenThere: avt@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Audio/Video Transport Core Maintenance <avt.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/avt>, <mailto:avt-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/avt>
List-Post: <mailto:avt@ietf.org>
List-Help: <mailto:avt-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/avt>, <mailto:avt-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Aug 2013 04:55:27 -0000

> If you are able to set the RTP timestamp reliably, then you must already have some sort of frame boundary identification.

Yes, we have a reliable way of determining when *VCL* NAL units cross a frame boundary: Look at the first slice header bit.  The one remaining problem (that I identified at the end of my last message) is: How to classify any *non-VCL* NAL units that appear between one VCL NAL unit and another VCL unit that has the first slice header bit.  Which of these belong to the last frame, and which belong to the next frame.

But perhaps this doesn't really matter?  Perhaps it's OK (from the point of view of receivers that check and try to optimize for the 'M' bit) if we always set the 'M' bit for the last VCL NAL unit of a frame, even if some subsequent non-VCL NAL units (appearing before the first VCL NAL unit of the next frame) also belong logically to the current frame??


> But you should realize that, in doing this, you have defeated the marker bit optimization. You have just shifted the latency from the receiver to the sender

No, not necessarily.  If the encoded video comes from a file (i.e., pre-recorded), then the latency involved in requiring the sender to examine the 'next' frame will usually be negligible.  But in any case, the situation we're talking about here (senders working from a sequence of pre-encoded NAL units) is one that usually occurs only in non-interactive environments, for which latency isn't crucial.


> So there is not much benefit in making the marker bit reliable in this case, and there is even the risk that it will add unnecessary latency.

That's certainly another possibility, I suppose: Recommend that if a sending implementation cannot determine whether or not a NAL unit ends an 'access unit' (frame) without having to scan subsequent NAL units, then it should not set the 'M' bit at all.  I don't like this, though (especially because it adds receiver latency without necessarily reducing sender latency (note above)).  But it's a possibility to consider...

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/