[AVTCORE] Proposed addition to VP9 RTP payload: advance signaling of inter-layer dependency

Jonathan Lennox <jonathan.lennox@8x8.com> Tue, 09 June 2020 21:27 UTC

Return-Path: <jonathan.lennox@8x8.com>
X-Original-To: avt@ietfa.amsl.com
Delivered-To: avt@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 352793A0915 for <avt@ietfa.amsl.com>; Tue, 9 Jun 2020 14:27:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=8x8.com
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id Ner9M1Fiiypa for <avt@ietfa.amsl.com>; Tue, 9 Jun 2020 14:27:40 -0700 (PDT)
Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com [IPv6:2607:f8b0:4864:20::830]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 799A83A0913 for <avt@ietf.org>; Tue, 9 Jun 2020 14:27:40 -0700 (PDT)
Received: by mail-qt1-x830.google.com with SMTP id d27so161125qtg.4 for <avt@ietf.org>; Tue, 09 Jun 2020 14:27:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=8x8.com; s=googlemail; h=from:mime-version:subject:message-id:date:cc:to; bh=EQX5b0oQuUOFJHh11IyTn+mj2mwLiM/r8x6ybnJ6i+k=; b=Payd52bJzdW/og6AJ4SqhqC8w+HH/5j4pY0Mn3LnLGwxS8kkHUFe7lP5MebOAaNvck s+9F7307FP6EvdzNXr3AQpBOnyJlKKP21BHL4f6r13jlU8nKzJI0VSVss8p1OC5yl5mp Xg50yCxOOrytrE4dNK0GbCNi/QRL7jKSvH/DA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:cc:to; bh=EQX5b0oQuUOFJHh11IyTn+mj2mwLiM/r8x6ybnJ6i+k=; b=QxTLu+kXiFkCpr4hlhwwX5Yv2YKJXhrX82MN9y7/TCHRQxOS2FRdtyln1qAzVsIeIj rGtJPD0f5/+FX1wP7id3YdH/tdRJ0xgPBGJYdBmyaOMXAuH+MBozdGQlpohhaIxydRLu FQArxOoAhgJovGuwnht5XIEE4ddVtjCfNzNEGWuHkGj4lb9895mLjfpOvAFWYWXOfRuo 12vA1oLpdKihlSb395qtgh2ZQtHCljA1jKe/aCdB3NqPawWdVKDjpuTIiiWIzJvxJ/Gz 8pwdIObDh56E/l1xk6VGfSQaIA7C0rmt4x/tmX32wzHLnQNRRM8knmIKXKutDD5e7vgf hKyQ==
X-Gm-Message-State: AOAM533p9ZKkLjdE7Noe/QwYWwzGsHgCo9QFrGPwkvp2Srj+2YjLfKzw 9buxNIz+jfW1r/c+9M5xZW/K8MS+W+U=
X-Google-Smtp-Source: ABdhPJyi0k5r64NDOkTvQM8ujVok0w6CHDxzz5ocELkKzk8fLCNjQJR410w9/2xgkvw9cdmeHeq/4A==
X-Received: by 2002:ac8:e0a:: with SMTP id a10mr26689840qti.244.1591738058648; Tue, 09 Jun 2020 14:27:38 -0700 (PDT)
Received: from ?IPv6:2601:86:101:74d0:fc19:b073:3447:4041? ([2601:86:101:74d0:fc19:b073:3447:4041]) by smtp.gmail.com with ESMTPSA id l184sm9694256qkf.84.2020. (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Jun 2020 14:27:38 -0700 (PDT)
From: Jonathan Lennox <jonathan.lennox@8x8.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_B7D6DAC9-CFEA-4934-9B96-93C7FBB0FE20"
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.14\))
Message-Id: <664C0F98-7017-4F3B-8D74-A3D88EE6671B@8x8.com>
Date: Tue, 9 Jun 2020 17:27:37 -0400
Cc: Sergey Silkin <ssilkin@google.com>
To: avt@ietf.org
X-Mailer: Apple Mail (2.3445.104.14)
Archived-At: <https://mailarchive.ietf.org/arch/msg/avt/78vIASd5dE_CtIxcZ_vW1PLrKng>
Subject: [AVTCORE] Proposed addition to VP9 RTP payload: advance signaling of inter-layer dependency
X-BeenThere: avt@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Audio/Video Transport Core Maintenance <avt.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/avt>, <mailto:avt-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/avt/>
List-Post: <mailto:avt@ietf.org>
List-Help: <mailto:avt-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/avt>, <mailto:avt-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 09 Jun 2020 21:27:42 -0000

Hi, all —

Sergey Silkin of Google has proposed an addition to the VP9 RTP payload.

The problem he’s observed is that when a lower spatial layer frame of a picture is not used for prediction by a higher layer, in the current spec it’s only possible to know this when the higher-layer frame is received (based on its “D” bit).  This means that if a middlebox or decoder would need to buffer the packets of the lower-layer frame in order to decide reliably whether or not it needs to be relayed or decoded.

Sergey’s proposal (with some wording tweaks by me) is to use the currently-reserved bit in the first byte of the VP9 payload header as a “Z” byte, defined as:

          Z: Not a reference frame for upper spatial
          layers. If set to 1, indicates that frames with higher
          spatial layers SID+1 of the current and following pictures
          do not depend on the current spatial layer SID frame.  This
          enables a decoder which is targeting a higher spatial layer
          to know that it can safely discard this packet's frame
          without processing it, without having to wait for the "D"
          bit in the higher-layer frame (see below).

This has been implemented (and is being used) in Chrome, which is by far the dominant VP9 RTP payload implementation.

As this solves a real problem and is being used in running code, my inclination is to add it to the spec. Are there any objections?

The pull request to add this to the spec is at https://github.com/juberti/draughts/pull/117 <https://github.com/juberti/draughts/pull/117> .

Jonathan Lennox
(Writing as document editor)