Re: [avtext] Frame marking & VP9 SVC

Bernard Aboba <bernard.aboba@gmail.com> Wed, 05 July 2017 16:52 UTC

Return-Path: <bernard.aboba@gmail.com>
X-Original-To: avtext@ietfa.amsl.com
Delivered-To: avtext@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99FFA131D7F for <avtext@ietfa.amsl.com>; Wed, 5 Jul 2017 09:52:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Level:
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 84DXVCsbkYxj for <avtext@ietfa.amsl.com>; Wed, 5 Jul 2017 09:52:36 -0700 (PDT)
Received: from mail-vk0-x234.google.com (mail-vk0-x234.google.com [IPv6:2607:f8b0:400c:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AED03131D7B for <avtext@ietf.org>; Wed, 5 Jul 2017 09:52:35 -0700 (PDT)
Received: by mail-vk0-x234.google.com with SMTP id y70so127913977vky.3 for <avtext@ietf.org>; Wed, 05 Jul 2017 09:52:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=thKnWCgJTuuSOfwD1OnKoFSKR59Q4zVCV12l+4Qzdf0=; b=GAIeCfxDaT+aQG5kYTb0JH91XRl5FUyRykL7NZNzvJ91a2No0OgELFjihzlH9h3iD6 YX8eJZrrqgSlwat7LtE31DFj6wx+Hw5OV/8YrRh+ITcSEubqvIDxTbrtIoqcscPWe1pQ cgmjAAKQGiSqAwPWrvarUc2hwRCL7abWmB83LQ2E7o2+rbn7+sjuVmgcMZmOESdwPhrt a7rRiyokAppomnsXxya8IsVayLeYw2Ea0d9pG8jgKzFmTJkD0owvsucR5Q0OMzsGJ1vK rhfZEtvBTHyZ6hO3moZQFs2sGZnM17GpBk8NHXQD889PZ2NcxyxiudG521ytIhTWuO5Y 2d5w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=thKnWCgJTuuSOfwD1OnKoFSKR59Q4zVCV12l+4Qzdf0=; b=d7lG+COp6ASZqOIYHAUU/e1nJZc1e/jlD0H+/4PIn/PweeJ+RFHrwOwPYr7USH8bd8 73gNx5cfu7fgil2WggiHgqI3KwF/RQsg/qzMZfCqLHZSJirqzF58BwrIJXtje/9XhGYO ytTZ5MfALIggamKOHVFNmumJOfyYZVG7qnGNo+WC/0v90fGb/FJOjW7LadXzxbow2y3v 4LY4nF5LbeJEgqeXGS3w85XB9rlwCEHGprhCTBRkm0Bm2Xjd+F914jIu722uEBDnm6MN Bm3BnWzCuspjKV37JplgY2hlIkYqrxBj7ORYmUlW4TDcs7nVy8rHo/2nsWQo/n7bjt2H 89Nw==
X-Gm-Message-State: AKS2vOw/AV7UhCvpp1iwSyBF5e+A9ONfhWAPi/5tFiwt90SzU0EqmA/8 3OIsLtfII0El/N8DTNG60Fa1ccp7Cw==
X-Received: by 10.31.209.134 with SMTP id i128mr24899757vkg.125.1499273554607; Wed, 05 Jul 2017 09:52:34 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.74.202 with HTTP; Wed, 5 Jul 2017 09:52:14 -0700 (PDT)
In-Reply-To: <A3A2CACA-FA2F-4BA2-8209-0F5B556EA732@vidyo.com>
References: <ebdc7854-b390-d0e4-cfd1-d7df9c65aba4@gmail.com> <D4FFF329.6BA66%mzanaty@cisco.com> <0b5c72c6-4f59-daba-f193-282ea10d1f07@gmail.com> <D513D706.6C7F3%mzanaty@cisco.com> <00166BCB-6452-4D72-B5DF-5A456B2304EF@vidyo.com> <9a4994d3-f033-e1da-7884-a55c31789c59@gmail.com> <30944D17-BEB9-4EC6-A97C-0700567563FB@vidyo.com> <918659db-1742-4dd4-6fa3-7d0797aa4582@gmail.com> <50b624f9-4ce7-099e-1730-1b3a0d986e69@cosmosoftware.io> <CAOW+2du7K1Cz2PzG3dddHajW3tS+kkS3BGs_xP_suymssH2pwA@mail.gmail.com> <a58018e7-149f-810a-a0d6-944cb8dbc0e5@cosmosoftware.io> <A3A2CACA-FA2F-4BA2-8209-0F5B556EA732@vidyo.com>
From: Bernard Aboba <bernard.aboba@gmail.com>
Date: Wed, 05 Jul 2017 09:52:14 -0700
Message-ID: <CAOW+2dtDkCVnxkjRHZ9YzaU+-A9fm-iCXvf5yYOZMMdgbP4K-A@mail.gmail.com>
To: Jonathan Lennox <jonathan@vidyo.com>
Cc: Sergio Garcia Murillo <sergio.garcia.murillo@cosmosoftware.io>, "Mo Zanaty (mzanaty)" <mzanaty@cisco.com>, "avtext@ietf.org" <avtext@ietf.org>, Alexandre GOUAILLARD <alex.gouaillard@cosmosoftware.io>
Content-Type: multipart/alternative; boundary="001a114e3a90a4c389055394d37b"
Archived-At: <https://mailarchive.ietf.org/arch/msg/avtext/6k6ThBGXasifNQSd7jEP44p0Jwg>
Subject: Re: [avtext] Frame marking & VP9 SVC
X-BeenThere: avtext@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Audio/Video Transport Extensions working group discussion list <avtext.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/avtext>, <mailto:avtext-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/avtext/>
List-Post: <mailto:avtext@ietf.org>
List-Help: <mailto:avtext-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/avtext>, <mailto:avtext-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 05 Jul 2017 16:52:39 -0000

Jonathan said:

"the “don’t use it with weird structures” requirement will make that
sufficient,"

[BA] There are structures where the B bit will not provide enough
information to indicate that upscaling is possible (e.g. B bit 0 would
indicate that the frame doesn't depend solely on the base layer, but P and
U bits would indicate upscaling is possible).

A "weird structures requirement" is somewhat unnatural. If the structures
are not illegal in the bitstream specification, encoders can generate them,
and decoders will be required to be able to decode them.


This can generate a mismatch between what SFUs are built to handle and what
endpoints do.  If the payload is encrypted, the SFU won't necessarily be
able to detect a "weird" structure and it can't act based on the P and U
bits, only the B bit. Even if the payload is unencrypted, the SFU may wish
to function based on the frame marking header alone, so as to avoid parsing
the payload for the P and U bits.





On Wed, Jul 5, 2017 at 8:56 AM, Jonathan Lennox <jonathan@vidyo.com> wrote:

> Sergio — the updated VP9 payload draft I submitted Monday includes a
> section describing its usage with frame marking.
>
> Let me know what you think?  It changes the S and T fields to reflect the
> latest VP9 and frame marking drafts, but it does not include the P or U
> bits.
>
> I think the intention is that frame marking’s B bit will provide a
> coarser-grain version of what P and U do (and the “don’t use it with weird
> structures” requirement will make that sufficient, though that may need to
> be made more specific).  What do you think?
>
> On Jun 28, 2017, at 10:23 AM, Sergio Garcia Murillo <
> sergio.garcia.murillo@cosmosoftware.io> wrote:
>
> Yes, at least in VP9 SVC you can define how many temporal and how many
> spatial layers do you want simultaneously in the encoding. Note also, that
> VP9 supports quality layers as spatial layers without any resolution
> change, so the term "spatial layer" is used to represent both spatial and
> quality layers.
> BR
> Sergio
> On 28/06/2017 16:18, Bernard Aboba wrote:
>
> Sergio said:  "So, shouldn't the LID reference to the spatial layer ID
> only and omit the quality layer id completely? Also, the spatial layer id
> is 3 bits on that draft."
>
> [BA] Question:  In VP9 (or AV1, for that matter) is it possible to use
> both spatial and quality scalability in the same encoding?
>
> On Wed, Jun 28, 2017 at 5:35 AM, Sergio Garcia Murillo <
> sergio.garcia.murillo@cosmosoftware.io> wrote:
>
>> Hi all again,
>>
>> Did you had a chance to review the changes to add the VP9 SVC spatial
>> layer information into the LID of the frame marking extension? (at
>> https://github.com/murillo128/draft-berger-avtext-framemarki
>> ng/commit/35cdb229108251f12bc1cdf77909df83a51d3994)
>>
>> Also, FYI, I have submitted a CL to libwebrtc to support framemarking
>> extension. Currently it supports VP8, H264 and non-SVC VP9, but I would
>> love to tackle VP9 SVC as soon as we get an agreement on the changes to the
>> LID:
>>
>> https://codereview.webrtc.org/2954503002/
>>
>> Best regards
>> Sergio
>>
>>
>> On 24/04/2017 11:28, Sergio Garcia Murillo wrote:
>>
>>> On 18/04/2017 17:40, Jonathan Lennox wrote:
>>>
>>>> One of the goals of Frame Marking is that an SFU can be agnostic to the
>>>> details of the layering, and just treat layer IDs as opaque layer
>>>> identifiers.  Putting non-layer bits into the LID field would break this
>>>> goal.
>>>>
>>>> At the IETF meeting in Chicago, both Bernard Aboba and I thought that
>>>> frame marking might need enhancements in order for spatial scalability to
>>>> be fully supported. I’m adding him to the Cc, as well as my co-authors on
>>>> the VP9 payload.
>>>>
>>>> Sergio, if you want to send a pull request to contribute text for the
>>>> VP9 spec (which would be welcome), it’s in Github at
>>>> https://github.com/juberti/draughts .
>>>>
>>> Currently the VP9 LID mapping is described on the frame marking draft,
>>> and not in the VP9 draft. We could move the full description from one text
>>> to another, but at least for me, it was easier to provide a small patch for
>>> current draft to address my comments:
>>>
>>> https://github.com/murillo128/draft-berger-avtext-framemarki
>>> ng/commit/35cdb229108251f12bc1cdf77909df83a51d3994
>>>
>>> I have not found the .xml in any github repo, so just downloaded the
>>> latest draft and applied the changes on top of it.
>>>
>>> Please, let me know what are your views about it.
>>>
>>
>>
>>
>>
>>
>
>
>