Re: [Cellar] AV1 mapping Matroska

Steve Lhomme <slhomme@matroska.org> Wed, 27 June 2018 11:31 UTC

Return-Path: <slhomme@matroska.org>
X-Original-To: cellar@ietfa.amsl.com
Delivered-To: cellar@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 228F81277C8 for <cellar@ietfa.amsl.com>; Wed, 27 Jun 2018 04:31:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.91
X-Spam-Level:
X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qa-FhKpa6qfw for <cellar@ietfa.amsl.com>; Wed, 27 Jun 2018 04:31:21 -0700 (PDT)
Received: from mail-pf0-x22c.google.com (mail-pf0-x22c.google.com [IPv6:2607:f8b0:400e:c00::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6935912777C for <cellar@ietf.org>; Wed, 27 Jun 2018 04:31:21 -0700 (PDT)
Received: by mail-pf0-x22c.google.com with SMTP id a11-v6so832895pff.8 for <cellar@ietf.org>; Wed, 27 Jun 2018 04:31:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=wreFHesrNAk8SDuv4sdFkQsgRWOBb4OGImaHWbc5P88=; b=hBDdqghFtioDs4a4sJoHSXPDjJhx3/TYXBFQ5NNq2byuPTFb1lvfFYVQBO4lAqpzc6 Td3bRs2eHEUAmiQwXlr0n/EW/nlj3aWm85472DAmYmefhW0+7068t05o5TEQlNnKOHHI 83hcxBrFA2Q+9aA1HCi7bLn0tKbyGgr2FwqmxQe8mL1p1aNzGJobt9fDCF84ei2/TJli h2zdnEGgNHyC0d5+uO83MMhYGsnwh/ltskO1gK9EJZqeyhQhvcunLQMWJK4rdpuWatFL iQDdM8r5s4bQ5Y+hRdZhoFx8Bf+NyO6DECDo69A1vUH0PtDc0DNesGCvwmg72MVWdIqX H8Bg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=wreFHesrNAk8SDuv4sdFkQsgRWOBb4OGImaHWbc5P88=; b=IdndJ+yofVV8vVeLCARoIkXH94UWV2u2+fe0/b2QbUPlcoBV+I8zoQzOhx/hO3Smxm kf/FnZtyh9FVNFGwCIcEqb5CPvKh0ConZ9d4qumiRZOcNo2ILsZhehkJz5Nl0TpKB7AW +hVZ7Td7EdhyGjHncGhqOxwaQ0qTnZm3n9VZi8rSy9q9KNfnwhClZbNr4BZ7iltk8afF I0GvGBfmj/YDq0OJqTcir5EAovRG6nu42Etky1wLunBcN/F9Tu5Yfwk8eGYWBW9Bzbdu 38jPvHma3/uLazy/Jfm8j5BUqezbqxydy3+iLhMhnimv2INTql1JYjfqeesycHUvZfoM IJ0A==
X-Gm-Message-State: APt69E1IljeskH03ykeZ6NpaLXdFBKj/ZliAj/CNuz3Ok7B71FIPJRN/ EFKXdikLS8JMl1YMEMfsgPhZPQaLpIVqfeaAS2qOEWeY
X-Google-Smtp-Source: AAOMgpfNr/C6grbmjKWnxnBlvxrlxdFQBAjWOALUBZGg8l0udQSmc34ORx06OmtuiLz/3ukFWc1t+liVOk/SSDCS/78=
X-Received: by 2002:a62:2253:: with SMTP id i80-v6mr5430116pfi.11.1530099080471; Wed, 27 Jun 2018 04:31:20 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Wed, 27 Jun 2018 04:31:19 -0700 (PDT)
In-Reply-To: <CAOXsMFKXUGVwCZEYmm7Z-f9r0qkjCwEPOrnvnsR=9NSxRQmhrg@mail.gmail.com>
References: <CAOXsMFLg0c6CVOhh6tkmX9nagZYX32GQYMJMfZgRQ4Pk+zoP9g@mail.gmail.com> <acb09138-8206-b28a-645a-f967d3bb5ddf@mediaarea.net> <CAOXsMFJm7MW63JV6fuVNYA4R=CzMuB+OREZ+Pb+yJLne5L2SLg@mail.gmail.com> <87lgb0ehbn.fsf@bunkus.org> <CAOXsMFKXUGVwCZEYmm7Z-f9r0qkjCwEPOrnvnsR=9NSxRQmhrg@mail.gmail.com>
From: Steve Lhomme <slhomme@matroska.org>
Date: Wed, 27 Jun 2018 13:31:19 +0200
Message-ID: <CAOXsMF+H5LQkYr5yuSLWRXTe_ZWo4eqztz52H02snZ-oBuopDw@mail.gmail.com>
To: Codec Encoding for LossLess Archiving and Realtime transmission <cellar@ietf.org>
Cc: Frank Galligan <fgalligan@google.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/cellar/xZH_yYRdMNI1ynhegfb8_GGnM2I>
Subject: Re: [Cellar] AV1 mapping Matroska
X-BeenThere: cellar@ietf.org
X-Mailman-Version: 2.1.26
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <cellar.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cellar>, <mailto:cellar-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cellar/>
List-Post: <mailto:cellar@ietf.org>
List-Help: <mailto:cellar-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cellar>, <mailto:cellar-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 27 Jun 2018 11:31:25 -0000

2018-06-27 11:30 GMT+02:00 Steve Lhomme <slhomme@matroska.org>rg>:
> 2018-06-27 10:58 GMT+02:00 Moritz Bunkus <moritz=40bunkus.org@dmarc.ietf.org>rg>:
>> Hey,
>>
>> I've only had time to skim over it. It's definitely understandable.
>>
>> For me the most important thing is to have the mapping in Matroska and WebM
>> be identical. As the reference encoder already produces WebM files, I
>> highly suggest we communicate with their developers regarding the
>> mapping. I cannot find a document for AV1/WebM bindings at the moment,
>> though…
>
> Ah I didn't know that. Now I can see the code. Especially webmenc.c
> with a TODO for Frank Galligan in there.
>
> It way for some Wip In Progress code, because I have been appointed by
> the powers that be at AOM to do the proper/official mapping.
>
> At least this code doesn't write any CodecPrivate which could work,
> except, as for VP9 we wouldn't know the decoding profile just from
> reading the TrackEntry which is a major problem to select the proper
> hardware decoder. And the MP4 mapping also integrates a kind of
> CodecProvate with the Sequence Header OBU in there.
>
>>> I will dig deeper but we may have to tweak CodecDelay to make it signed.
>>
>> You cannot change the type of an existing element. It would potentially
>> break the interpretation of existing files created with the old
>> semantics. Let's take the value 144, which as an unsigned EBML integer
>> would be stored as 0x81 0x90. If you read that back as a signed EBML
>> integer, you'd get the value -128.
>
> Given a codec delay sounds like some time the codec needs when fed
> data before it outputs data. So I have hope it matches and I did the
> math wrong.
>
> In any case I don't know if we should include the flags like MP4 does.
> Because it seems very MP4 specific (and it's a draft and may go away).
> Unlike the rest of their "Codec Private" it should not be fed to the
> decoder (the OBUs we have can be fed as-is to the decoder).

In page 20 of https://aomediacodec.github.io/av1-spec/av1-spec.pdf
there's a pseudo-code on how to handle the
[initial_display_delay_present_flag]. The values to use may differ for
each "operating point" (An operating point specifies which spatial and
temporal layers should be decoded).

It doesn't say clearly in the MP4 draft but the delay (in frame count)
means it should be the maximum delay of all operating points. Which
also means it will add too much delay for some operating points. So
IMO it's not good.

An hypotethical delay in Matroska would also use something similar,
but again it won't be a correct delay in some cases. It's really an
internal codec thing. So I think we should not handle it at all or say
that [initial_display_delay_present_flag] SHOULD be 0.

Let alone the fact that it's expressed in number of frames that may
not all have the same duration.

And finally in MP4 it starts with these bits that are mostly copied
from the Sequence Header OBU that just right after. So IMO it's
unnecessary in that case, adds more data and is prone to errors...

>> Kind regards,
>> mosu
>>
>> _______________________________________________
>> Cellar mailing list
>> Cellar@ietf.org
>> https://www.ietf.org/mailman/listinfo/cellar
>
>
>
> --
> Steve Lhomme
> Matroska association Chairman



-- 
Steve Lhomme
Matroska association Chairman