Re: [Cellar] AV1 mapping Matroska

Jerome Martinez <jerome@mediaarea.net> Mon, 02 July 2018 11:14 UTC

Return-Path: <jerome@mediaarea.net>
X-Original-To: cellar@ietfa.amsl.com
Delivered-To: cellar@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B77A130EE3 for <cellar@ietfa.amsl.com>; Mon, 2 Jul 2018 04:14:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xbzGQAdLQB8Q for <cellar@ietfa.amsl.com>; Mon, 2 Jul 2018 04:14:07 -0700 (PDT)
Received: from 9.mo178.mail-out.ovh.net (9.mo178.mail-out.ovh.net [46.105.75.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C9A0D1292AD for <cellar@ietf.org>; Mon, 2 Jul 2018 04:14:07 -0700 (PDT)
Received: from player760.ha.ovh.net (unknown [10.109.108.51]) by mo178.mail-out.ovh.net (Postfix) with ESMTP id 299491A5AA for <cellar@ietf.org>; Mon, 2 Jul 2018 13:14:05 +0200 (CEST)
Received: from [192.168.2.120] (p3EE2DF2E.dip0.t-ipconnect.de [62.226.223.46]) (Authenticated sender: jerome@mediaarea.net) by player760.ha.ovh.net (Postfix) with ESMTPSA id E76FE20088 for <cellar@ietf.org>; Mon, 2 Jul 2018 13:14:03 +0200 (CEST)
To: cellar@ietf.org
References: <f603a9f5-d7dc-6640-1f53-f4d7b62a788e@googlemail.com> <CAOXsMFJJbHbh3=Gnu7Uag4Skq5+=jOimFNPxuicgDiVdbven4w@mail.gmail.com> <dada3690-c8da-05db-4ddb-32476c1170f5@xiph.org> <CAOXsMF+CPW6VJsFaeahuL00hnvq5dUO-gHE3sYXPr5iVy8s1aQ@mail.gmail.com> <87efgmdqou.fsf@bunkus.org> <20180702103455.GQ4839@michaelspb> <CAOXsMF+Q-RzwapNOamqc9G9XttnzMXyUuSY8_HFAA+h+4YsMJA@mail.gmail.com>
From: Jerome Martinez <jerome@mediaarea.net>
Message-ID: <36a0dc7e-8579-d99c-4beb-e8f4ce019eff@mediaarea.net>
Date: Mon, 2 Jul 2018 13:14:02 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0
MIME-Version: 1.0
In-Reply-To: <CAOXsMF+Q-RzwapNOamqc9G9XttnzMXyUuSY8_HFAA+h+4YsMJA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Ovh-Tracer-Id: 647673925405184145
X-VR-SPAMSTATE: OK
X-VR-SPAMSCORE: 0
X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtiedrvdejgdefiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecu
Archived-At: <https://mailarchive.ietf.org/arch/msg/cellar/Jo5wr0vKxy6UZktjPhkfOugCmPU>
Subject: Re: [Cellar] AV1 mapping Matroska
X-BeenThere: cellar@ietf.org
X-Mailman-Version: 2.1.26
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <cellar.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cellar>, <mailto:cellar-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cellar/>
List-Post: <mailto:cellar@ietf.org>
List-Help: <mailto:cellar-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cellar>, <mailto:cellar-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Jul 2018 11:14:11 -0000

On 02/07/2018 12:51, Steve Lhomme wrote:
> 2018-07-02 12:34 GMT+02:00 Michael Niedermayer <michael@niedermayer.cc>cc>:
>> Hi
>>
>> On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote:
>>> Hey,
>>>
>>>> Looking at the term "coded video sequence" (CVS) in other codecs (H.264
>>>> and H.265) it seems it's a common term. And for those codec one Segment
>>>> correspond to one CVS, with the parameters of that CVS stored in the
>>>> CodecPrivate (SPS + PPS for H.264 for example). So we should probably go
>>>> in that simple way.
>>> I have quite a lot of h.264 samples here, mainly M2TS from DVB, where
>>> SPS/PPS change mid-stream, often multiple times. For such files the first
>>> occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key frames
>>> are still prefixed with the currently active SPS/PPS — because that's the
>>> only way to signal that stuff has actually changed.
>> IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which the
>> global header applies to, not just the first.

Not possible with e.g. real time streaming (you don't know in advance 
the next SPS/PPS)

>> This way an application can know all resolutions, aspect ratios and so on
>> from just the header without the need to scan through all random access
>> points.
> My understanding is that it's how it's done in MP4:
>
> "Each AVC sample entry, which contains the AVC video stream decoder
> specific information, includes a group of SPSs and PPSs. This group of
> parameter sets functions much like a codebook. Each parameter set has
> an identifier, and each slice references the parameter set it was
> coded against using the parameter set's identifier."
>
> I have to check but I don't think Sequence Headers OBUs have
> identifiers like that in AV1.

Not in AV1 (I just updated my own code for AV1  1.0.0 spec, still not 
there).
But it actually does not answer the issue because in the example there 
is usually still only the "set id" of 0 (the new SPS replaces the old 
one, not a new number).

Actually IIRC in HEVC (same for AVC) there are explicitly 2 different 
codec identifiers:
- "hvc1" is for *all* SPS/PPS in the global header (CodecPrivate), and 
there is no SPS/PPS in the stream itself
- "hev1" is for *no* SPS/PPS in the global header (CodecPrivate), and 
key frames are prefixed with the currently active SPS/PPS
(I personally don't see the need of 2 different identifiers, as it quick 
to see if SPS/PPS is in the file, but there is surely a reason)

The issue is not only for AV1, but for all formats permitting to have 
the init in the stream itself (AVC, HEVC, AV1...).

IMO we should not restrict to only one possibility in Matroska spec, as 
they are for different needs (non real time vs real time), whatever is 
the format (AVC, HEVC, AV1...).



> By the way, this is one mapping for V_AV1. There could be other AV1
> mapping in Matroska where there is no CodecPrivate and there could be
> multiple coded video sequences in it. But it would require the use of
> CodecState and CueCodecState which is currently not an option for
> WebM. So for now I will concentrate on a single coded video sequence.
> And if needed in the future we can do a different one.

If WebM decides that real time AV1 is for them, they may decide to 
discard CodecPrivate usage and put sequence header in the "Block" (as 
they currently do with up to date AOM build)
So restriction will not be there in practice.

Jérôme