Re: [Cellar] AV1 mapping Matroska

Steve Lhomme <slhomme@matroska.org> Sat, 30 June 2018 07:44 UTC

Return-Path: <slhomme@matroska.org>
X-Original-To: cellar@ietfa.amsl.com
Delivered-To: cellar@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB3EE13102E for <cellar@ietfa.amsl.com>; Sat, 30 Jun 2018 00:44:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.909
X-Spam-Level:
X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ml-1MM9mmQq4 for <cellar@ietfa.amsl.com>; Sat, 30 Jun 2018 00:44:14 -0700 (PDT)
Received: from mail-pl0-x235.google.com (mail-pl0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7659213102C for <cellar@ietf.org>; Sat, 30 Jun 2018 00:44:14 -0700 (PDT)
Received: by mail-pl0-x235.google.com with SMTP id m16-v6so5524304pls.11 for <cellar@ietf.org>; Sat, 30 Jun 2018 00:44:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=378RXwcZQEjPt7B5sCE6hV2ygqx2PlicNqU+zLFUdwg=; b=y0X72CtK1XoJcQgjHN6zJxrifn+hGKcU10RQd6O6hAThJh8pVDnu5ROEeMO1VCbo7G FjVm3IA8TGF8Kh8xAhAHIYTADHz7bePy7Pt7SuuNlxYHUdak+Q53LadtJDCp2cKiKWN5 7M7qTKNs7AimFV2bsVuibAsbF2R5ix9eSZfjYm4uHMUsD/wux5vHSRPa71gAqf9Dzwjl 03DilrK9nJiU6tIXilmuKKxVtoxSlR7KTMTdOKXMC4TMBfluSBYrgHTHX4mcuH/w0fWc CyDNSaCnnGkppQo6kQg1DIDbmH9ssG/NbQYc//G6UNJnIPNKKvVHRJ6FiA0kvGfZkzX2 6gDg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=378RXwcZQEjPt7B5sCE6hV2ygqx2PlicNqU+zLFUdwg=; b=bDBxhOvHHrYL73zXKC113z9rK69XkfV7XcUetMyI+7D4/1rG8mnAppV4aKzn64184I F0rBvwRw7/7wZtOkI41K3TBmgm6lP+JthsGIxvaVsmD9D3YMHan1B763Gphu5fpLxlBf xswywm5MpTgjuQVc+Mzh0F1FeWgjDZL6l0E4Xn81uUz5f/NZy03R5SA3ApG+MlO6I27e 7OfdEP4zP9UMwk2rfTOxKV6jBh6CYOkO0aNvCIqsvzXJShyvyMP45UtB17olv8SW2dAJ 9RD0ALOt4rf0EyNOE4uonNPQjIXCb6pKT+JGIQqMEwrDqwHUc7ArOJ7VsRB6jL08u4ra shWQ==
X-Gm-Message-State: APt69E0X85M3L3V8/poOVUufYjfuHYCon7DfracPfleWK0MAZ07i9rAd 7z2sh8DAWa9sJ346r52UhEaVgvatGNUNKFwu8i7Ofw==
X-Google-Smtp-Source: ADUXVKJPUUOMBMMQ5Up/PYGiHtgEJMpeFBvxLjgb8VRK8fc5OEqbraFUU4jW+c/GMBgfm7Xo2rCzPJhcnEORGX63+Xw=
X-Received: by 2002:a17:902:bd95:: with SMTP id q21-v6mr17729422pls.237.1530344653757; Sat, 30 Jun 2018 00:44:13 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Sat, 30 Jun 2018 00:44:13 -0700 (PDT)
In-Reply-To: <f603a9f5-d7dc-6640-1f53-f4d7b62a788e@googlemail.com>
References: <f603a9f5-d7dc-6640-1f53-f4d7b62a788e@googlemail.com>
From: Steve Lhomme <slhomme@matroska.org>
Date: Sat, 30 Jun 2018 09:44:13 +0200
Message-ID: <CAOXsMFJJbHbh3=Gnu7Uag4Skq5+=jOimFNPxuicgDiVdbven4w@mail.gmail.com>
To: Andreas Rheinhardt <andreas.rheinhardt@googlemail.com>
Cc: Codec Encoding for LossLess Archiving and Realtime transmission <cellar@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/cellar/36bfEllZeMfcVd6gtUkIfHltZe8>
Subject: Re: [Cellar] AV1 mapping Matroska
X-BeenThere: cellar@ietf.org
X-Mailman-Version: 2.1.26
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <cellar.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cellar>, <mailto:cellar-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cellar/>
List-Post: <mailto:cellar@ietf.org>
List-Help: <mailto:cellar-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cellar>, <mailto:cellar-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 30 Jun 2018 07:44:17 -0000

Hi,


2018-06-30 0:09 GMT+02:00 Andreas Rheinhardt
<andreas.rheinhardt@googlemail.com>om>:
> Hello,
>
> I have only just started reading the specification, but I already have
> some comments:
>
> 1. Your proposal says that DisplayHeight and DisplayWidth must be set so
> that they agree with what is said in the CodecPrivate/in the bitstream
> so that one can't use the Matroska fields anymore to override what is
> set in the bitstream. If I am not mistaken, then AV1 would be unique in
> this regard among all the video codecs that are allowed in Matroska. I
> don't see a reason for this at all and propose that there shouldn't be a
> special treatment for DisplayHeight and DisplayWidth in AV1.

You're right. This is a mistake on my part. Instead of MUST it's a
SHOULD. Because Matroska allows to override the codec internal values
to display this differently, including changing the aspect ratio or
forcing physical dimensions. I will change that.

> 2. Two sequence header OBUs with the same [seq_profile],
> [high_bitdepth], [seq_level_idx] and [color_range] can still differ in
> the [BitDepth] derived from them due to [twelve_bit]. Therefore I wonder
> if this is the right restriction. Wouldn't it be better if it is
> demanded that the derived [BitDepth] coincide for every sequence header OBU?

It depends if it influences the output of the decoder or the decoder
that can be used. My main concern here is that we have the necessary
information to setup the decoder to start playback and be done with
the duration of the Segment. Otherwise that means the CodecPrivate is
sufficient.

There's a fine line between being too restrictive and forcing multiple
Segment for a simple internal change in the codec. So we don't want to
restrict too many field from the Sequence Header. But on the other
hand we want to ensure a Segment is consistent, especially regarding
seeking.

That being said, you're right [high_bitdepth] and [twelve_bit] are the
ones needed to make up the actual [BitDepth] so I will change that.

> 3. Given that AV1 seems to only use one Sequence Header OBU at a time
> couldn't one use the CodecState and CueCodecState elements to designate
> where the currently active Sequence Header OBU can be found so that one
> doesn't need to repeat the Sequence Header with every keyframe?

Yes, that would be a good use for when seeking is used. But these
elements were ruled out as deprecated because they were of no use
until now. I know VLC doesn't read them for example.

> (Btw: There are currently conflicting recommendations regarding the
> existence of Sequence Header OBU if there is actually only one Sequence
> Header OBU. On the one hand, they should be omitted, on the other hand
> every block marked as keyframe should start with a Sequence Header OBU.)

You mean in my document or the AV1 specs ? My understanding is that
the AV1 bitstream allow Sequence Header OBUs to be found within the
stream. And doesn't mention which elements are allowed to change or
not. Maybe there will be profiles for such restrictions. It won't be
stored in the bitstream (which is frozen) but could be extra data we
put in the CodecPrivate.

> Andreas Rheinhardt/mkver
>
> PS: This is my first mail to this list and I hope that it will be
> presented as a reply to Steve Lhomme's proposed AV1 mapping in Matroska,
> but given the fact that I only subscribed today I can't simply reply,
> but had to explicitly set the "In-Reply-To" field. Hope it works.

It worked perfectly. Thanks for taking the time to review this !

> _______________________________________________
> Cellar mailing list
> Cellar@ietf.org
> https://www.ietf.org/mailman/listinfo/cellar



-- 
Steve Lhomme
Matroska association Chairman