Re: [Cellar] AV1 mapping Matroska

Steve Lhomme <slhomme@matroska.org> Mon, 02 July 2018 07:38 UTC

Return-Path: <slhomme@matroska.org>
X-Original-To: cellar@ietfa.amsl.com
Delivered-To: cellar@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7D4A1130F11 for <cellar@ietfa.amsl.com>; Mon, 2 Jul 2018 00:38:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.909
X-Spam-Level:
X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8XZQ0PjXCGvS for <cellar@ietfa.amsl.com>; Mon, 2 Jul 2018 00:38:55 -0700 (PDT)
Received: from mail-pl0-x236.google.com (mail-pl0-x236.google.com [IPv6:2607:f8b0:400e:c01::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 48C14130E54 for <cellar@ietf.org>; Mon, 2 Jul 2018 00:38:55 -0700 (PDT)
Received: by mail-pl0-x236.google.com with SMTP id z9-v6so7527783plo.1 for <cellar@ietf.org>; Mon, 02 Jul 2018 00:38:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=srnBCtW0M5+UsaYWzvdF3UDB+TVGvu/kPJJEXlyqIx8=; b=OzjHUACN4o80aNctbpn8yydz3ThoJcnv0EDqTR7DD3GCqryzONaRS8kweCKbgECD4U AEnH/yaTunQskTe6Gt8UqXVbBOpaVKP92aWfLUIWrFBBLmsMzxtX6XR8RD8doZgXfMwD c3y3iy1CzCH1e2VRw4VvPD9KL1rBn6TEa/Q+OR937/ln0Lwya5bUzcqEO8jSU/P5dK1P iO6/xlrkanRjis/gLw64pbRa6JsEIRof6pJBRWRq9q2aBxHI9KV0hFiN1xN88VNf+x4l WwnO1npg79NWm/FXPbF+8LUhf1NT5suiigMMYwq6/bIi4wJYd8SHOjzHW6cv5WkyQfjw Su6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=srnBCtW0M5+UsaYWzvdF3UDB+TVGvu/kPJJEXlyqIx8=; b=WCZOJVEJSWlcbUw9jE8e5IPkQiUv/WjKy1lm4EMzWOGFYFKnzEsoz/wG18wsjMbzza TybjCShRNM2MsA1lCTi3FjqtnZvTru9xUDlvE1rMqMoN7X4Fu1q6d4gViGIyaJSVwrQ5 bgNPBHXTzCB+emOpRyHJripkMeEMl061ZrOcls4UDOJZSP9gLzryBHFoM25lhUxYvK5W 24TQM3RLoDHAwH1yRTvbBwiDf2OM/7JSqp+49TuYLvH1Q0uCAGgiiqWU0yrvU278nE1i ZgZJYzOvP96KcasyZ/61pB63g+tfv0aJp3JKFAqJRkDcqz3pEo58tKGri2MPzBHJg234 3FxA==
X-Gm-Message-State: APt69E3u3IZ9BCzfoaSAc0iBX7igGfFoxU8Gyh8mAJcd3Uc0jIOE8VDb S7Qb+DU7M8LP3qzPGtAlOqvcC8PUfPFQY/YIp/ZcVw==
X-Google-Smtp-Source: ADUXVKIwMokwvknoLRzLsz6zOAByBFG3NrVaCAs26+OczfDgkFrl4S8Tp8slMf3GCeCZulc6bGt+rhSRFQBAIvxhfv4=
X-Received: by 2002:a17:902:2f43:: with SMTP id s61-v6mr24533410plb.274.1530517134534; Mon, 02 Jul 2018 00:38:54 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a17:90a:17af:0:0:0:0 with HTTP; Mon, 2 Jul 2018 00:38:53 -0700 (PDT)
In-Reply-To: <dada3690-c8da-05db-4ddb-32476c1170f5@xiph.org>
References: <f603a9f5-d7dc-6640-1f53-f4d7b62a788e@googlemail.com> <CAOXsMFJJbHbh3=Gnu7Uag4Skq5+=jOimFNPxuicgDiVdbven4w@mail.gmail.com> <dada3690-c8da-05db-4ddb-32476c1170f5@xiph.org>
From: Steve Lhomme <slhomme@matroska.org>
Date: Mon, 2 Jul 2018 09:38:53 +0200
Message-ID: <CAOXsMF+CPW6VJsFaeahuL00hnvq5dUO-gHE3sYXPr5iVy8s1aQ@mail.gmail.com>
To: "Timothy B. Terriberry" <tterribe@xiph.org>
Cc: Codec Encoding for LossLess Archiving and Realtime transmission <cellar@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/cellar/UI-ER055fHOYCPsnXe0boV-fi1E>
Subject: Re: [Cellar] AV1 mapping Matroska
X-BeenThere: cellar@ietf.org
X-Mailman-Version: 2.1.26
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <cellar.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cellar>, <mailto:cellar-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cellar/>
List-Post: <mailto:cellar@ietf.org>
List-Help: <mailto:cellar-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cellar>, <mailto:cellar-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Jul 2018 07:39:06 -0000

2018-06-30 15:54 GMT+02:00 Timothy B. Terriberry <tterribe@xiph.org>rg>:
> Steve Lhomme wrote:
>>
>> You're right. This is a mistake on my part. Instead of MUST it's a
>> SHOULD. Because Matroska allows to override the codec internal values
>> to display this differently, including changing the aspect ratio or
>> forcing physical dimensions. I will change that.
>
>
> Keep in mind that "SHOULD" tends to have a somewhat stronger force in an
> IETF specification than in normal English. If you're going to use SHOULD and
> not MUST, it is a good idea to explain when someone might reasonably violate
> that SHOULD (if you can't because you can't think of a reason, then you want
> MUST; if you can't because there are too many reasons, then you want MAY).

I changed the DisplayWidth/DisplayHeight mapping to MAY because there
is no strong requirement needed. PixelWidth/PixelHeight should be
fixed for the whole Segment and are mandatory. But the display ones
can be edited even by a user to change the aspect ratio. Using MAY
just means that the default values should come from those bits
mentioned.

>> You mean in my document or the AV1 specs ? My understanding is that
>> the AV1 bitstream allow Sequence Header OBUs to be found within the
>> stream. And doesn't mention which elements are allowed to change or
>> not. Maybe there will be profiles for such restrictions. It won't be
>
>
> See Section 7.5 "Ordering of OBUs":
>
> "Sequence header OBUs may appear in any order within a coded video sequence.
> Within a particular coded video sequence, the contents of
> sequence_header_obu must be bit-identical each time the sequence header
> appears except for the contents of operating_parameters_info. A new coded
> video sequence is required if the sequence header parameters change."

Ah I kinda missed that part.

> So the question you want to answer is how what is stored in a Matroska file
> maps to a "coded video sequence". My opinion (as an individual) is that a
> single EBML Document should correspond to a single coded video sequence.

That would be a Segment in Matroska.

> That means that nothing in the sequence header can change (except for the
> operating_parameters_info).

Section 7.5 starts with this:

"A bitstream conforming to this specification consists of one or more
coded video sequences."

So there can be bitstreams with various coded video sequences, which
parameters may change as the text right after the one you pasted
suggests:

"A new coded video sequence is required if the sequence header
parameters change. Any sequence header in a bitstream which changes
the parameters must be contained in a temporal unit with temporal_id
equal to zero."

My understanding is that an AV1 bitstream can change a lot of
parameters for a whole stream. A coded video sequence is strict but so
much that people might tend to use more than one even in simple files
(although it would probably lose some of the codec efficiency). So
either we allow more than one coded video sequence (the direction I
took) and so we need to define what parameters must not change for all
the Sequence Header OBUs found. Or we only allow one coded video
sequence per Segment. It simplifies things a lot, but I fear we'll end
up with more than one Segment per "file" which is not what most people
use nowadays, even if it's legal to do so.

I'd like more input on this on what they expect the best practices for
"coded video sequence" will be. It's not obvious to me from just
reading the specs.

Looking at the term "coded video sequence" (CVS) in other codecs
(H.264 and H.265) it seems it's a common term. And for those codec one
Segment correspond to one CVS, with the parameters of that CVS stored
in the CodecPrivate (SPS + PPS for H.264 for example). So we should
probably go in that simple way.

>
>
> _______________________________________________
> Cellar mailing list
> Cellar@ietf.org
> https://www.ietf.org/mailman/listinfo/cellar



-- 
Steve Lhomme
Matroska association Chairman