Re: [Cellar] Mapping of MOV clean aperture values to Matroska

Steve Lhomme <slhomme@matroska.org> Mon, 03 December 2018 07:45 UTC

Return-Path: <slhomme@matroska.org>
X-Original-To: cellar@ietfa.amsl.com
Delivered-To: cellar@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 18250130E0F for <cellar@ietfa.amsl.com>; Sun, 2 Dec 2018 23:45:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.359
X-Spam-Level:
X-Spam-Status: No, score=-3.359 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-1.459, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7IdoXF6vg12f for <cellar@ietfa.amsl.com>; Sun, 2 Dec 2018 23:45:47 -0800 (PST)
Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CE782130E03 for <cellar@ietf.org>; Sun, 2 Dec 2018 23:45:47 -0800 (PST)
Received: by mail-pl1-x62f.google.com with SMTP id e5so6027009plb.5 for <cellar@ietf.org>; Sun, 02 Dec 2018 23:45:47 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=wgo/Eo7XbdCrnUrV764PTDYU05OcvdEtpNv1Li3bnqA=; b=asi4AjCy3mSL0PYC8eh3lJymG3isoy2MbpzRj/wMQ6jme30KFETULy/CggQvcns0/M hFqyhWMxrNfROZh3Wq9zzi8N0YHpipiApF6I3DWkUc0zi+SoDgznGYQVKMzBjZNUTkd7 rrvL9hwhZ4FVmYii3aCBEUkqAHxx7VmNWI4/1a2aq7zLn/Fby80olR9bqV/zBW+QUyE3 UeARxKFwaHHytVdWXv0n0p2TfUGnsFTB3A43cqnyVBxYOqzBLVO4BDWL9N48kQbiDCZr 7tWKvqMdMQS5qU108o4r6oamy+D6nfgD7P9tpaohJdCHzYPhvORu90xedYag2vj2Ml/P MCIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=wgo/Eo7XbdCrnUrV764PTDYU05OcvdEtpNv1Li3bnqA=; b=VfQHTzUxzn8zsgliewlh7Mpp0qx6lWyVAAwBT4SzIG0PrCqCMlxt5K5Npv82z7+bXG NRcwh7X1vZgzcUwtY2Aj7Gtr41l5pRqLsG1Tl0CibhRW8iT7tVaLKYBW7PAVJgDG9NF5 M2k1Fkn1xohoWu13/Riq/RT+uHaD3rqwIbK2g+Gu35dQcIhSYrTvXLgWg2ItE1eCOKqj pJ2gE/4NFk3m4yeBbXuDwt/Ez8zBSHRj4iKS+cOdjwKls9nQsAfYMh8x7RIVDeqLY8tV n7fGjsWERGx5e9YA3cCG6XNf0lfKlsHTFnyFLCoqFRNJJhAPBnSLbXnDU34N7tLomZKW r9XA==
X-Gm-Message-State: AA+aEWYnhrI/JFzFScZINxkN8yeYzrX1FE6pOCN7WPpKnoPfBHBXCeH2 C4OjBKVcEzE152Vq/MdhArlj1MsMfTmPXM4wUNJnTl4slT4=
X-Google-Smtp-Source: AFSGD/UZtgVDvOnbdRdZQ5AC9X+Tl+0b5tIF5s4U9DllL+QJOhj0tyR8lAuS3JJW71KI0XmA1N8stsrDaAUyKpIxYoo=
X-Received: by 2002:a17:902:442:: with SMTP id 60mr155256ple.73.1543823147206; Sun, 02 Dec 2018 23:45:47 -0800 (PST)
MIME-Version: 1.0
References: <CAO7v-1Qci+eGHBbQYuHYn-fSb+WA+ac-=Z89zkcdcYpQaKGQEw@mail.gmail.com> <353D4CFD-370C-4509-B7BE-3D572EC2DEFB@dericed.com> <CAO7v-1RqTcC4ty0O-Qk99g_FoRGhyNb_uTDcj6JxcAbM34sscg@mail.gmail.com> <CAOXsMFL7hv=iA7WTnv_FHJSmcOzHspXfF57Xi+PY7BJR5A+nUw@mail.gmail.com> <CAO7v-1QEuW5oQBd5PBq2cD8ZPLU9kjd1L9c8v=KJ2TKMNcJyiQ@mail.gmail.com>
In-Reply-To: <CAO7v-1QEuW5oQBd5PBq2cD8ZPLU9kjd1L9c8v=KJ2TKMNcJyiQ@mail.gmail.com>
From: Steve Lhomme <slhomme@matroska.org>
Date: Mon, 3 Dec 2018 08:45:37 +0100
Message-ID: <CAOXsMFJVLusiRZpFNCAwkZCA6CjnbYvx06ARopdCPswqBCoVpA@mail.gmail.com>
To: Kieran O Leary <kieran.o.leary@gmail.com>
Cc: Codec Encoding for LossLess Archiving and Realtime transmission <cellar@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/cellar/BtD1Z4tWa328-Dedw6hE7mwlm7I>
Subject: Re: [Cellar] Mapping of MOV clean aperture values to Matroska
X-BeenThere: cellar@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <cellar.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cellar>, <mailto:cellar-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cellar/>
List-Post: <mailto:cellar@ietf.org>
List-Help: <mailto:cellar-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cellar>, <mailto:cellar-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Dec 2018 07:45:50 -0000

Le dim. 2 déc. 2018 à 17:18, Kieran O Leary <kieran.o.leary@gmail.com>; a écrit :
>
> Hi Steve,
>
> On Sun, Dec 2, 2018 at 3:44 PM Steve Lhomme <slhomme@matroska.org>; wrote:
>>
>> Hello,
>>
>> For VLC the values are read but they are likely lost along the way to
>> the display by the decoder or the video output. You may try this file
>> to check if your video output in VLC supports cropping:
>> http://v2v.cc/~j/theora_testsuite/offset_test.ogv
>>
> That's pretty cool. ffplay and mpv seem to display the full image, but VLC displays a crop. At Teasdale in my home Ubuntu 18..04 desktop.

This is one tricky corner case that we don't fully support everywhere.
But we're supposed to.

>> This illustrates the issue with cropping at the container level (and
>> ffmpeg) the container may have some values set but some codecs also
>> have such info and may overwrite this value. And there's not really
>> any way to tell who is write. It may depend on the codec and the
>> container.
>
>
> I"m mostly familiar with uncompressed video within MOV, which is then compressed to FFV1/MKV, and in these cases you would only have values in the container. I hadn't thought of the other codecs that could have stream-level cropping.

There's Theora at least (see above) probably some modern MPEG ones as
well since the "blocks" have to be some power of 2 sizes but the main
source may not be.

>> In Matroska we assume that we override what the codec says.
>> But I don't think that's how it's handled in ffmpeg. Also there's the
>> question of what these values apply to: on the decoded surface, which
>> may include padding on the right/bottom or the area that's supposed to
>> be visible. In Matroska the "PixelWidth/PixelHeight" correspond to the
>> visible area, on which we apply extra cropping. (this is currently not
>> written anywhere).
>>
>> As for the MOV atom, is there are spec somewhere ? Or is it clear that
>> it's some cropping ?
>
>
> I think this is as good of a spec that is freely available - the 'clap section' https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html#//apple_ref/doc/uid/TP40000939-CH205-125850

OK, so we should support it in VLC (which we don't) and that seems to
map more or less to Matroska values.

>> Is it in the supposedly visible area or a
>> supposedly padded area (which may not correspond in ffmpeg to the
>> padding Quicktime would use) ?
>
>
> Unfortunately I don't fully understand what you mean here by supposedly visible and padded. PAL video has padded black pillarboxing (i have seen expections to this) that is not intended to be viewed. These clean aperture values specify the area that is to be displayed to the viewer. These values are also essential to generating the correct aspect ratio. If the 720x576 pixels is transformed with a PAR of the PAL standard - 59:54, then you get 786x576 aka 1.364:1, but if the 59:54 value is applied with the clean aperture values taken into consideration, then you will get the correct 4:3 aspect ratio..

As said above modern codecs use Blocks of 8x8, 16x16, etc to encode
pictures. There's codec padding if your have a source of 3x3 for
example. But to use SIMD (fast assembler code) you may want to align
to 32 pixels (to use AVX-2 for example) that's purely a decoder
optimization and depends on the implementation. But that's extra
padding.

libavcodec (in ffmpeg) requires a padding of 32 on all planes even if
the source doesn't require it. That means it forces a first layer of
padding/cropping. But the container/codec one should be applied
nonetheless. It's probably not done here.