Re: [Cellar] Mapping of MOV clean aperture values to Matroska

[Steve Lhomme <[e-mail]>]

Le dim. 2 déc. 2018 à 17:18, Kieran O Leary <[e-mail]> a écrit :
>
> Hi Steve,
>
> On Sun, Dec 2, 2018 at 3:44 PM Steve Lhomme <[e-mail]> wrote:
>>
>> Hello,
>>
>> For VLC the values are read but they are likely lost along the way to
>> the display by the decoder or the video output. You may try this file
>> to check if your video output in VLC supports cropping:
>> http://v2v.cc/~j/theora_testsuite/offset_test.ogv
>>
> That's pretty cool. ffplay and mpv seem to display the full image, but VLC displays a crop. At Teasdale in my home Ubuntu 18..04 desktop.

This is one tricky corner case that we don't fully support everywhere.
But we're supposed to.

>> This illustrates the issue with cropping at the container level (and
>> ffmpeg) the container may have some values set but some codecs also
>> have such info and may overwrite this value. And there's not really
>> any way to tell who is write. It may depend on the codec and the
>> container.
>
>
> I"m mostly familiar with uncompressed video within MOV, which is then compressed to FFV1/MKV, and in these cases you would only have values in the container. I hadn't thought of the other codecs that could have stream-level cropping.

There's Theora at least (see above) probably some modern MPEG ones as
well since the "blocks" have to be some power of 2 sizes but the main
source may not be.

>> In Matroska we assume that we override what the codec says.
>> But I don't think that's how it's handled in ffmpeg. Also there's the
>> question of what these values apply to: on the decoded surface, which
>> may include padding on the right/bottom or the area that's supposed to
>> be visible. In Matroska the "PixelWidth/PixelHeight" correspond to the
>> visible area, on which we apply extra cropping. (this is currently not
>> written anywhere).
>>
>> As for the MOV atom, is there are spec somewhere ? Or is it clear that
>> it's some cropping ?
>
>
> I think this is as good of a spec that is freely available - the 'clap section' https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html#//apple_ref/doc/uid/TP40000939-CH205-125850

OK, so we should support it in VLC (which we don't) and that seems to
map more or less to Matroska values.

>> Is it in the supposedly visible area or a
>> supposedly padded area (which may not correspond in ffmpeg to the
>> padding Quicktime would use) ?
>
>
> Unfortunately I don't fully understand what you mean here by supposedly visible and padded. PAL video has padded black pillarboxing (i have seen expections to this) that is not intended to be viewed. These clean aperture values specify the area that is to be displayed to the viewer. These values are also essential to generating the correct aspect ratio. If the 720x576 pixels is transformed with a PAR of the PAL standard - 59:54, then you get 786x576 aka 1.364:1, but if the 59:54 value is applied with the clean aperture values taken into consideration, then you will get the correct 4:3 aspect ratio..

As said above modern codecs use Blocks of 8x8, 16x16, etc to encode
pictures. There's codec padding if your have a source of 3x3 for
example. But to use SIMD (fast assembler code) you may want to align
to 32 pixels (to use AVX-2 for example) that's purely a decoder
optimization and depends on the implementation. But that's extra
padding.

libavcodec (in ffmpeg) requires a padding of 32 on all planes even if
the source doesn't require it. That means it forces a first layer of
padding/cropping. But the container/codec one should be applied
nonetheless. It's probably not done here.