Re: [Cellar] Matroska Elements to support frame side data

Tobias Rapp <> Mon, 05 November 2018 15:41 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 82E26130DF2 for <>; Mon, 5 Nov 2018 07:41:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id VROKDksR5tgK for <>; Mon, 5 Nov 2018 07:41:23 -0800 (PST)
Received: from ( [IPv6:2a02:2410:b000:101:3000::c]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id A88C4130DE6 for <>; Mon, 5 Nov 2018 07:41:22 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTPS id 4A026A044F for <>; Mon, 5 Nov 2018 16:41:17 +0100 (CET)
Received: from mailix ( []) by (Postfix) with ESMTPA id C9F2D816A3 for <>; Mon, 5 Nov 2018 16:41:16 +0100 (CET)
Received: from [] ( []) by mailix with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128) ; Mon, 5 Nov 2018 16:41:16 +0100
References: <> <> <> <> <>
From: Tobias Rapp <>
Organization: NOA GmbH
Message-ID: <>
Date: Mon, 5 Nov 2018 16:41:16 +0100
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-PPP-Message-ID: <>
Archived-At: <>
Subject: Re: [Cellar] Matroska Elements to support frame side data
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 05 Nov 2018 15:41:27 -0000

On 05.11.2018 12:10, Jerome Martinez wrote:
> On 05/11/2018 10:07, Tobias Rapp wrote:
>> I should clarify: Storing the timecode value as a _single_ number 
>> makes processing a lot easier. [...]
> It adds another complexity: you need to transport somewhere else the 
> configuration of this time code value (DP/NDP, frame rate).

I would say that the complexity is just shifted from read to write side. 
It depends on the assumed balance of write and read (archive vs. reuse, 
ingest vs. outgest) whether to see this as adding or removing complexity.

> which is not always available, if I do e.g. realtime transcoding from 
> SDI and catch SMPTE ST 12, I don't have immediately the frame rate (I 
> need to wait for frame number going back to 0) so I would have to wait 
> up to 1 second before writing the Matroska header if frame rate info is 
> in track header.

You could reserve some space in the header to write the frame-rate 
information later.

> For reference MP4/MOV or MXF time codes use numbered values, SMPTE ST 
> 12, SDTI in MXF or GXF use a format which is more like a string time 
> code. strict conversion between time code format (especially keeping the 
> user bytes) is not always doable.
> In my opinion we should not debate here of the best time code format (it 
> would be in a metadata format registry, and both string time code and 
> numbered time code may exist), just permit any of them so I like the 
> proposal from Moritz, especially because I would use it too for 
> RAWcooked the same way as with time code configuration in the track 
> header for DP/NDP and frame rate info (if I store some RAWcooked 
> configuration in the track header, I can improve a lot the comrpession 
> of RAWcooked metadata).
Indeed the timecode format is of second interest (even though 
implementation of multiple formats also adds complexity).

The idea of adding general metadata for each frame is nice, and storing 
DPX tags into it is fine. But I wouldn't store timecode that way, 
regardless of the serialization format. When you want to seek to a 
specific timecode position you have to read all frame metadata in 
advance. If timecode is stored separately, possibly as an own track, it 
allows you to do the mapping from timecode to timestamp first and then 
use the cue table to perform the locate operation efficiently.