Re: [Cellar] Matroska Elements to support frame side data

Moritz Bunkus <> Mon, 05 November 2018 09:29 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id B52551277BB for <>; Mon, 5 Nov 2018 01:29:23 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id gFWq_uDhhRVX for <>; Mon, 5 Nov 2018 01:29:22 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 3FD701274D0 for <>; Mon, 5 Nov 2018 01:29:22 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=mail2018100901; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:Date:In-reply-to:Subject:Cc:To:From:References; bh=aDe7HUiy182PjVwdfNvoj/MyXy23YRzQXhDIPDIoXYY=; b=tzHnF+emphpJqblE2/cMVw+5Lc78a6A7yS9SjfEk4S3/DWTd5WYQzf7K7o7hcHFICsgvba+R2M+Rzv6QQVlosfuXQVlfU2Gzi4W7AsinwtC2CT22MkL3ouwviflQqBWhF5EXI9zdVl1puemVJv0D5ZBb7oz/NGoLa0v9SvaQQBo=;
Received: from ([2a01:4f8:190:8147::105:1]:42112) by with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from <>) id 1gJbC1-0004G2-1j; Mon, 05 Nov 2018 10:29:09 +0100
Received: from sweet-chili.local (unknown []) by (Postfix) with ESMTPS id E5948654000D; Mon, 5 Nov 2018 10:29:02 +0100 (CET)
Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTP id 2613A4DC4D87; Mon, 5 Nov 2018 10:29:02 +0100 (CET)
References: <> <> <>
User-agent: mu4e 1.0; emacs 26.1
From: Moritz Bunkus <>
To: Dave Rice <>
Cc: Tobias Rapp <>,
In-reply-to: <>
Date: Mon, 05 Nov 2018 10:29:02 +0100
Message-ID: <>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Archived-At: <>
Subject: Re: [Cellar] Matroska Elements to support frame side data
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 05 Nov 2018 09:29:24 -0000


> In other thoughts on this suggestion, I think it could make it difficult
> to easily understand if a file has a particular type of side data. For
> instance if only a few Clusters somewhere in the Segment contain a
> certain type of side data, it would require parsing every Cluster to know
> what types of side data are available. This uncertainly wouldn’t be the
> same issue if the side data was itself a Track.

It's not entirely necessary to use a full track for side data. We can simply
signal the presence of side data in the track headers and refer to it from
the side data in the block groups. This would also mean we only have to
store the string identifying the side data type once (in the track headers)
instead of in each block.

(I'll use "BlockMetadata" as the basis for all element names
here. Initially I proposed "FrameMetadata", but "BlockMetadata" is fine
with me, too.)

For example:

+- TrackEntry
 +- TrackBlockMetadata (Master)
  +- TrackBlockMetadataType (String, required)
  +- TrackBlockMetadataID (Unsigned Integer, required)


+- BlockGroup
 +- BlockMetadata (Master)
  +- BlockMetadataID (Unsigned Integer, required, refers to existing
     TrackBlockMetadataID in track headers)
  +- BlockMetadataString (Unicode String, optional)
  +- BlockMetadataBinary (Binary, optional)
  +- BlockMetadataUInteger (Unsigned Integer, optional)
  +- BlockMetadataSInteger (Signed Integer, optional)
  +- BlockMetadataFloat (Float, optional)

with the restriction that exactly one of (BlockMetadataString,
BlockMetadataBinary, BlockMetadataUInteger, BlockMetadataSInteger,
BlockMetadataFloat) must exist.

Advantages as I see them:

• Less overhead (no repeated string parsing required)
• Quicker parsing (no repeated string parsing required)
• Presence of meta data is known upfront
• Not using a full-blown track for meta data would alleviate the need to
  specify how all those track and block features (e.g. BlockDuration,
  TrackDefaultDuration…) apply to a "meta data track".

Kind regards,