Re: [Cellar] On the multiplicity of Info elements

Moritz Bunkus <moritz@bunkus.org> Fri, 15 January 2016 08:18 UTC

Return-Path: <moritz@bunkus.org>
X-Original-To: cellar@ietfa.amsl.com
Delivered-To: cellar@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 593531A0383 for <cellar@ietfa.amsl.com>; Fri, 15 Jan 2016 00:18:35 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.903
X-Spam-Level:
X-Spam-Status: No, score=-1.903 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XyZgYDc3X7hx for <cellar@ietfa.amsl.com>; Fri, 15 Jan 2016 00:18:34 -0800 (PST)
Received: from liselle.bunkus.org (liselle.bunkus.org [IPv6:2a01:4f8:151:7310::105:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E62AA1A036F for <cellar@ietf.org>; Fri, 15 Jan 2016 00:18:33 -0800 (PST)
Received: from sweet-chili.local (unknown [10.55.4.6]) by liselle.bunkus.org (Postfix) with ESMTPS id CBC45B35675 for <cellar@ietf.org>; Fri, 15 Jan 2016 09:18:31 +0100 (CET)
Received: by sweet-chili.local (Postfix, from userid 1000) id 4156B61FE69; Fri, 15 Jan 2016 09:18:31 +0100 (CET)
Date: Fri, 15 Jan 2016 09:18:31 +0100
From: Moritz Bunkus <moritz@bunkus.org>
To: cellar@ietf.org
Message-ID: <20160115081830.GD4063@bunkus.org>
References: <CAOXsMF+gc0d2LEisfHm0jnjDGQKcYquEMBt7FnZ_uuSNF=C0iw@mail.gmail.com> <568AC10F.9030303@gmx.de> <CAOXsMFKJJhzU-3CYqguDePY42T+Vvhx9ytAfvoM6xyqaZY+N4g@mail.gmail.com> <FCC4DC05-44CD-4C2B-8C59-8E3E5B494DC0@dericed.com> <568D6710.8000605@gmx.de> <692F039A-180D-4535-B4A1-529A777573F5@dericed.com> <5693D5D6.6030709@xiph.org> <F6B37DB3-EDCB-4BD2-9B0D-F8A4F353F36E@dericed.com> <20160112133656.GJ4063@bunkus.org> <1E1F9DD9-0B41-4B3D-AC7A-D59E72D90C33@dericed.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="4NAIfEpSjlXpZEyn"
Content-Disposition: inline
In-Reply-To: <1E1F9DD9-0B41-4B3D-AC7A-D59E72D90C33@dericed.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-Virus-Scanned: clamav-milter 0.98.7 at liselle
X-Virus-Status: Clean
Archived-At: <http://mailarchive.ietf.org/arch/msg/cellar/7tt9ebAa4FXMcjzaJswdWoeHhnc>
Subject: Re: [Cellar] On the multiplicity of Info elements
X-BeenThere: cellar@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Codec Encoding for LossLess Archiving and Realtime transmission <cellar.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cellar>, <mailto:cellar-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cellar/>
List-Post: <mailto:cellar@ietf.org>
List-Help: <mailto:cellar-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cellar>, <mailto:cellar-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Jan 2016 08:18:35 -0000

Hey,

> There are certainly Matroska files that use CRC-32 within Clusters.

True, I'm just debating their value ;) Sure, you can detect that there's
a bit error in that cluster, but what are you going to do? Drop the
whole cluster? OK, you can inform the user about it and she can watch
the appropriate section in the file in order to determine how bad it is.

My current goal for MKVToolNix is to handle CRCs everywhere:

1. mkvmerge will write them for each level 1 element

2. mkvmerge will check existing CRCs and emit warnings about elements
   where the CRC doesn't match the data

3. mkvpropedit will include them in each level 1 element it changes/adds
   as will the GUI's header/chapter/attachment editors

4. mkvinfo will print existing CRCs and emit warnings about elements
   where the CRC doesn't match the data

Unfortunately all of those require changes in libEBML. The ones required
for 1. and 3. are not that bad (handling of masters in which a CRC
element is already present) and are ABI-compatible.

The changes for 2. and 4. are much more involved, though; for one CRC
elements read from storage must be passed through to the application (at
the moment libEBML tries to handle them transparently while mkvinfo
wants to output the element's presence and data); additionally the
verification is currently broken (the value's correct on disk, though)
and I haven't really understood why that is. It's also very slow. Fixing
this most likely requires changing the ABI, and making it fast/use less
memory bandwidth will require even more ABI-breaking changes.

It'll take some time until this has been implemented.

Kind regards,
mosu