Re: [Cbor] typed arrays for complex numbers

Gregory Allen <gallen@arlut.utexas.edu> Mon, 15 July 2019 14:55 UTC

Return-Path: <gallen@arlut.utexas.edu>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 51C3A1201A3 for <cbor@ietfa.amsl.com>; Mon, 15 Jul 2019 07:55:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level:
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=arlut.utexas.edu
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nxCkXTd_OapI for <cbor@ietfa.amsl.com>; Mon, 15 Jul 2019 07:55:23 -0700 (PDT)
Received: from esa1.arlut.utexas.edu (esa1.arlut.utexas.edu [146.6.211.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F2873120172 for <cbor@ietf.org>; Mon, 15 Jul 2019 07:55:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arlut.utexas.edu; i=@arlut.utexas.edu; q=dns/txt; s=default; t=1563202523; x=1594738523; h=from:message-id:mime-version:subject:date:in-reply-to:cc: to:references; bh=yvzwuowTl5GlZe3FTheg5crZBn3TqJMSpeF130x3u6E=; b=D8W/DXR+tB+4L+i4HPy4Z4d1hqSRDcE6HjfZZJZApnseasgwMDV6Yj6/ YnDq/cUKWOaM4IKCa/G6oLd53q4jRwSKo6QyiJvutwUej4Dx+CIZZfMDa okUdo48lDm04YkdewvUNCB4DuDEF5NDCcdze+G0viLPKPUVVUbI97Eg4L MOkAaQ8vhKaWr5LBScALNDVQJVFgp2OtHnfLsjP8HQQ/X1qI+DMkQcHnB cnffRggs94iybqIz0KhE70IeDDPQtJ+x+oCluYo4aATVIXLThCzrgMhWc moh2AAxkse79adIWkyOtriKg89lTKcQUhWO6niwKPPuQAklydrbPMCufR Q==;
IronPort-SDR: v9EAmfgLppKbMh2uFC8oZqMy+OeOMTvlQZbod8AYkwj1QvRyAJbJ3661t0Jotf9Ebof8WYRn4O rwNSRK/1Hh22+ao83yDIxul7ZA3eSqE2WLPfz90I0uOVNMSfG69YySc/mW9zcv+KTrnIzitXjC KPWe3uiwJDuxgeew5/exQBfl92rygFsXE0U5feD8OzAP5ikNEkND3Lz84bc38w8/oNwCf41TfN 6Wl+RRBIvO6bPq5+QhA+sf++Z/eIFJCIO/vdEb/APNIScEnU0VQ8zUAEGRTwxerDPbaYBQXRL7 LoE=
X-IronPort-AV: E=McAfee;i="6000,8403,9319"; a="7296499"
X-IronPort-AV: E=Sophos; i="5.63,493,1557205200"; d="p7s'?scan'208"; a="7296499"
Received: from intpfx1.arlut.utexas.edu ([10.14.8.87]) by esa1.arlut.utexas.edu with ESMTP; 15 Jul 2019 09:55:22 -0500
Received: from intesa1.arlut.utexas.edu (intesa1.arlut.utexas.edu [10.14.8.10]) by intpfx1.arlut.utexas.edu (Postfix) with ESMTP id 11BA678ED5 for <cbor@ietf.org>; Mon, 15 Jul 2019 09:55:22 -0500 (CDT)
X-IronPort-AV: E=McAfee;i="6000,8403,9319"; a="2587370"
X-IronPort-AV: E=Sophos; i="5.63,493,1557205200"; d="p7s'?scan'208"; a="2587370"
Received: from syrinx.arlut.utexas.edu ([10.14.104.95]) by mail.arlut.utexas.edu with ESMTP; 15 Jul 2019 09:55:22 -0500
From: Gregory Allen <gallen@arlut.utexas.edu>
Message-Id: <328EA863-7747-4723-983E-5B1EF248A2A1@arlut.utexas.edu>
Content-Type: multipart/signed; boundary="Apple-Mail=_D268AD3F-EA69-4CE3-ABE1-B69CA691B932"; protocol="application/pkcs7-signature"; micalg=sha1
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Date: Mon, 15 Jul 2019 09:55:21 -0500
In-Reply-To: <20190712141659.GA1505@meili>
Cc: Carsten Bormann <cabo@tzi.org>, cbor@ietf.org
To: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
References: <15400EA6-DA5F-499E-B9A5-B98E0A6887CB@arlut.utexas.edu> <49F73FF9-1952-421E-9C0D-A9B2821FCEC7@tzi.org> <20190712141659.GA1505@meili>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/gU-i6svu9CTtRCKzqJ78vliRyMQ>
Subject: Re: [Cbor] typed arrays for complex numbers
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Jul 2019 14:55:26 -0000

Perhaps “complex” deserves a special tag because it’s so common, but it’s easy to see how a tag for each type of compound data doesn’t scale well.

Following Jeff’s line of reasoning could lead to something like structured data types in NumPy [https://docs.scipy.org/doc/numpy/user/basics.rec.html]. HDF5 supports similar constructs.

The idea is that one describes the type, and then an array can contain elements of the user-defined type. This can be used to describe blobs containing homogeneous arrays of arbitrary structures.

For example (from NumPy):

    cymk_type = [(‘c’, np.uint8), (‘y’, np.uint8), (‘m’, np.uint8), (‘k’, np.uint8)] # list of (name, type) pairs
    x = np.zeros([640, 480], dtype=cymk_type)

I didn’t propose this before because it’s obviously much greater in scope than adding a “complex” tag. It’s not even clear to me what approach I would want to take to add this to CBOR.

Thanks,
-Greg

> On Jul 12, 2019, at 9:17 AM, Josef 'Jeff' Sipek <jeffpc@josefsipek.net>; wrote:
> 
> On Fri, Jul 12, 2019 at 02:23:23 +0200, Carsten Bormann wrote:
>> On Jul 12, 2019, at 01:24, Gregory Allen <gallen@arlut.utexas.edu>; wrote:
>>> 
>>> I could imaging a “complex” tag, that’s a modifier for a dataType, so that one could do:
>>> 
>>>   multi-dim([dim, complex(ta-float32le(bstr)) ])
>>> 
>>> However, that doesn’t scale well to having several different compound types such as those mentioned above.
>> 
>> Well, it requires a new tag for each of these compound types, but if these are indeed different types, then tags are cheap (as long as they don’t have to come out of the 1+1 space :-).
>> 
>> So if this helps, we could indeed define tags (for complex, quaternion,
>> octonions, sedenions, RGB, RGBA) that extract groups of 2, 4, 8, 16, 3, 4
>> elements out of an array (tagged or classical CBOR).
>> (Well, RGB is a bit weird, because there are so many color spaces.  Maybe
>> just have tags for three- and four-dimensional color spaces.)
> 
> I'm far from a color expert, but as far as I know even for RGB color spaces
> the values aren't always stored in one order.  At the very least there are
> RGBA and ARGB.
> 
> Currently my best idea amounts to:
> 
> 	color(["RGB", [R1, G1, B1, R2, ...])
> 	color(["RGBA", [R1, G1, B1, A1, R2, ...])
> 	color(["CMYK", [C1, M1, Y1, K1, C2, ...])
> 
> I don't know if using a text string to identify the color component names
> and order is an anti-pattern in CBOR but it is similar to the currency
> related tags discussed some months ago.
> 
> This could even be extended to support planar formats (where the values for
> each component are clumped together to allow better compression) and chroma
> subsampling (where there aren't the same number of samples for each
> component).  With either:
> 
> 	color(["RGB", [R1, R2, ...], [G1, G2, ...], [B1, B2, ...])
> 	color(["YUV420", [Y1, Y2, ...], [U1, U2, ...], [V1, V2, ...])
> 
> or:
> 
> 	color(["RGB", [R1, R2, ..., G1, G2, ..., B1, B2, ...])
> 	color(["YUV420", [Y1, Y2, ..., U1, U2, ..., V1, V2, ...])
> 
> I suppose the second would need to identify the number or ratio of values
> for each component to support chroma subsampling, so the first would
> probably be better.
> 
> Going down the rabbit hole of storing color data, there are also CCDs which
> red/blue pixels.
> 
> 
> Anyway, beware I know just enough about this to be dangerous :)  I'd
> probably stay away from defining tags for colors until an actual expert
> comes along.
> 
> Jeff.
> 
>> Alternatively, supplying the group size separately maybe can do all of these at omce, but then cannot distinguish quaternions from RGBA or CMYK.
>> 
>> Grüße, Carsten
>> 
>> _______________________________________________
>> CBOR mailing list
>> CBOR@ietf.org
>> https://www.ietf.org/mailman/listinfo/cbor
> 
> -- 
> My public PGP key can be found at https://www.josefsipek.net/pgp/