Re: [Cbor] hildjj/cbor-map-entries: Explicit Map datatype for CBOR, in array format

Kio Smallwood <kio@mothers-arms.co.uk> Thu, 18 February 2021 19:51 UTC

Return-Path: <kio@mothers-arms.co.uk>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6DB773A169B for <cbor@ietfa.amsl.com>; Thu, 18 Feb 2021 11:51:15 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.496
X-Spam-Level:
X-Spam-Status: No, score=-1.496 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, KHOP_HELO_FCRDNS=0.4, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Pxdeu7ELw8gS for <cbor@ietfa.amsl.com>; Thu, 18 Feb 2021 11:51:13 -0800 (PST)
Received: from authenticated.a-painless.mh.aa.net.uk (painless-a.thn.aa.net.uk [IPv6:2001:8b0:62::26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 297393A1665 for <cbor@ietf.org>; Thu, 18 Feb 2021 11:51:12 -0800 (PST)
Received: from a-webmail.thn.aa.net.uk ([2001:8b0:62::22] helo=webmail.aa.net.uk) by painless-a.thn.aa.net.uk with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <kio@mothers-arms.co.uk>) id 1lCpKQ-00074d-UP; Thu, 18 Feb 2021 19:51:10 +0000
Received: from cpc105076-sgyl40-2-0-cust233.18-2.cable.virginm.net ([82.4.24.234]) by webmail.aa.net.uk with HTTP (HTTP/1.1 POST); Thu, 18 Feb 2021 19:51:08 +0000
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=_45759845b993f2194077b1ad6c78f4ab"
Date: Thu, 18 Feb 2021 19:51:08 +0000
From: Kio Smallwood <kio@mothers-arms.co.uk>
To: Carsten Bormann <cabo@tzi.org>
Cc: Laurence Lundblade <lgl@island-resort.com>, Joe Hildebrand <hildjj@cursive.net>, "Dale R. Worley" <worley@ariadne.com>, cbor@ietf.org
In-Reply-To: <05798259-49C4-4442-84A3-9FD13D0BDB5B@tzi.org>
References: <87zh02kpf5.fsf@hobgoblin.ariadne.com> <0faeb37c36d5b1f40c37f82e62be1be9@mothers-arms.co.uk> <ED6F6174-2643-4B7F-8ED7-414DD5FD9C39@tzi.org> <F6BDBA57-34F4-46FE-9806-6A227AC214EA@mothers-arms.co.uk> <DC0D444C-3231-4029-B0BB-5EC02863B72A@cursive.net> <999B078D-C4B2-4B04-8487-7ECEFEB953DF@cursive.net> <94bbc990fcf388ae5fd01c06274ed58a@mothers-arms.co.uk> <72053102-1BA1-4281-9F17-CF7A65EEA907@island-resort.com> <05798259-49C4-4442-84A3-9FD13D0BDB5B@tzi.org>
Message-ID: <cbc320f27386e06a674a02c0f2f1b6e6@mothers-arms.co.uk>
X-Sender: kio@mothers-arms.co.uk
User-Agent: Roundcube Webmail/1.3.16
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/GvzGJsYt8WwZzQc7r_GFVYeLkXA>
Subject: Re: [Cbor] hildjj/cbor-map-entries: Explicit Map datatype for CBOR, in array format
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Feb 2021 19:51:15 -0000

I can give an example in Python... 

Tuples (among other types) with immutable values can be used as dict
keys: 

   {('a','b'): 'value'} 

But when serialized the tuple is converted to a CBOR array: 

A1               # map(1)
   82            # array(2)
      61         # text(1)
         61      # "a"
      61         # text(1)
         62      # "b"
   65            # text(5)
      76616C7565 # "value" 

Therefore the cbor2 decoder implementation has to know that it is being
called in a context where a hashable datatype is required and in this
case return a Tuple instead of the normal mutable List object. 

The test cases for "exotic map keys" can be found here: 

https://github.com/agronholm/cbor2/blob/818ba624df535ac7cd300aaa6fb963a83b3656e0/tests/test_encoder.py#L457


The intended use case is not for a robust wire protocol, since it would
as you say make interoperability extremely difficult, but it does allow
flexible serialization of complex objects. The Python Pickle module is
the model we are following. 

https://docs.python.org/3/library/pickle.html 

Cheers, 

Kio 

On 2021-02-18 17:52, Carsten Bormann wrote:

>> On 2021-02-18, at 17:48, Laurence Lundblade <lgl@island-resort.com> wrote:
>> 
>> This is semi-related and maybe something I'm a little confused about.
>> 
>> My understanding is that map keys/labels can be anything in CBOR including a map or array.
> 
> Yes.  
> 
> Or a Tag!
> 
>> What to do?
>> 
>> - Don't standardize any IETF protocols that use exotic map keys?
> 
> Yes.
> 
> (I don't know what's exotic, but on the implementation side generally any data structure that is mutable is a bit ugly as a map key.  E.g., languages where strings are mutable already need to jump through extra hoops here.  But, e.g., C++ handles maps with non-trivial keys just fine; these are certainly not exotic at all to a C++ programmer.)
> 
>> - Add text to Kio's proposal to restrict map keys to text string, byte string and integer?
> 
> No.
> 
> (E.g., think what https://tools.ietf.org/id/draft-ietf-cbor-tags-oid-02.html#name-distinguished-name-in-cbor- would look like without tag factoring.)
> 
>> - Write some warning against this somewhere?
> 
> Should be obvious to anyone skilled in the art :-)
> 
> (The reason CBOR can do this is that many platforms can do it -- so not supporting it in the format would have made CBOR less useful for native code/protocols for specific platforms.)
> 
> Grüße, Carsten
> 
> _______________________________________________
> CBOR mailing list
> CBOR@ietf.org
> https://www.ietf.org/mailman/listinfo/cbor