Re: [Cbor] hildjj/cbor-map-entries: Explicit Map datatype for CBOR, in array format

Joe Hildebrand <hildjj@cursive.net> Thu, 18 February 2021 19:58 UTC

Return-Path: <hildjj@cursive.net>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 95D6D3A17C8 for <cbor@ietfa.amsl.com>; Thu, 18 Feb 2021 11:58:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.697
X-Spam-Level:
X-Spam-Status: No, score=-1.697 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (public key: not available)" header.d=cursive.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ECS_CPbTEm9O for <cbor@ietfa.amsl.com>; Thu, 18 Feb 2021 11:58:27 -0800 (PST)
Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com [IPv6:2607:f8b0:4864:20::334]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0191D3A17C5 for <cbor@ietf.org>; Thu, 18 Feb 2021 11:58:26 -0800 (PST)
Received: by mail-ot1-x334.google.com with SMTP id s107so3002290otb.8 for <cbor@ietf.org>; Thu, 18 Feb 2021 11:58:26 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cursive.net; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=XRHf66J+j+I+h8Eib3nAecFuhEotnk7NRhwhsJ9Tbdo=; b=ZX9JXvNlpq7yksRa9G2azg3dBnuZex1SvEslUM0bqvGv/W9CeOUKA7JA1THaO4fKZJ SizZcvG1bGzzyAWGIv8+l/9vLBX8JSNDFmmJLgCbZkydlR3MPCO9p6FoOP2WGw+75m9+ UFoQyxn1Prry/wpEQkwzgKywIosuY6TWgw5V8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=XRHf66J+j+I+h8Eib3nAecFuhEotnk7NRhwhsJ9Tbdo=; b=G1xGrI28qk6KpMckMetFerHAJ44QhfVBp4+4J1plcsGSGJ2KUDZ8/4uExFhGMERZwY SRUnBJvKdXWw/HasJnCUoIPtKn0WduMBjOl5X9Uoa7en4qc1Ovm+DrzqsNIJb7OqYioe IOmhuPVjju/w1Fmodys1H0hBI2ixIOVsolxzy2KtbBKOJ6VWuJwVGYaiCKKxRXJXYVwV Js4qeESqgqsXPZ3yp4pDyIEALO8h3dFWug+VjXwljylRLsYN5oiOCRSKZa5FF0J5DJwP zjf/V7nB7UpUJNw/SzhwRITdjpxqJfMZCMKX463HIWY2mBvyWWH8oalsUkYOExjbkQ3M z+Wg==
X-Gm-Message-State: AOAM533UM/Br47pCkOnerSm46GKgUkxaHqh7E3ZArOO9V+FOCPJffFCc RU36y6VsF2UK3JxhziFAjpsejg==
X-Google-Smtp-Source: ABdhPJwo11ht68IXuB9nZP75womlAxeYlhyhyZI2BR2Wjs8ZXcxw07O7Eh54/qSHyd3wmId5cQVbyw==
X-Received: by 2002:a9d:1717:: with SMTP id i23mr4259170ota.179.1613678306102; Thu, 18 Feb 2021 11:58:26 -0800 (PST)
Received: from ?IPv6:2601:282:200:3758:7479:b77e:8b77:b5d5? ([2601:282:200:3758:7479:b77e:8b77:b5d5]) by smtp.gmail.com with ESMTPSA id z18sm1411294oic.1.2021.02.18.11.58.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 18 Feb 2021 11:58:25 -0800 (PST)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\))
From: Joe Hildebrand <hildjj@cursive.net>
In-Reply-To: <cbc320f27386e06a674a02c0f2f1b6e6@mothers-arms.co.uk>
Date: Thu, 18 Feb 2021 12:58:23 -0700
Cc: Carsten Bormann <cabo@tzi.org>, Laurence Lundblade <lgl@island-resort.com>, "Dale R. Worley" <worley@ariadne.com>, cbor@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <2631BC6B-F0D2-4756-B129-1B52F18EB6B1@cursive.net>
References: <87zh02kpf5.fsf@hobgoblin.ariadne.com> <0faeb37c36d5b1f40c37f82e62be1be9@mothers-arms.co.uk> <ED6F6174-2643-4B7F-8ED7-414DD5FD9C39@tzi.org> <F6BDBA57-34F4-46FE-9806-6A227AC214EA@mothers-arms.co.uk> <DC0D444C-3231-4029-B0BB-5EC02863B72A@cursive.net> <999B078D-C4B2-4B04-8487-7ECEFEB953DF@cursive.net> <94bbc990fcf388ae5fd01c06274ed58a@mothers-arms.co.uk> <72053102-1BA1-4281-9F17-CF7A65EEA907@island-resort.com> <05798259-49C4-4442-84A3-9FD13D0BDB5B@tzi.org> <cbc320f27386e06a674a02c0f2f1b6e6@mothers-arms.co.uk>
To: Kio Smallwood <kio@mothers-arms.co.uk>
X-Mailer: Apple Mail (2.3654.60.0.2.21)
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/a7A6uH1fKQH-pOL_K25mlYF1BlE>
Subject: Re: [Cbor] hildjj/cbor-map-entries: Explicit Map datatype for CBOR, in array format
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Feb 2021 19:58:29 -0000

Oh, that reminds me.  If https://github.com/tc39/proposal-record-tuple progresses a little bit more, I'm also going to be interested in explicit mutable and immutable tags.

Putting an immutable tag around a map key would make Kio's example very clear.

— 
Joe Hildebrand

> On Feb 18, 2021, at 12:51 PM, Kio Smallwood <kio@mothers-arms.co.uk> wrote:
> 
> I can give an example in Python...
> 
> Tuples (among other types) with immutable values can be used as dict keys:
> 
>    {('a','b'): 'value'}
> 
> But when serialized the tuple is converted to a CBOR array:
> 
> A1               # map(1)
>    82            # array(2)
>       61         # text(1)
>          61      # "a"
>       61         # text(1)
>          62      # "b"
>    65            # text(5)
>       76616C7565 # "value"
> 
> Therefore the cbor2 decoder implementation has to know that it is being called in a context where a hashable datatype is required and in this case return a Tuple instead of the normal mutable List object.
> 
> The test cases for "exotic map keys" can be found here:
> 
> https://github.com/agronholm/cbor2/blob/818ba624df535ac7cd300aaa6fb963a83b3656e0/tests/test_encoder.py#L457
> 
> The intended use case is not for a robust wire protocol, since it would as you say make interoperability extremely difficult, but it does allow flexible serialization of complex objects. The Python Pickle module is the model we are following.
> 
> https://docs.python.org/3/library/pickle.html
> 
> Cheers,
> 
> Kio
> 
> On 2021-02-18 17:52, Carsten Bormann wrote:
> 
>> 
>> 
>>> On 2021-02-18, at 17:48, Laurence Lundblade <lgl@island-resort.com> wrote:
>>> 
>>> This is semi-related and maybe something I'm a little confused about.
>>> 
>>> My understanding is that map keys/labels can be anything in CBOR including a map or array.
>> 
>> Yes.  
>> 
>> Or a Tag!
>> 
>>> What to do?
>>> 
>>> - Don't standardize any IETF protocols that use exotic map keys?
>> 
>> Yes.
>> 
>> (I don't know what's exotic, but on the implementation side generally any data structure that is mutable is a bit ugly as a map key.  E.g., languages where strings are mutable already need to jump through extra hoops here.  But, e.g., C++ handles maps with non-trivial keys just fine; these are certainly not exotic at all to a C++ programmer.)
>> 
>>> - Add text to Kio's proposal to restrict map keys to text string, byte string and integer?
>> 
>> No.
>> 
>> (E.g., think what https://tools.ietf.org/id/draft-ietf-cbor-tags-oid-02.html#name-distinguished-name-in-cbor- would look like without tag factoring.)
>> 
>>> - Write some warning against this somewhere?
>> 
>> Should be obvious to anyone skilled in the art :-)
>> 
>> (The reason CBOR can do this is that many platforms can do it — so not supporting it in the format would have made CBOR less useful for native code/protocols for specific platforms.)
>> 
>> Grüße, Carsten
>> 
>> _______________________________________________
>> CBOR mailing list
>> CBOR@ietf.org
>> https://www.ietf.org/mailman/listinfo/cbor
> 
>