Re: [Cbor] Consolidated set of tags for map-like entities

Emile Cormier <emile.cormier.jr@gmail.com> Tue, 16 March 2021 17:52 UTC

Return-Path: <emile.cormier.jr@gmail.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EEB443A0553 for <cbor@ietfa.amsl.com>; Tue, 16 Mar 2021 10:52:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cP4DiAdNpYWr for <cbor@ietfa.amsl.com>; Tue, 16 Mar 2021 10:52:46 -0700 (PDT)
Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D5ECE3A047D for <cbor@ietf.org>; Tue, 16 Mar 2021 10:52:46 -0700 (PDT)
Received: by mail-pj1-x102f.google.com with SMTP id ha17so10281180pjb.2 for <cbor@ietf.org>; Tue, 16 Mar 2021 10:52:46 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pp+PNErXUtQHg9Gm0zPoALy5HFSZAeDiiTc1ZFZ1uNM=; b=c50mlAT9jIMUQdwcwsL5H8ls8mApJUcdbo2P9dyc4/hOJf+zc05oUJNeBI7fA1oArq NguQqiqWzZxwD/wH0BU0nh6DCaPKfZo4ip9N3dj12+uEAA3iTc1ya3L5hT2WMU7su80b OdTU4QIaVj/YcUtjpWrpmdbL3YWC4l81Ia7Bt6egymk+FH36XsMA8JOXBHDJ7fAWLp9y h7RBwQZLZfMRmu6wbaVrkiChBKg94y5VNjsTLvIpRDI/DHJdOCFrXUrjQDcceOZ4Qxqd JdGO6ZDt28VTI68Zv//HB46wlJYjpe0IrrRvs9CMPfD7SealfVV5uCxMib0HqOJaA2eo P1cQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pp+PNErXUtQHg9Gm0zPoALy5HFSZAeDiiTc1ZFZ1uNM=; b=YTXhWrD5qO28Rj74c6lUDuP9D+ZX8I/yBH4mCUh9PkKcm5nkKA8nJKwwZwlXuRxiZv khPsNV5uHGuHDRSKUHR91f0aNQMr88/laLBCMJjG6xywNGIKLpaNbYACQffQJMsNSTz7 HzZSXPvmS4/VdtYI0asmzN9NuQaMNi+JFDXmygyvt++ajBAu+LjprEyrj3FsgKQqzgiM GqsM/5wC/x5LficMa+uBlIv1nMe5Klre0O+g+dDV0Pvwr1HNX5JUADlqY1D0m6gcU2HE 86ECljMVZT/X+p/PRBdUxZIElDhrlSYwooHcRLfXvbLfLOkWDuJe2pavm9dT+zkYoOT/ RDXQ==
X-Gm-Message-State: AOAM531NSR+yfidqCN6oT2PXPaJLux8kpcn5xoPLlmsLVLHJDFzUdion Auc7WNT3BqAVTZMPQ8V2B+XMErCmG5RhTkox/CxBrz7Efj0=
X-Google-Smtp-Source: ABdhPJxQdqdybjw/pTkN7GZ9C03xtytxutrLZs4IsP8Pj0kcdioJorkcI0bEnvu3U+ARMTJu2YdOjXyU5DnJKQgjYGU=
X-Received: by 2002:a17:90a:5898:: with SMTP id j24mr252083pji.110.1615917166318; Tue, 16 Mar 2021 10:52:46 -0700 (PDT)
MIME-Version: 1.0
References: <CAM70yxC7o3CNz67Yx3-yLt3tPQaTRaezOO99Ssa-3ppu-NwD=Q@mail.gmail.com> <C2B0934A-2AAB-4A45-A354-407782B4CA0B@cursive.net> <FA5E389C-C577-4D42-A19E-A3A50F14F730@tzi.org> <16642498-C82F-41CE-A06D-6C762AEEB6F9@cursive.net> <CAM70yxC3px4U_aVv0yNNL8t-m+Vt3ftLKvUfhsCNHS2sqzM5Fg@mail.gmail.com> <1629FF2E-05D6-456E-AF3C-913F8CB3E55C@cursive.net> <CAM70yxBv5JwSCUML1mJdDssQwUhTOy2QJKFOsEQ+DppOsXbmyg@mail.gmail.com> <D855B6A2-573D-4F8D-8666-55BA5A7F4DA1@cursive.net>
In-Reply-To: <D855B6A2-573D-4F8D-8666-55BA5A7F4DA1@cursive.net>
From: Emile Cormier <emile.cormier.jr@gmail.com>
Date: Tue, 16 Mar 2021 14:52:35 -0300
Message-ID: <CAM70yxDKq-rj=vK+g2S08E7iWnhYWDM6MpoopnKQBYMx-g2B7Q@mail.gmail.com>
To: Joe Hildebrand <hildjj@cursive.net>
Cc: Carsten Bormann <cabo@tzi.org>, cbor@ietf.org
Content-Type: multipart/alternative; boundary="000000000000af57d005bdab08e4"
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/Q4XYw2XFuaVSRNbN76t4tV9Vm80>
Subject: Re: [Cbor] Consolidated set of tags for map-like entities
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Mar 2021 17:52:51 -0000

The benefit of using map instead of array whenever possible is better
interoperability with other decoders that ignore the tags. For example, if
I send a map tagged 132 (unordered, homogenous, unique keys), an "agnostic"
Javascript decoder will store it in a Map or Object, and the application
will at least know it's a set of key-value pairs.

The disadvantage of this approach is obviously the inconsistency. Sometimes
it's map and sometimes it's array. The decoder needs extra logic for using
the correct major type code.

On Tue, Mar 16, 2021 at 2:20 PM Joe Hildebrand <hildjj@cursive.net> wrote:

> Your approach makes sense to me.
>
> Is there any benefit to using map over array for your approach?  They're
> roughly the same on the wire.
>
> —
> Joe Hildebrand
>
> > On Mar 15, 2021, at 3:15 PM, Emile Cormier <emile.cormier.jr@gmail.com>
> wrote:
> >
> > Indeed, I try to feed bytes from the wire into its resulting data
> structure as directly as humanly possible in my C++ implementation, while
> not having to write decoders by hand for every possible data structure.
> While CBOR itself is schema-less, our applications use CBOR (and JSON) to
> transfer entities with known schemas at the application level. I can
> therefore skip generating intermediate "document" trees and store the CBOR
> directly into the intended data structure (this is the idea behind
> https://github.com/OneNoteDev/GoldFish, which is partly inspiring my
> work-in-progress C++ serialization library).
> >
> > I know I could use Flat Buffers with fixed schemas, but I'd end up
> having to copy everything to standard library containers unless I'm willing
> to have Flat Buffers "infect" my entire application instead of remaining at
> the boundary. I also like the self-describing nature of CBOR for diagnostic
> purposes, plus it's easier to transcode it into JSON for web browser
> clients.
> >
> >
> > On Mon, Mar 15, 2021 at 5:21 PM Joe Hildebrand <hildjj@cursive.net>
> wrote:
> > Ah, I think I see the difference in implementation that others must
> have.  I always generically decode the data item first, *then* apply the
> tag transformation.  This allows me to return a generic Tag implementation
> if I don't support the tag, or a more-specific type or error-check if I do
> support the tag.
> >
> > Others must do special-case decoding of the data item for each tag they
> support.
> >
> > —
> > Joe Hildebrand
> >
> > > On Mar 15, 2021, at 2:09 PM, Emile Cormier <emile.cormier.jr@gmail.com>
> wrote:
> > >
> > > I don't know anything about "mt5", but with my proposed tag set, you
> could check for the presence of tag 132 (homegenous key) and you would know
> after decoding the first key if all the key-value pairs can be safely
> decoded into a JS Object. Of course, this requires that the sender "play
> along" and actually bothers to use the tags.
> > >
> > > On Mon, Mar 15, 2021 at 4:44 PM Joe Hildebrand <hildjj@cursive.net>
> wrote:
> > > > On Mar 15, 2021, at 12:51 PM, Carsten Bormann <cabo@tzi.org> wrote:
> > > >
> > > > On 2021-03-15, at 19:50, Joe Hildebrand <hildjj@cursive.net> wrote:
> > > >>
> > > >> Data Item being major type 5 should really only be used if the keys
> are homogeneously strings, in my opinion.
> > > >
> > > > We are using a lot of integer labels with mt5, so why would we make
> that restriction?
> > >
> > > This is a consequence of my implementation that others probably don't
> have.  When decoding mt5, I always keep a set of [key,value] tuples until
> all of the items have been decoded.  Then, if the keys are all strings, I
> convert the tuples to a JS Object, otherwise I convert them to a Map.  If
> I've already converted to a Map in mt5 processing, it's too late to (for
> example) throw an error if there is an unexpected duplicate key.
> > >
> > > —
> > > Joe Hildebrand
> >
>
>