Re: [Cbor] Record proposal

Kris Zyp <kriszyp@gmail.com> Thu, 11 November 2021 15:35 UTC

Return-Path: <kriszyp@gmail.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8778D3A08D5 for <cbor@ietfa.amsl.com>; Thu, 11 Nov 2021 07:35:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Level:
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q5pAZrkBKgnt for <cbor@ietfa.amsl.com>; Thu, 11 Nov 2021 07:35:48 -0800 (PST)
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D4F413A0C75 for <cbor@ietf.org>; Thu, 11 Nov 2021 07:35:46 -0800 (PST)
Received: by mail-ed1-x533.google.com with SMTP id z21so25700327edb.5 for <cbor@ietf.org>; Thu, 11 Nov 2021 07:35:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=5lLiAjMT+2bHtqFCRp36tZ3lPAi7EFCMcXEPpSp2bFQ=; b=KOPC5nt+/MWt5Im2vFGzVakKJjcOCLzz34ay+aRj/TvTq/rikM4SggxRKnSNiJkgMJ 8Ei9mo8rNFDew0zlQAdGyaLPAxB3eash6bWrC802n8ImhrjvdVaBY+cOsaJBedw/MGcX cRl9Zt9/q7yc5VSLUerJthUg6y35o48YEIvjb5tnf1s1uRFihzvi15cV/WvOwz2MriWa pur3xGULKjDaxRsIiyy/WY0jufAkOfniBlQ1Pjg38EEPGCyUZL6tTQs6N0UIZ24FrG6D XLlyHuClIDfAS8L9jMvf97useIPzhO5DkzEOYbvuAVBerTm+0k1Gplu55lLeQxnzan/P p7SQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5lLiAjMT+2bHtqFCRp36tZ3lPAi7EFCMcXEPpSp2bFQ=; b=cQ0O/qceLymPg5qSmFgVud/Vmz2XmuK31M8n/sgWP6ERPPyZBr3v5J6/LJ3i6yhv7P Iao/UwC8aD18TTZn20o60oY7u8lCP5tbiLScBs3cHHj+39ntZdvRNoEYMhf2hLXK0TBy oyX+3ou8VHrTxrSH2LGxdBncawrFx+s9fh0xpXx+H6tGqtjHRXYt4SRzTHBLGSdo65Mm HpHNWIUcaAO8cvvlR9JOeSViSpF5b5kCnzY+qWPunIXK+z1oF/sAmOYsPsZzR7HB5OuR MlqNxNBoBJzTs0WobqsZ/GOnK8pImaoxvohDNft/h7yrb9pcg4AQiOax1PnhIgs7GaCh irTA==
X-Gm-Message-State: AOAM531uQch+SbxATAP4hN763e/M0+C/oym9Xht7JueAnc5xxXPU/Uzg h67mNCybkG34vj5YcA5q6+Oaw5ateWjuBb1v/vXPQtnI7C0=
X-Google-Smtp-Source: ABdhPJxveBwnP9ZbpE1yaPWzttdAhSp0vv39AtqJE3m7LJJSP5ZcVUNcY2NfRD+JAkouRpLMAtt4xb3MguQn3BWz244=
X-Received: by 2002:a17:907:9056:: with SMTP id az22mr10394334ejc.107.1636644943948; Thu, 11 Nov 2021 07:35:43 -0800 (PST)
MIME-Version: 1.0
References: <CAEs2a6vNhrHhaiPUNtbJ68WYfbrprETPr+kmWNJgNXMSawyBig@mail.gmail.com> <E3F121DA-95EE-43C6-BC72-E3763C034944@tzi.org> <CAEs2a6uZrT9FFP6qa+hPV2sYO0y+xJJmLaF-pPoynE2vqspfBg@mail.gmail.com> <CAEs2a6uHyvvghAMCN=UmhpJMoiES7zoPmGi-bATZWXgjA068Mg@mail.gmail.com> <YY0B4YxuMuw20umu@hephaistos.amsuess.com> <CAEs2a6utA=GQSx2Ln=5wnoNdS6z+0ExdCcfNXG6cAg=1MxnT=w@mail.gmail.com> <901541DC-A520-44CD-AA8D-F2CE77F03FA0@tzi.org>
In-Reply-To: <901541DC-A520-44CD-AA8D-F2CE77F03FA0@tzi.org>
From: Kris Zyp <kriszyp@gmail.com>
Date: Thu, 11 Nov 2021 08:35:32 -0700
Message-ID: <CAEs2a6sZd4s-DJ3R_M4BLwO12s8i2AGfv0yXCaWdy+baOuAEqw@mail.gmail.com>
To: Carsten Bormann <cabo@tzi.org>
Cc: =?UTF-8?Q?Christian_Ams=C3=BCss?= <christian@amsuess.com>, cbor@ietf.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/gh_V2kJKe4mbzGQxVRvi4D3PPSI>
Subject: Re: [Cbor] Record proposal
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 11 Nov 2021 15:35:54 -0000

Thank you for the explanation of your allocation strategy, really
appreciate it, that makes sense. It does seem a little surprising to
me that some of the 1+1 tags that are explicitly for very specific
technologies (Perl, YANG, Internet of Data things, etc.) would be
considered more "common" than a record structure designation, a
concept that I think is broadly used across many/most programming
languages. And it seems like many of these registrations would not
have a greater need for being short than a tag that specifically
yields significant and broad opportunities for dramatically improving
encoding efficiency/compactness (I don't know if you are interested in
some stats on that?). I would certainly argue that the broad semantic
application and the significant encoding efficiency benefits would
justify a 1+1 tag for a tag record. But if only 1+2 tags are being
permitted or strongly preferred, I can certainly adjust to that
criteria. If I resubmit with a 1+2 tag would that be easier for
registration, and would it be possible to have it submitted with
possible future consideration for upgrade to a 1+1 tag?

> ...requiring data in the first record that also usually sets the base configuration...
> (In cbor-packed, we put the setup *around* the data — another variant that you might want to look at.)

It seems like the significant problem with using an "around" type of
structure is that it requires an a priori knowledge of the structure
of the data that will be encoding prior to beginning encoding. IMO, I
don't believe this is a constraint that should be enforced on
encoders. This constraint has implementation implications that can
significantly degrade performance (since encoders that are unaware of
the structures that will potentially be serialized, must "discover"
these prior to beginning encoding). This also can preclude many valid
use cases for encoding data reusable structures that are not fully
known prior to encoding (perhaps data is still being read from a
database, I/O, or other sources and may include various data
structures yet to be known). This seems like a particularly
problematic constraint, considering that it seems like CBOR has a very
smart and prescient design for facilitating the serialization of
"streaming" data with indefinite length arrays which are are extremely
valuable for situations when encoding can begin on data that is
streaming into an encoder (not all known ahead of time). (And
"streaming" is specifically mentioned as an intended use case in RFC
8949)

I had actually also been meaning to ask how this type of streaming of
data might work with the combination of cbor-packed and
cbor-sequences. It seems like a particularly valuable use case for
packed CBOR is in being able to create a socket for sending a sequence
of CBOR messages (between two remote hosts, for example), and sending
a packing table with values that are/will be frequently used in the
stream/sequence of CBOR messages. However, I am not sure if this is
actually possible using standard CBOR sequences and packing tables
because a packing table is wrapped around a single CBOR rump/value,
can't be used to indicate that the packed values can be referenced by
future CBOR items in a sequence? I know CBOR packed explains that
packed values can be defined at an application level, so perhaps this
simply requires an out-of-band or non-standard mechanism for
communicating that a set of packed values can be referenced by
messages in a CBOR sequence?

Anyway, thank you again for the update and explanation and all your
efforts, really appreciate it! And if you believe that resubmitting
this tag registration with a 1+2 tag (probably 29299 based on your
ascii suggestion) would be the best course for getting registered, I
can proceed with that.

Thanks,
Kris