Re: [Cbor] Considerations for converting from other serializations formats like Protocol Buffers and FlatBuffers

Jonathan Beri <jmberi@gmail.com> Mon, 13 April 2020 15:30 UTC

Return-Path: <jmberi@gmail.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4356E3A1801 for <cbor@ietfa.amsl.com>; Mon, 13 Apr 2020 08:30:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.086
X-Spam-Level:
X-Spam-Status: No, score=-2.086 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dx5D0MKkSJiu for <cbor@ietfa.amsl.com>; Mon, 13 Apr 2020 08:30:16 -0700 (PDT)
Received: from mail-ua1-x933.google.com (mail-ua1-x933.google.com [IPv6:2607:f8b0:4864:20::933]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6E4473A1812 for <cbor@ietf.org>; Mon, 13 Apr 2020 08:29:56 -0700 (PDT)
Received: by mail-ua1-x933.google.com with SMTP id g10so3242233uae.5 for <cbor@ietf.org>; Mon, 13 Apr 2020 08:29:56 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jc7iYndLaYVa7VMPbMtrhwmBmvfb4qJwSMXPh23EiWI=; b=bg8nzIEoRzSPt5Ayeo5yFL59nhQiWkIBOf2W/M/EYbFrURolTjVRK8+II8on4JeFBL +0s6N5N0xbcpIR+zlGmcDS+2luLoi2XXcraRgNANxNboXEVFccQgIYEPY6aSR4Aaga7y z+/eyL9GvLt/aDUuA1vIYJJb+Qb+E6x5oYFHY0IX3T8iRgBN6bhvbdTzKCgzXvbMOYlN 1vy30T9JvpY4c0agLkm4JPmyS5EjLBrZVGAg6p+TB1wLHjmyjUoVXJ2zQLw7ZiUsv9sa EMpMmGHa63i9xziO2mgOg1VvLYmxc6EQuZoAE4LgEtwHjkAw1U3SaKVsOd7bCAx9RVhF w/Ww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jc7iYndLaYVa7VMPbMtrhwmBmvfb4qJwSMXPh23EiWI=; b=Jeo6vP/p/5TO7UDqcX26JZIgYU/wgNaPy3w3mpzwHzBb5PIMQD6UMtZ/wm+BOwhavn THAVt5fgYV7Fa01dF4OxJvOOYQxb2Nmcrf6JQcT3lb5th6n0N9Fvme5OpAvLC+6OCZV9 EWCAZORcKTLi9kUpRq3gNAKX+1BMPPReePJpq109E1WiJ4QNJCTeSZhQZu/8bJxWjeQ9 mkZvNUDYQe1fhJBrStYNLXQYc8G8Y3Zjum0Pmli78uNLIdqQ3IsL+JgkIQlBe8lqYjjz r/crotYNe3S5aHYDQxN/Mq+x5kwUohWilAuHyfJclyy5zQlQHPoav5FDYn10dnUF2ez+ 5F/Q==
X-Gm-Message-State: AGi0Pua30Lnd8eeeHn8S32LIVFovIVGjH4pqm9mWtdB0DANoxMfRwfL3 lPEKbwTxOwBxWpQUmNJxxJ7a6AXYIZsbUq6vluyphy6C
X-Google-Smtp-Source: APiQypINuSQ3iqBVuMj9I5gA9dNgBV6F8f1HrUb04LaRU99dAwYSDpbClu97vyDu9FC+YmyYmx2xQPLiR/G3gB5ha6c=
X-Received: by 2002:ab0:74cd:: with SMTP id f13mr11874729uaq.111.1586791789957; Mon, 13 Apr 2020 08:29:49 -0700 (PDT)
MIME-Version: 1.0
References: <CANcmUPHvT+o4uckLOTH54h=ZS3pubhyeQ4guxbmdRcVkC91mWA@mail.gmail.com> <F545572B-900C-4FCB-8D7D-1961242AEA8F@tzi.org>
In-Reply-To: <F545572B-900C-4FCB-8D7D-1961242AEA8F@tzi.org>
From: Jonathan Beri <jmberi@gmail.com>
Date: Mon, 13 Apr 2020 08:29:13 -0700
Message-ID: <CANcmUPGrr=7HeUpam_G+W+FtsaFY2o4gWef93GP=hDZRA60EKA@mail.gmail.com>
To: Carsten Bormann <cabo@tzi.org>
Cc: cbor@ietf.org
Content-Type: multipart/alternative; boundary="000000000000f916c605a32dc07e"
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/vJEnHWITzAhrQiyx9er7KZhNKJg>
Subject: Re: [Cbor] Considerations for converting from other serializations formats like Protocol Buffers and FlatBuffers
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Apr 2020 15:30:27 -0000

Very helpful, thank you Cartsen! I just posted to the T2TRG.

On computation - I've also noticed formats that weren't originally designed
cross the open internet lack a framework for signing and encryption, like
JOSE & COSE. I don't immediately see challenges when going between X and
CBOR, especially if X doesn't define S&E. Anything that comes to mind?

On Fri, Apr 10, 2020 at 8:59 AM Carsten Bormann <cabo@tzi.org> wrote:

> Hi Jonathan,
>
> Such a conversion is always “interesting" because of the different generic
> data models of the different data formats.
>
> CDDL essentially is used to define a subset of the generic data model of
> CBOR.  It also can “parse” or “construct", so it could be used as a
> front-end/back-end to such a conversion format, but we would need a
> Format-X back-end/front-end as well.  And some of this will be idiomatic
> (e.g., very idiomatic in JSON), so you don’t want to need to say the
> idiomatic things all the time.  Some conversions will even need
> computations (say, a JSON enum into a CBOR bit string).
>
> We have done something related to a conversion process (uncompressed vs.
> compressed) in RFC 4997.  This notation was also saddled with needing to do
> its own binary decoding/encoding (which still would be useful as another
> front-end/back-end of the above!), but the idea of having statements that
> connect on two sides to two different worlds is immediately applicable.
> SCHC <
> https://tools.ietf.org/html/draft-ietf-lpwan-ipv6-static-context-hc-24>
> is a contemporary attempt to do a bidirectional conversion, but it is
> somewhat specialized to “packet”-type data.
>
> Note also that T2TRG has done a little work on “YOUPI” <
> https://tools.ietf.org/html/draft-petrov-t2trg-youpi-01>, which is
> another way to look at binary data.
>
> As you can see, I think some of this is research, and it borders on the
> WISHI/One-data-model work that we are doing in T2TRG.  I would really like
> it if you could bring in your ideas there.
>
> Grüße, Carsten
>
>
>
>
> > On 2020-04-10, at 17:34, Jonathan Beri <jmberi@gmail.com> wrote:
> >
> > Hello,
> >
> > I'm looking into a use case that would require converting data from
> existing systems that output in well-defined serialization formats
> (specifically Protocol Buffers and FlatBuffers) to CBOR. The goal is in
> part to use other specifications like CoAP with this data as as well an
> existing implementation of CoAP.
> >
> > Has there been any investigation into converting to these formats? Any
> lessons or points to consider? I've been reading up on CDDL - might that
> help in implementing a conversion process?
> >
> > Many thanks.
> >
> > --
> > Jonathan Beri
> > linkedin.com/in/jonathanberi
> > _______________________________________________
> > CBOR mailing list
> > CBOR@ietf.org
> > https://www.ietf.org/mailman/listinfo/cbor
>
>

-- 
Jonathan Beri
linkedin.com/in/jonathanberi <https://www.linkedin.com/in/jonathanberi/>