Re: [Cbor] eliding CBOR diagnostic notation binary contents

Michael Richardson <mcr+ietf@sandelman.ca> Mon, 14 March 2022 16:02 UTC

Return-Path: <mcr+ietf@sandelman.ca>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 01C043A0163 for <cbor@ietfa.amsl.com>; Mon, 14 Mar 2022 09:02:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.109
X-Spam-Level:
X-Spam-Status: No, score=-2.109 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=sandelman.ca
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TwmPV0LmrQbE for <cbor@ietfa.amsl.com>; Mon, 14 Mar 2022 09:02:43 -0700 (PDT)
Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 09CA83A043E for <cbor@ietf.org>; Mon, 14 Mar 2022 09:02:25 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by tuna.sandelman.ca (Postfix) with ESMTP id 0A45A38AD9; Mon, 14 Mar 2022 12:12:07 -0400 (EDT)
Received: from tuna.sandelman.ca ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with LMTP id oAtKNew9qdu7; Mon, 14 Mar 2022 12:12:03 -0400 (EDT)
Received: from sandelman.ca (obiwan.sandelman.ca [209.87.249.21]) by tuna.sandelman.ca (Postfix) with ESMTP id 44A9738AD7; Mon, 14 Mar 2022 12:12:03 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=sandelman.ca; s=mail; t=1647274323; bh=+YUXlkz7GcHJDbStu2BCr4NAAacGEStCmoFRUz63H7Y=; h=From:To:Subject:In-Reply-To:References:Date:From; b=M96xVZiFbeJkqljlZ09XKvTab0kdmaQQc7/1A7w/69yly4Ec+0FkIktGIZLl/2mGk IOgrvc7ljucWtIA3lkcsKNTc9LY/Zq6qQke0ouQcBbtmX6tLeCmAAyak1IJtouSMRP 2Fd3kpn92FdGRPz0f8nL04Kb2Xs2Q+5u7VsCZpnTJ2opdAPkvmf+p85Bagvz2N0BwD REtb9UFvOud9R3QyWhQP0QBa/1OarZxxRQNpzaBLxy8ejlXFoErEX5Uiqk2uIi/BfI R98U5sZgjY25fvo9Pht25zUlwUpucnPrISXzLax8noNj5HcZiVKh+kpP1FUqClQUnW wJqV/JgUZ4B5Q==
Received: from localhost (localhost [IPv6:::1]) by sandelman.ca (Postfix) with ESMTP id A2BBE26E; Mon, 14 Mar 2022 12:02:17 -0400 (EDT)
From: Michael Richardson <mcr+ietf@sandelman.ca>
To: Carsten Bormann <cabo@tzi.org>, cbor@ietf.org
In-Reply-To: <980C5C89-A934-4AA5-B234-037057C004C9@tzi.org>
References: <289545.1643998363@dooku> <982B35D9-8541-4AB6-88DE-55018DB5BB6B@tzi.org> <27168.1644006597@localhost> <D6BA4596-C08F-4CDF-AEB4-20C264D3BB34@tzi.org> <YgO2eOiH5gMdwSGd@hephaistos.amsuess.com> <13A46D26-E131-42C8-B325-C57857246699@tzi.org> <YhY8jT6/APLawwBF@hephaistos.amsuess.com> <9709.1645661896@localhost> <83575A66-894D-4013-9E89-AA3E0DB4641B@tzi.org> <30798.1645665697@localhost> <5FA0EAE5-2E02-4968-847D-8F516C2FCC65@tzi.org> <26078.1647206969@localhost> <980C5C89-A934-4AA5-B234-037057C004C9@tzi.org>
X-Mailer: MH-E 8.6+git; nmh 1.7+dev; GNU Emacs 26.1
X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m<nPbLgmtKK-5dC@#:k
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature"
Date: Mon, 14 Mar 2022 12:02:17 -0400
Message-ID: <21472.1647273737@localhost>
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/5F83JqPpV3bnybr-znPjWnuLHlE>
Subject: Re: [Cbor] eliding CBOR diagnostic notation binary contents
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 14 Mar 2022 16:02:49 -0000

Carsten Bormann <cabo@tzi.org> wrote:
    >> Signed PGP part
    >>
    >> Carsten Bormann <cabo@tzi.org> wrote:
    >>>>> Actually processing elisions should be a special flag.  That should
    >>>>> address your concern.
    >>>>
    >>>> Yes, that works for me.
    >>
    >>> In cbor-diagnostic, this flag is now “-ah”.
    >>
    >> Cool.
    >>
    >>> $ gem install cbor-elision
    >>
    >>> $ echo "h'...4711...'" | diag2pretty.rb -ah
    >>
    >> The direction I really want to automate is cbor2pretty, when generating examples.

    > Are you sure about cbor2pretty (annotated hexdump) vs. cbor2diag?

Uhm.  Both probably.  I'm often confused about which one is which :-)
I find myself using pretty often because it is easy to load them up as
reference values for unit test cases.

    > A hexdump is much harder to have elisions that you can read in again:
    > The length information implicitly conveyed by the syntactic structure
    > of diagnostic mode is missing.
    > So you don’t know how many bytes the elision is supposed to cover.
    > (Unless you start parsing the annotations…)

Agreed, but for showing some things the hexdump is necessary, but it is not
useful to show long sequences like signatures or embedded payloads.

    >> There are some ways in which I can see doing this:
    >> 1) some option which just elides any bstr longer than X.
    >> 2) some option which elides based upon some other (content-relevant) tag.
    >> 3) some option which takes some kind of CBOR-PATH to specify what to elide.
    >>
    >> I think that "CBOR-PATH" is not yet a thing, but I think I've heard you talk
    >> about how we can essentially use JSONPATH to do this?

    > Also, it might be useful to mix in a CDDL spec and build a path based on the parse tree.

yes, this would certainly be interesting: ".elidable" or some such attribute.

    > CBOR-Path should be able to handle all of these, but it still needs to
    > be invented (after JSONPath is done, presumably).

I am blissfully ignorant of JSONPath situation.

What do we do in the meantime?  I think that generating (diag format)
examples is the killer-app here.  Maybe even to the point where kramdown
should have some "```` foobar" for doing this.
I guess the CDDL matching part maybe.  I suppose this also validates the CBOR
against the CDDL along the way, which is a win, except when it's a pain.


--
Michael Richardson <mcr+IETF@sandelman.ca>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide