[Cbor] Packed CBOR review

Brendan Moran <Brendan.Moran@arm.com> Wed, 16 June 2021 09:22 UTC

Return-Path: <Brendan.Moran@arm.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 66AE33A0C30 for <cbor@ietfa.amsl.com>; Wed, 16 Jun 2021 02:22:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=armh.onmicrosoft.com header.b=hjiGxspD; dkim=pass (1024-bit key) header.d=armh.onmicrosoft.com header.b=hjiGxspD
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h6eeBAjcTgYr for <cbor@ietfa.amsl.com>; Wed, 16 Jun 2021 02:22:10 -0700 (PDT)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2064.outbound.protection.outlook.com [40.107.22.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EEE813A0C2F for <cbor@ietf.org>; Wed, 16 Jun 2021 02:22:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=95eOSXxMST2/qFwwBX4grFcnmSdfUQQPeVv2Q9ge8vg=; b=hjiGxspDLjWLmXvTPVFMEaLIqKijAwaQSJQyP01VSkJHKTqSqIROnSatLv0xVG/6jMlWT9xZ/mALMDg+LoP5MJrr7bxQWu78BPUcMFsyiCoX0/5PBuIF20VS0JlQi9VNcz8UpmyQjVNaGSSDz/GN+USFgGW/kQ539KNsZNYmSTY=
Received: from AS8PR04CA0151.eurprd04.prod.outlook.com (2603:10a6:20b:331::6) by AM5PR0801MB1988.eurprd08.prod.outlook.com (2603:10a6:203:48::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.15; Wed, 16 Jun 2021 09:22:06 +0000
Received: from AM5EUR03FT027.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:331:cafe::84) by AS8PR04CA0151.outlook.office365.com (2603:10a6:20b:331::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.18 via Frontend Transport; Wed, 16 Jun 2021 09:22:06 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; ietf.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;ietf.org; dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT027.mail.protection.outlook.com (10.152.16.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.16 via Frontend Transport; Wed, 16 Jun 2021 09:22:05 +0000
Received: ("Tessian outbound cdfb4109116c:v93"); Wed, 16 Jun 2021 09:22:05 +0000
X-CheckRecipientChecked: true
X-CR-MTA-CID: 4fcf5aa42e84ac3e
X-CR-MTA-TID: 64aa7808
Received: from 4d208d83bf4e.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id AB0E89F2-6EF3-4AEE-9A93-A44A7213D3CE.1; Wed, 16 Jun 2021 09:21:49 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4d208d83bf4e.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 16 Jun 2021 09:21:49 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Y/x4jVYYLYe0MPR94SoFnCzHqYOZmMJVTfdBFcP/xNVys3/8F0BKdQ1CE67SXVbCwWMKr2sVTcN6adAWcHZS9KrKdPee26jTyhF/AKK8iVrPTL47aNeSGnsarNjkRSY12NdIG0Th4H7HvEwe0WN6CSM/h3jiB8Wbwld8ryIhS4iJqBcwKXbOAQ8i7lx/NCMNm/c+HN568RclCQ6+VimMCKptuko7ozuJflqwtm1CTXfPMykyCOOQIRm9KCe1hzY/aVVwll3N49lAIPRyaMe2B5ucyP81UXOLLPS3bofqydaZt8cVAvPj2VOymhIVGLtmxxpFIGKY13FHqVUndhnizQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=95eOSXxMST2/qFwwBX4grFcnmSdfUQQPeVv2Q9ge8vg=; b=A+X8bOk0RwgXuZEj9Um2RfxFJRJ00+8ZBc6/EBEs+VdQaFso7ZlY2CXLX1YJ3/GCzsqUuoh73Z2YZabEyR31NwFIvTNKIU+h/bWlzs2eFECTbc3WbE+ua1eGL0V3mmvI47k2hFATL0zbFBm9XVdoKMlfs5FhhIt+h0iTTVzrgEMvKLME29ZV68m24GXEb3/XURN95G8yUE1OlsdWV75Cr84NeQRdhnVP74eq5Prr0u7WmYvTeYa8SClqb7mJipB4hiB4StuWodhAM1ll0CJzbTQ4G/+GqvjubgPqD5IduHfQR5ZGduQnI7B6NOJNpj4hUVI/PqH0+Hv/yiCvtVSi0Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=95eOSXxMST2/qFwwBX4grFcnmSdfUQQPeVv2Q9ge8vg=; b=hjiGxspDLjWLmXvTPVFMEaLIqKijAwaQSJQyP01VSkJHKTqSqIROnSatLv0xVG/6jMlWT9xZ/mALMDg+LoP5MJrr7bxQWu78BPUcMFsyiCoX0/5PBuIF20VS0JlQi9VNcz8UpmyQjVNaGSSDz/GN+USFgGW/kQ539KNsZNYmSTY=
Received: from DBAPR08MB5576.eurprd08.prod.outlook.com (2603:10a6:10:1ae::11) by DBBPR08MB6153.eurprd08.prod.outlook.com (2603:10a6:10:20e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4219.22; Wed, 16 Jun 2021 09:21:47 +0000
Received: from DBAPR08MB5576.eurprd08.prod.outlook.com ([fe80::3487:4e34:2e16:5521]) by DBAPR08MB5576.eurprd08.prod.outlook.com ([fe80::3487:4e34:2e16:5521%3]) with mapi id 15.20.4219.025; Wed, 16 Jun 2021 09:21:46 +0000
From: Brendan Moran <Brendan.Moran@arm.com>
To: "cbor@ietf.org" <cbor@ietf.org>
Thread-Topic: Packed CBOR review
Thread-Index: AQHXYpEHVGMWvsNpNkS/R6f99TIMjw==
Date: Wed, 16 Jun 2021 09:21:46 +0000
Message-ID: <8713C3AB-71C0-4EC0-8977-15F80EC11309@arm.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.100.0.2.22)
Authentication-Results-Original: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=arm.com;
x-originating-ip: [80.7.184.196]
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-Correlation-Id: fa454ccc-4c19-4ce1-8035-08d930a8354b
x-ms-traffictypediagnostic: DBBPR08MB6153:|AM5PR0801MB1988:
X-Microsoft-Antispam-PRVS: <AM5PR0801MB19880E758AF9FB207F28E9AEEA0F9@AM5PR0801MB1988.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original: 70COaduRsYAV9G4RgctqY4kpxIoYhKZ7qURnxbXFpQRZjmnqQiZUjIDoEQv05Vsc7w//xwTYu1SMoNjTH1u//EHtgxc7+wGv798gcx1hgaSEljyHEeKJH9Brsfg+TQ8Ck0D5sRfkVI38za+WctVxXb+E/Uzp4aLgWFkVTi7HP/Dmq3Pe+8dReitTJj0BfSSUQ/zNGmhSKEdTRTahWJJZYlikemxm2U+I+B53l9iq5mP+43TS/lpbiK9OU/BnjJWQHvYXfTEN4IDJCL9TCDrXMX5kWK7tYfNDLXMFkYptjaKph56q/pqdL2LTE4WcSUlH5ZGhlfw1cVVxqAvZh9TB0z8vlFzOoqtipofiyPSbkqzLAgverUzOsAhdMZNFlccywzF25ZDP2Z3u+8G0dgBu9GAZgyzlBtKndcJ2KVuJLfC/h0iFiJUPmbEsDkbw2IJSXQ9D/Y/Z5BhrhcWLN4inEiHWEkAVGiA757PGwjMakdA8HdPNQJi2Ji1EqUE4V+Kex6mSSDiuLlkmWJmMW2uh05V4H8K5rSBXm2y4wXTN1B+nwKtsR5qPrP6idzgO0qfZWPg/l8hk81A28RDclUMeYHwP7ZzbTnc2BbM9Jh4b5ZHXDVbIQQVZTa2RjXFSLW6RB8MottM6+AsxohVbUIwW0uXeHHzHmDZ3dCdfrOh967Y=
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5576.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39850400004)(346002)(136003)(396003)(366004)(36756003)(478600001)(6512007)(6486002)(316002)(7116003)(6916009)(26005)(186003)(8676002)(2616005)(8936002)(5660300002)(66476007)(86362001)(64756008)(83380400001)(33656002)(66556008)(66446008)(66946007)(71200400001)(2906002)(76116006)(91956017)(6506007)(38100700002)(122000001)(3480700007)(45980500001); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: iDkaxZ3U8T7frLsYXnxq/2BCf0mo6SGtrSL6pXF+WAbFnQKuQeY8h67O+EwPvN0PJg37S5rJ6AMsLVKZhEGcRWsp/549mP6zJNeos58wNdwPxr4PQIHhwmBdckVaDsWGfINoIFe82N5tSn04fSBGuqK08RKvGxOz4q5ttr43uSPSQcWg2N5oTvsPoGU/Wxbn9BPRfi9DmNhxDEb7VLrTteKHNq4fv1v+U0sP9si6XqhXK1FIw9eFV/B+OBRxTrlADFlIdH9NqUnpVibuQpKy/l29wOzq3Xfp8SuK4ke+VXSjv8pzyU6onZ9Fcu8FUy8hwBU+Fjk7bG/tV9OVUTVMmjXhrWe3nF5XQb7sMvaRsejNGQajRQT/PMcvzPagcfLkU/EgL2+B3jwMsjnk5nUyYRwPFBDZqzZRRUnZJ+oKymdmpxBD9uGCL3HWn6YmE3Nf189GA4apBTvnkVIOzenmstqEzFNNL3UYsftdX2g2bQJh4JdjfMSRDEkKVPmZQCUZ4H7uI29+7Y9J3WiA7ME9Tk11E4FmgQMjCDLdNnJuWWLPSowELeBMZNW+rA3T3Zffao79jn1QP0K66jY4ZGFkfuLfItMeVGX2JMF7qh+uhTx2S6OCAEX1lbKNOrECXMgHcm1NIdZqPAmWdoEWhZtZ+IXLM/V6mPqXD5I3isAtA5a664pzFViRwlu8yQ48tElNds3nyWeCmIYDRZ/0+MtWxAjFB/ovvTdfLVPjsP6FCDB1SGA0pDADH2DqkgLfblZXbmeuXOZHCj15mmLeesFyHtYs0SgC3eOMBLFTWj4UGHpKJE7lvVNvhLuME9btAe2iIP4ckc4RL1TQpOF9fFCFXM70KFfiM7mwT0s7k3ZTsVwru5nkxmiMV7zM/vFzEFXJps25toM3DNDOq68b/Bw8xPwGx2BYuN0ynhmO8rSQwgWybx3hCibsjjy/nehr9Mqr4YhOEMbCxm9usFY4e8ke8m1L+EFbmBSE9BLI3xN2qL9lr8bkzSGUsv7jOpIliPwAmZU6EJ9TQQlWYvsPKMzyD8ox6Ea3qevBYJgnCuduf9KAGBzLAmgkQM7C/oghwjc5ZCpfa3SaXmCuHPdAeoHyUESfw+ITuOxZaZqstmg2iQSPEWYLs65vhzpCHlPAow68zJTmumuaSRClMnsunW5U94WgGyTa1+T93OwyzdunxfmFImn7l6zA2R1NUONAwv7VWjvdmaB4ljwDPH2A8K5PsK4XrvW9L3b3+to6Had7E6211EVO2zvRwo9VP5iY7MgU0zL63+AdojWtFlbVMRMecJTW9wXStgW3+IbjhKO90W2Vq71lCsCKgLC4M9npvUQM
x-ms-exchange-transport-forked: True
Content-Type: multipart/alternative; boundary="_000_8713C3AB71C04EC0897715F80EC11309armcom_"
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6153
Original-Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT027.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs: 9b286837-be2b-4543-3fa7-08d930a82a2d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 110PYBdhxATv/7Cnrerif22yod1Gtj+4hJ0ZOPZZG0fLCpvpkQAO30uZ/ADw6qoS/BOcLiVhDTD8JiN4uPNTaojqZ6UQEuOspZM3HZ9i236kc3WfZ9XU4/7t80ODFYzUF1qb/VrJEn3g9c71rfjfvgRR8/Qby3vpUOl2jwP100qqVlc+o8x1cf6UD7T63WNyV1HTAieqKQuEwQz0VUu4baxGBNOvEtxzu2A58q3CgXI3+u9UcQBlrVvZoX9wpAaRXvgF8X7nbDB1s5DxktqXd+913n9vj29w6R+X6yTSoqCMG6OG8Hdt9tZZd735Vih8dQAP2FLrnQMWKR61PEmtdCfcRXo+MHeNm9h+lkqyKCQNIHrBdQJruCvLMjzonEFV71M0GnDx8xx0+N6RqtHyp2LhIGvV7/Omn/Pgon50xfTNI8Q3zHEynY0hWyc4rNeV9nLb9AjEnQtCUCuIzQu9bv21MNxlL38UYj+5//bYf47eU64/6Guo9DlnZ8/uE9W2WBi4Up1rYgZuEx/DmnrYtcbkPVgofAhPG/VegEdWo637et8QxVOZNvtEz74Om3wBZVfQC50w1mWaVoXaL4q1Nqzm4wBvr1ZQdoTcoh5tdU/khi3qZOs5hSXZMepmv3PQRvzArJFhdBeyS9YG+FMW5y4CJqud9IT/rN3qYIZj4Fg=
X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(136003)(396003)(346002)(376002)(39850400004)(46966006)(36840700001)(82310400003)(356005)(6512007)(70586007)(2906002)(70206006)(2616005)(86362001)(3480700007)(8676002)(5660300002)(186003)(36860700001)(33964004)(316002)(47076005)(478600001)(36756003)(82740400003)(81166007)(8936002)(45080400002)(6506007)(33656002)(336012)(6486002)(7116003)(6916009)(83380400001)(26005); DIR:OUT; SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2021 09:22:05.5705 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fa454ccc-4c19-4ce1-8035-08d930a8354b
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT027.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1988
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/D31PMYUQO5WZe2cmaxQ899ZR8qc>
Subject: [Cbor] Packed CBOR review
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 16 Jun 2021 09:22:17 -0000

As promised, here is my review of draft-ietf-cbor-packed.

Overall, I have few comments and I am confident that this draft will improve compactness of cbor structures—at least the ones I care about—with little overhead.

I don’t see any explanation of how to handle hierarchical packing. This would resolve the issue on Page 7.


      |  ISSUE: Not sure that we want to use the efficiencies of
      |  overriding, but having default values supplied out of a
      |  dictionary to be overridden by a rump sounds rather handy.
      |  Note that there is no way to remove a map entry from the table.


My take on this issue is: handle this the same way as hierarchical packing.

I do have a proposal for hierarchical packing. It’s a single, simple rule: References overflow from any table to a parent table. This means that the parser holds a stack of packing tables. Any time it encounters a reference, it attempts to apply that reference to the topmost table in the stack, deducting the size of the table from the reference and traversing the stack to the next table in the stack each time the reference is larger than the number of elements in the table.


Example:

Raw data:
{
    "label1" : "value1",
    "label2" : "value2",
    "label3" : "value3”
}

Compressed:

51([
    [],
    ["label"],
    [],
    51(
        [
            [],
            ["value"],
            [],
            {
               225("1") : 6("1"),
               225("2") : 6("2"),
               225("3") : 6("3")
            }
        ]
    )
])

If the outer table were implicit:

[
    [],
    ["value"],
    [],
    {
       225("1") : 6("1"),
       225("2") : 6("2"),
       225("3") : 6("3")
    }
]


I would also propose adding another element to the table: an identifier for a pre-shared reference table:

[
    “MyTable”,
    [],
    ["value"],
    [],
    {
       225("1") : 6("1"),
       225("2") : 6("2"),
       225("3") : 6("3")
    }
]


Best Regards,
Brendan

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.