Re: [Cbor] Interactions of packed CBOR and tags

Brendan Moran <Brendan.Moran@arm.com> Fri, 04 September 2020 11:05 UTC

Return-Path: <Brendan.Moran@arm.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 634FD3A0C84 for <cbor@ietfa.amsl.com>; Fri, 4 Sep 2020 04:05:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=armh.onmicrosoft.com header.b=teMaBdHp; dkim=pass (1024-bit key) header.d=armh.onmicrosoft.com header.b=teMaBdHp
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RcGHsaWt4GBH for <cbor@ietfa.amsl.com>; Fri, 4 Sep 2020 04:05:38 -0700 (PDT)
Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50051.outbound.protection.outlook.com [40.107.5.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1DAED3A0C77 for <cbor@ietf.org>; Fri, 4 Sep 2020 04:05:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cfW0/y4Pt9DIX9CkTCliyVOR63RGRW+XQJpg42r81HY=; b=teMaBdHp+Em9mttCew1si5paGg81FCYxf3y3SToRxVtxilAplctOf4dxbwvqoNcNqXWP/TZ2jej6a//URt0sXKlz7LY4xNLabkT7cEbFPOQMSdEMti/UZvi4oEZOCN5wEfoG9pzD7HFLvgOk4abqG0FGoXyCqtlCU5pzPixXD1I=
Received: from AM5PR0201CA0008.eurprd02.prod.outlook.com (2603:10a6:203:3d::18) by AM0PR08MB5444.eurprd08.prod.outlook.com (2603:10a6:208:185::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3348.16; Fri, 4 Sep 2020 11:05:35 +0000
Received: from AM5EUR03FT060.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:3d:cafe::45) by AM5PR0201CA0008.outlook.office365.com (2603:10a6:203:3d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3348.15 via Frontend Transport; Fri, 4 Sep 2020 11:05:35 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; ietf.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;ietf.org; dmarc=bestguesspass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT060.mail.protection.outlook.com (10.152.16.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3348.16 via Frontend Transport; Fri, 4 Sep 2020 11:05:35 +0000
Received: ("Tessian outbound 195a290eb161:v64"); Fri, 04 Sep 2020 11:05:35 +0000
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6dfa4dbdd6bcdc23
X-CR-MTA-TID: 64aa7808
Received: from bdf2e32a422a.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 05880597-4F91-400A-B16F-AC905017CB72.1; Fri, 04 Sep 2020 11:05:29 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bdf2e32a422a.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 04 Sep 2020 11:05:29 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RtUFfX582qNjzuJzTdKqk2GmVnC+iFwOuVh+/lNQzXVlwIdGi9juqBPM3K/H5gQySSVVmMBEyLBcmxW4tHA1JqmT55ml3/2Hj5koJ7k8IHfI+C5i4DZw+b66h54e2mq6tFjtVIfAQIfqYd4hGs1Q1kzsOlX6d2FXzwiJQ4WP332KkjgeRAcfWc6+bjgDI2FEWiIHtBYFBsSNHMDD7QWDVib5kuC5mktF9Ci2YylUCCVJT33hNYHnhZfN2v1QeCS6PDV764/huMRT6LAZc8NvvxUMD2lSdhSoGGDOoEhZLyLPoEVY4GABe3OJvrFKKdDSXgRYP0kz0iZ1Z5QbcXuGZQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cfW0/y4Pt9DIX9CkTCliyVOR63RGRW+XQJpg42r81HY=; b=AGFMT22tnBOGU2Tyrviqyai5ThgnkNONGKnmo8y42vXzaTJBCO7e9zfnCajus8rSct2+IPY2jUBXx6+NDwY2xuSW03OqbFEXu21HXzDAC/wb4hMKXkbuVmPKIJVyjR9omsSeozG8aYhDJqwEubCDzmP/5iplyh9luVQmBr/5Fg/5L9ioFOYqNBOAvbPd/EB2wVJpNOWVRyQk9nkhT4ZEatae4FDhKU3k2zi1rLGFZbEauf+cKji2jWklXzTn2JqPy9AxnSAdqDxcZ2uPcumXmcyP6HuW8rPiQ3iijSSouf4lJhFQ5wWlsfLaKlhRHD0JB7rGPa2ByOkPAe33cA/kGw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cfW0/y4Pt9DIX9CkTCliyVOR63RGRW+XQJpg42r81HY=; b=teMaBdHp+Em9mttCew1si5paGg81FCYxf3y3SToRxVtxilAplctOf4dxbwvqoNcNqXWP/TZ2jej6a//URt0sXKlz7LY4xNLabkT7cEbFPOQMSdEMti/UZvi4oEZOCN5wEfoG9pzD7HFLvgOk4abqG0FGoXyCqtlCU5pzPixXD1I=
Received: from AM6PR08MB4738.eurprd08.prod.outlook.com (2603:10a6:20b:cf::10) by AM7PR08MB5399.eurprd08.prod.outlook.com (2603:10a6:20b:104::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3348.15; Fri, 4 Sep 2020 11:05:22 +0000
Received: from AM6PR08MB4738.eurprd08.prod.outlook.com ([fe80::a98d:5ebe:dc1d:ea56]) by AM6PR08MB4738.eurprd08.prod.outlook.com ([fe80::a98d:5ebe:dc1d:ea56%3]) with mapi id 15.20.3348.017; Fri, 4 Sep 2020 11:05:22 +0000
From: Brendan Moran <Brendan.Moran@arm.com>
To: Jim Schaad <ietf@augustcellars.com>
CC: "cbor@ietf.org" <cbor@ietf.org>, Carsten Bormann <cabo@tzi.org>
Thread-Topic: [Cbor] Interactions of packed CBOR and tags
Thread-Index: AdZ8s0xpKERBvw7yTZyxz2a31flriAABj4uAAAP6fwABR/KiAAAIntwAAAHnt4AAAqfxAAAjUnmA
Date: Fri, 04 Sep 2020 11:05:21 +0000
Message-ID: <94909856-483E-4FFD-BB6E-59C79623FF6C@arm.com>
References: <00c101d67cb5$2588b790$709a26b0$@augustcellars.com> <E30F54B6-1A63-48AC-89AE-61983654B5A9@tzi.org> <00cc01d67cc9$766c7b60$63457220$@augustcellars.com> <4AE9B2FA-EEB3-4B45-96E4-9DC85118567D@arm.com> <016f01d6820b$bc7d7cc0$35787640$@augustcellars.com> <25F7B7D3-7ED7-4062-8000-21D1AF1A69C3@arm.com> <018001d6821d$fb710980$f2531c80$@augustcellars.com>
In-Reply-To: <018001d6821d$fb710980$f2531c80$@augustcellars.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3608.120.23.2.1)
Authentication-Results-Original: augustcellars.com; dkim=none (message not signed) header.d=none;augustcellars.com; dmarc=none action=none header.from=arm.com;
x-originating-ip: [217.140.106.51]
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-HT: Tenant
X-MS-Office365-Filtering-Correlation-Id: a191c4a8-7a36-4305-33e8-08d850c272bc
x-ms-traffictypediagnostic: AM7PR08MB5399:|AM0PR08MB5444:
X-Microsoft-Antispam-PRVS: <AM0PR08MB5444370160D6186ED29B82AFEA2D0@AM0PR08MB5444.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original: SssrF4UPD5G4Q7OgSTlBdStaeh+XxL0nZnVdUHqR3fOyGNV7oKlRwIn//Iu6fXM0jJYGCAVNeAG0uJQbuSH6eqjyCPrkZfVzjEh5/ET8/ihvm7sF01kR6dF1N7xzILn/JxeEbvDQQMIeBn/Iv7ir+1byQMWd09+yfQcLnvJu3icDxz1V7JYt7l7m/NCGL4Bc97o4ALmmFe6VYTd/vz+2EsGvdWAS0vaV0SXyCCxwtW5NBGIh8v5AD6pRH+d385kwFgQGw/TVAzShuM+nbSXM4+YDLj4NxdEFhe95RkKNUXjwLhC0+vhIx/Xl9OFvDsvBkfw2Z+3RjwtZV/t9ecXXquxe4UAcbc+mVlp0/JQgpkhxHgJlZwVHqFc26NCjgJJaMhlnq+fKUTQUcbc/HLquSg==
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR08MB4738.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(396003)(376002)(136003)(346002)(366004)(6486002)(2616005)(186003)(26005)(6506007)(6916009)(8936002)(83380400001)(4326008)(53546011)(6512007)(2906002)(966005)(316002)(66556008)(66476007)(478600001)(54906003)(33656002)(91956017)(76116006)(66946007)(64756008)(5660300002)(8676002)(71200400001)(86362001)(66446008)(36756003); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: T9NTdrDKPIbV6wPKI9v0hLlQ8Dhb4HotHYoPHTjUBSlMtu7dPg9s8H5lgsp6xb6TAYk6Q3cF40+C5fajKks2OmSkQ8v1NzmtjKoXf+rZ6VfFE3PgE//kKPMhESlmBfQqV+4wDblW897FduACztY+fIwBLtkQvlEjrsP5eTOwJxcJAd3HpRSZ5yV5M1PugehpVMJRMmVNp+RI8uj4ZyLCkcQw6fIxoJBYaZKkCbCNLrhjgfB1ju5DSuHBurrq9AV61LJT7ktiX8cwr7KA+IYSZAkjJBzd0S9M6vu4mdSShmP/nkI/DVapU6OwU/JwOyQDVzd+6NJV9LMlOYujHO0SWqSKJSbRGps2rvAVVe19g/Ps1sxT8tEb/Kwh7rRrxbN2cm5WRadWwN334wHwJqAKldx78nhlz91FOY9y0/CDnBgi80eUc7EiUv1ZPMC1133CSkrV8H0+80Ws8ytMzSjbOrBGq/YdmUfGKYlcefnWcC6OxqpbrBA0JBl7REmv7WRP5/pxilGC2yiu9T36lYKacmAoRT0yvY7YE+JaH5R3eSZy9mvCluDygGeMjro91SW+FiriuIc3n7HkOQIm1GFbHjf9nyKtMSOf45gRvDdTaxppY4d3mlMrjh499LtUkteaD4Zk9DdOWITIcYxcQA9bCQ==
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <0703311519F18A429E8C0012E9E5129F@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5399
Original-Authentication-Results: augustcellars.com; dkim=none (message not signed) header.d=none;augustcellars.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs: c7b749c3-2060-4b17-5b78-08d850c26b0d
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: y1zGwsbo9rOp54ZGIst57Nph7vQs2HnM0YLjL3ggYRw9tu62qlmMGM4pjrTcc8rFXd1Tqw67zUmw/LUamv2pzzj3e0o3BZpCjEEjDApDiZDPlge6QQYWQsTsyAKPoyHzQb1HgghjTQBx7xpmp9io4Yjkko7oCPh47tNxH/mkqEMua9i2hETuLsvv0wjtpPWu+HleIj9DF/RzF1U8El0MMJDRf03UcbYa66O/fjh7UCyEvedQe3d2j+mfdTNYjlSFP40ds4Ikyg7e8gfz9MH3bSH1HGBhjtENVK8KSoE1wqDCQRZN+kg6I7S4uCuEqZwAsB71DZMZFDATHBKakb7/uZa5A9MUUh6q+4XUrCkhmpar40Iv66lV2Uj6IRiSqbjWAxESnoH/auZF/GbADVuORLBQxqAT6Y1iB+gi2lNsHuWgIKiOCjHkp3FHHwnUTFmbqdAORmIVt1UUjnXQFITx3Ys7cWyYiHs5Vevfo2E5Soc=
X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(39860400002)(396003)(376002)(136003)(346002)(46966005)(82740400003)(86362001)(6512007)(336012)(36906005)(47076004)(2616005)(356005)(81166007)(33656002)(36756003)(82310400003)(83380400001)(316002)(186003)(70586007)(70206006)(6862004)(5660300002)(6486002)(8676002)(966005)(26005)(8936002)(4326008)(53546011)(6506007)(2906002)(54906003)(478600001); DIR:OUT; SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2020 11:05:35.1089 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a191c4a8-7a36-4305-33e8-08d850c272bc
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT060.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5444
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/MMmRet2tSSwoOte_73fL6oaJ8YM>
Subject: Re: [Cbor] Interactions of packed CBOR and tags
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 04 Sep 2020 11:05:41 -0000


> On 3 Sep 2020, at 19:13, Jim Schaad <ietf@augustcellars.com> wrote:
>>
>> [JLS]  I don't find this to be very convoluted at all and it is what
>> my compression algorithm generates automatically.  I look at the piece
>> I split off from the prefix and see if I have "enough" duplicates to
>> compress them down.  I am still trying to decide how to do some
>> extraction from "the middle" of things.  Consider looking at
>> ["www.merch.ietf.org", "www.datatracker.ietf.org"]
>>
>> It would be nice to think about compressing that as
>>
>> 6([ "www.", "merch", "datatracker"], [".ietf.org""],
>> 224(225(simple(2))), 224(226(simple(2)))]
>>
>> Where we have pulled both prefix and postfix strings extracted and maximize the amount of commonality.
>>
>> [/JLS]
>
>
> [JLS] After correcting both of our notations, in this case the it seems that both of the notations are the same size.  From your response, I am not sure if you believe that the notation I gave was not legal under the current draft or not.
[BJM]

It doesn’t look legal to me. The draft says:
> Packed-CBOR = #6.6([rump, [*prefix], *shared])


Normalising it with the -01 draft, I think we get the following (45 bytes):
>> 6([[6(224(simple(2))), 6(225(simple(2)))], ["www.", "merch", "datatracker"], ".ietf.org"])

That’s 2 bytes better than what I suggested.

[/BJM]

>
>> Back to the observations I made above.
>>
>> 1. Compressing the domain names doesn’t work very well. Maybe this is unique to domain names, but I think we need more data on that.
>> 2. There’s a lot of regularity left here, but it’s all in the form of sequences of array elements. For example, the sequence [1, simple(1), 2] shows up regularly, as does [simple(4), 5, 0 , 6].
>>
>> [JLS] This would be taken care of by the fact that we have discussed doing the same prefix processing on arrays as well.  This was brought up specifically for the CRI case where we saw that this was going to be an issue.
>
> I’m glad to hear that. Did I miss something on the mailing list? I assumed that this was still TBD since I didn’t see it there. Does that mean that the use of Tag 6 as the first prefix is being dropped so that array references are explicit? Or are array prefixes prohibited in the first slot? Or something else?
>
> [JLS] https://mailarchive.ietf.org/arch/msg/cbor/QYqEfdkQ2iE98M1on5QZA35kgrM/ is the message where Carsten made a proposal which has not gotten into the draft yet.

[BJM]
I’d read that, but I didn’t realise it was any more than an off-hand comment. Do I understand correctly that the following is the intent?

[
  [1,2,3,4],
  [1,2,4,8]
]

Is packed to:

6([
  [
    224([3,4]),
    224([4,8])
  ],
  [
    null,
    [1,2]
  ]
])

(Yes, this is a bad example, since it inflates the original.)

[/BJM]

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.