Re: [Cbor] Handling duplicate map keys

Laurence Lundblade <lgl@island-resort.com> Tue, 26 November 2019 01:36 UTC

Return-Path: <lgl@island-resort.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C594C120F9C for <cbor@ietfa.amsl.com>; Mon, 25 Nov 2019 17:36:31 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hnj9oh2ljafY for <cbor@ietfa.amsl.com>; Mon, 25 Nov 2019 17:36:30 -0800 (PST)
Received: from p3plsmtpa06-03.prod.phx3.secureserver.net (p3plsmtpa06-03.prod.phx3.secureserver.net [173.201.192.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5FE7B12006F for <cbor@ietf.org>; Mon, 25 Nov 2019 17:36:30 -0800 (PST)
Received: from [192.168.1.80] ([76.167.193.86]) by :SMTPAUTH: with ESMTPA id ZPmHi6i921oMRZPmHiPTdT; Mon, 25 Nov 2019 18:36:29 -0700
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
From: Laurence Lundblade <lgl@island-resort.com>
In-Reply-To: <32467.1574668950@dooku.sandelman.ca>
Date: Mon, 25 Nov 2019 17:36:28 -0800
Cc: cbor@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <0A1548F5-98A0-46AB-8CDA-991F83AB2DD8@island-resort.com>
References: <CANh-dX=EVDa4EwrgWKof4kCD3nkfV3BvH0Cg5ZKOivmXJ_Dm8g@mail.gmail.com> <E5C74E1F-A5F4-4AB8-9787-3999C4697C3B@island-resort.com> <CANh-dX=1gkAtfSG-yzCsVAkk=oLM-=dN_JLCr1kQK3d6Jb0fSw@mail.gmail.com> <F81E1A57-6072-44FA-A148-8F3ED7520791@island-resort.com> <32467.1574668950@dooku.sandelman.ca>
To: Michael Richardson <mcr+ietf@sandelman.ca>
X-Mailer: Apple Mail (2.3445.9.1)
X-CMAE-Envelope: MS4wfBmJVtEnv7IenvwWO/1TwHE4cjZj1St0uZ1K9eyyZCkvJQJoU/vSiGgC9wTenlubyEY2djayg4rqqUQwcd6HODWhyNw6wEnYePmzygLxK23CqeqdZqhv lfFIjTg5TkYULjG9IimQ6RS9w443+i7LirB93CNDh8zOT58BTE0UGtVWOMG4uUQSfvmurRj9IQdh0DWUoN1ghQYZUMpwUcVcVfM=
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/p-XXOUrf1biflqQJY6_N9Ub9mEM>
Subject: Re: [Cbor] Handling duplicate map keys
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Nov 2019 01:36:32 -0000

Hi Michael

> On Nov 25, 2019, at 12:02 AM, Michael Richardson <mcr+ietf@sandelman.ca> wrote:
> 
> 
> Laurence Lundblade <lgl@island-resort.com> wrote:
>> My intention was that “pass all items up” covers your case.
> 
> python, ruby, lua, java, etc. dictionaries can not "pass all items up"
> The reason why JSON is so successful is because it maps into native
> dictionaries (maps) trivially.

Yes, they fall into one of the other two categories:
- They do duplicate checking and return an error
- They return no error and are either take first, take last or take random

> 
> In order to pass things up, a different structure will be necessary,
> at which point many developers and protocol designers will see no value in
> CBOR over a bespoke binary encoding.
> 
> It is this refusal to define a specific algorithm for dealing with duplicates
> that is leading to security issues.  It's not the duplicates themselves that
> is the problem, it is the ambiguity.
> 
> I am less afraid to declare an answer and make some decoders non-compliant.
> I think that they will get fixed.

I would hope fix them means that they will error out when a duplicate is detected.

It probably doesn’t cost any more than take first and is the more clear, clean and secure thing to do IMO.

Seems like it will be easier to get agreement on that behavior than take first. Also, there will always be some implementations that error out. We can’t get them all to do take first.

LL