Re: [Ideas] Computing collisions

Dino Farinacci <farinacci@gmail.com> Tue, 13 December 2016 18:35 UTC

Return-Path: <farinacci@gmail.com>
X-Original-To: ideas@ietfa.amsl.com
Delivered-To: ideas@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 81E5F129435 for <ideas@ietfa.amsl.com>; Tue, 13 Dec 2016 10:35:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.7
X-Spam-Level:
X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4hnIctQ52MWa for <ideas@ietfa.amsl.com>; Tue, 13 Dec 2016 10:35:49 -0800 (PST)
Received: from mail-pf0-x244.google.com (mail-pf0-x244.google.com [IPv6:2607:f8b0:400e:c00::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 64707129641 for <ideas@ietf.org>; Tue, 13 Dec 2016 10:35:49 -0800 (PST)
Received: by mail-pf0-x244.google.com with SMTP id y68so6374618pfb.1 for <ideas@ietf.org>; Tue, 13 Dec 2016 10:35:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=zHXxINJB20I3OlT+oMDTFNUtSxcVJFOgswp7yndAsbc=; b=xwrw48FIErrMwtqWyxkh0y1IwqylRwK9SfRkt3l1XE9QXT+FcPra95l7s83cul/U9x vnEcAooZVkzAfNwaysl9PVjnJsY7hnsHVN3eGUd67UcaHOYd/h+oDiLgAvf43X9nolYD 6cPcNxcLVgYtqv34lWe52C+W5QvJ6S1QKG8FqqyPgx6xJuN2JtH+Ypm/tl6ZvZoBuYeu dHsj/6UNy7+ei2whczks7Srm0SZRSRvYoqT9bTqhB95O1Giwyv1NQgreVDXavQXtSuzp G0gBqCqnqCJcEST3KinX/Oo+135O+Wm5AYjoK/36ZHLV0wL20iLKFFoIr2yBRiMreVPU 918Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=zHXxINJB20I3OlT+oMDTFNUtSxcVJFOgswp7yndAsbc=; b=U8ECtrisP7x2Io/YN6RYmE2q2CWjkHiUC6i/DiUajZFsevTgwKAAoQC4LQWPrfToL4 h9szIEW815HEG5mKJqdVdxaMPP0JDs3w4WgWg0dKg0WKZgA2g3Aay4zFT61ZoriMjEUW 2hAB6Kn3mbH1xBNYbfqo5+p5iHjiCjh3KyqPLAf8WnOkdMo+tzYvNbjnyCU0lUZvQrZA sQOu7IzGoG2bZ6+6KjNG+938nqQKxnf7+w5f86LfLW4ffLSv+pXSY1eg6mGKh1jA0oOj JFrtFxJhMUZjrBfGu/biQNwSL8A27+Za+iSYJiC8f9Tu39Q4ePiKh6ONUyZhqwR1OlM/ Ic2Q==
X-Gm-Message-State: AKaTC02VmRUU3SrMesuWPOYIZFJGkWz7szKYmFE/frH0GZjPHcT/GlSUR6Ze9s3ELkwvIw==
X-Received: by 10.98.144.86 with SMTP id a83mr103312883pfe.107.1481654148398; Tue, 13 Dec 2016 10:35:48 -0800 (PST)
Received: from [10.197.31.157] (173-11-119-245-SFBA.hfc.comcastbusiness.net. [173.11.119.245]) by smtp.gmail.com with ESMTPSA id y2sm81547380pff.82.2016.12.13.10.35.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Dec 2016 10:35:47 -0800 (PST)
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\))
From: Dino Farinacci <farinacci@gmail.com>
In-Reply-To: <83be2a30-1b46-2397-976e-6e2736921e19@htt-consult.com>
Date: Tue, 13 Dec 2016 10:35:43 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <6AA7A937-AEC5-4E6F-9D87-87BC257EF9F8@gmail.com>
References: <fb888f85-22ed-5e87-bf84-2b6244d587e6@htt-consult.com> <CAG-CQxqds9osqTTAi_Gm-VCVKGo49pW5OVALxYiYCrqNQ2yvRA@mail.gmail.com> <83be2a30-1b46-2397-976e-6e2736921e19@htt-consult.com>
To: Robert Moskowitz <rgm-ietf@htt-consult.com>
X-Mailer: Apple Mail (2.3251)
Archived-At: <https://mailarchive.ietf.org/arch/msg/ideas/jyqn9yYbDg7Yi197vaV5SD2SkqY>
Cc: Padma Pillay-Esnault <padma.ietf@gmail.com>, ideas@ietf.org
Subject: Re: [Ideas] Computing collisions
X-BeenThere: ideas@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Discussions relating to the development, clarification, and implementation of control-plane infrastructures and functionalities in ID enabled networks." <ideas.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ideas>, <mailto:ideas-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ideas/>
List-Post: <mailto:ideas@ietf.org>
List-Help: <mailto:ideas-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ideas>, <mailto:ideas-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Dec 2016 18:35:51 -0000

In LISP, an xTR registers with a 128-bit xTR-ID. This is not the EID. So we could have an xTR register with a random EID it chooses. Then the map-server determines it is a collision because another xTR-ID has registered with that same EID. The map-server could select a random EID and check locally if it collides. If it doesn’t, then the xTR registers with a new EID (and makes appropriate interface address assignments locally).

Note the draft-farinacci-lisp-eid-anonymity-01 specs that ephemeral-EIDs can be used. So when an xTR decided to choose a random IPv6 address as an EID and registers. It can ask the map-server if it collides. Today, the map-server Acks Map-Register messages, but it could Nak and say “go get a new random ephemeral-EID”. I like this approach better because the allocation happens at the sites within their own address prefix range and gives you some obfuscation at the same time.

If people thing the second approach makes sense, I can spec out in the above draft how Map-Servers could NAK when there are collisions. Please comment.

Dino

> On Dec 13, 2016, at 5:35 AM, Robert Moskowitz <rgm-ietf@htt-consult.com> wrote:
> 
> Here is some results.
> 
> On 12/11/2016 12:03 PM, Padma Pillay-Esnault wrote:
>> Hello Bob
>> 
>> Thanks for sending this formula.
>> 
>> If we assume that the population is not necessarily a flat space and that it is in different separate instances, this could help reduce the collision.
>> 
>> Per the ideas problem statement draft, we have discussed about the different types of devices and whether we could have some policy on what they can do. Eg. a camera should not be shopping for clothes or access bank accounts .... wouldn't it make sense to put them in different instances with different access policies?
>> 
>> What are your thoughts on this?
> 
> Let's work with a 24 bit ID space which is a population of 16,777,216
> 
> Let's then assume a random distribution of 2^14 (16,384) devices.  This is a 99.97% probablity of a collision.  A sure thing.
> 
> Now let's assume that there are 256 (2^8) classes of devices and that the population is evenly divided.  That is 64 devices per class.  We now have reduced our ID space to 16 bits (65,536), but the probablity of a collision with only 64 devices is 3.08%!
> 
> But if the division is not equal between the classes and one class has 512 devices the probablity jumps up to 84.67%; again an almost sure thing.
> 
> So we can see from this little exercise that dividing our ID space into groupings can actually reduce the collision risk.  Two caveats are clear though.  How many classes to use (you can see my effort at this in draft-moskowitz-hierarchical-hip) and what if one class out classes all the rest?
> 
> You still need to manage collisions.  You just may be telling fewer losers to select a new ID.
> 
> Now if instead you are talking about reusing your ID space on the assumption that these devices are NEVER interacting and thus never 'seeing' the inevitable collisions, then, yes, reuse the ID space.  But we all know what assume also spells.
> 
> 
>> Padma
>> 
>> On Thu, Nov 17, 2016 at 1:48 AM, Robert Moskowitz <rgm-ietf@htt-consult.com> wrote:
>> Here is the equation:
>> 
>> probability of collision = 1 - e^{-k^2/(2n)}
>> 
>> Where n is your max population size (e.g. 2^64)
>> 
>> and k is your deployed population (e.g. 7B)
>> 
>> Bob
>> 
>> _______________________________________________
>> Ideas mailing list
>> Ideas@ietf.org
>> https://www.ietf.org/mailman/listinfo/ideas
>> 
> 
> _______________________________________________
> Ideas mailing list
> Ideas@ietf.org
> https://www.ietf.org/mailman/listinfo/ideas