Re: [Ideas] Computing collisions

Stewart Bryant <stewart.bryant@gmail.com> Tue, 13 December 2016 14:09 UTC

Return-Path: <stewart.bryant@gmail.com>
X-Original-To: ideas@ietfa.amsl.com
Delivered-To: ideas@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 089C412958A for <ideas@ietfa.amsl.com>; Tue, 13 Dec 2016 06:09:58 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id slc0LbpIJ6nV for <ideas@ietfa.amsl.com>; Tue, 13 Dec 2016 06:09:55 -0800 (PST)
Received: from mail-wj0-x231.google.com (mail-wj0-x231.google.com [IPv6:2a00:1450:400c:c01::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 52FC11296E8 for <ideas@ietf.org>; Tue, 13 Dec 2016 06:09:55 -0800 (PST)
Received: by mail-wj0-x231.google.com with SMTP id tk12so101669538wjb.3 for <ideas@ietf.org>; Tue, 13 Dec 2016 06:09:55 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=wwQoMeLunGKNkSxyJQi8OP66HmDXGXYsrIt0kU9++3I=; b=n3PdV5NhHkyjLQ7BfpqcWRFDUvGyPch/EFJhIPHWYGbGx41y9+zDfppOk4X/AE5LGf 7lczPouhqwSF68Tu7JYMDvx95nmegbU+PrxVqgUDTLE0Hj+ukZWP6A1XHpWleticpuls kinJ/Ug3WZFGxNZGev4WwP5aaadKTN4v9awCRc7FNlavyr1GxH26ULSS2Ayr0Tbrymx3 P9NGIJQyF4IWBfBLvRO4JO+w/YKbosCNIAcJmJWwbk+WZRJRu/M5ykH1N+k0JH9Ge/6e sQm3GPEIlPxPwv6t6aaGCBpxL6dYVy5WnT/A4VRocGt4vrJJnVrEylQkLEFMakd1TvPS hcHw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=wwQoMeLunGKNkSxyJQi8OP66HmDXGXYsrIt0kU9++3I=; b=fH+BXSdMipybwv4ZEwgcbu+vJdYXD6EgWBDroTMBlxir0rFe2jZiWRlixq4+ZkTYrk 8JcUgDQLTnmroU13XUFTitDbCzaHHYN6VJtbLwo9TiD4I+x87d6kUIKrKBqJtYGeD/yX 3FJL/2kjEk/UGxHWrfPllXQE3swDwbjm2NRlzEn4jpfQeOGqjBrByRMjLxHhFOeQLr8T 6CJB4LB8/JqxW12ggrkNMxlLG6x/+UXIekouN5dQf7UCCD0JgSfN1hYas6fS0m/Yfd1b FSdY6W5/i8nzBxckeiR+SzosCcdWqtyucdtqPgYaTp7+2ZgiGNnYzXtOgTo9hKW1EaMS d8bQ==
X-Gm-Message-State: AKaTC03IRAzSWzBqKFTBTnW2a2XQ+RcBKJOrpL0NsMtCncWhppFTFE15Lw/qan25faQylw==
X-Received: by 10.194.58.237 with SMTP id u13mr37359042wjq.10.1481638193531; Tue, 13 Dec 2016 06:09:53 -0800 (PST)
Received: from [192.168.2.131] (host213-123-124-182.in-addr.btopenworld.com. [213.123.124.182]) by smtp.gmail.com with ESMTPSA id b3sm62445523wjy.40.2016.12.13.06.09.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Dec 2016 06:09:53 -0800 (PST)
To: Robert Moskowitz <rgm-ietf@htt-consult.com>, Padma Pillay-Esnault <padma.ietf@gmail.com>
References: <fb888f85-22ed-5e87-bf84-2b6244d587e6@htt-consult.com> <CAG-CQxqds9osqTTAi_Gm-VCVKGo49pW5OVALxYiYCrqNQ2yvRA@mail.gmail.com> <83be2a30-1b46-2397-976e-6e2736921e19@htt-consult.com>
From: Stewart Bryant <stewart.bryant@gmail.com>
Message-ID: <0f4117fb-1c93-7475-fd4c-004384c1e8f5@gmail.com>
Date: Tue, 13 Dec 2016 14:09:49 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1
MIME-Version: 1.0
In-Reply-To: <83be2a30-1b46-2397-976e-6e2736921e19@htt-consult.com>
Content-Type: multipart/alternative; boundary="------------93D18C8F19BDAE85524F2823"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ideas/PGaZ6QkUHIdb2bv1pF-kdIjQF58>
Cc: ideas@ietf.org
Subject: Re: [Ideas] Computing collisions
X-BeenThere: ideas@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Discussions relating to the development, clarification, and implementation of control-plane infrastructures and functionalities in ID enabled networks." <ideas.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ideas>, <mailto:ideas-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ideas/>
List-Post: <mailto:ideas@ietf.org>
List-Help: <mailto:ideas-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ideas>, <mailto:ideas-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Dec 2016 14:09:58 -0000

Of course there cannot be any human intervention in the selection 
process or the odds of a collision will go up hugely.

Stewart


On 13/12/2016 13:35, Robert Moskowitz wrote:
>
> Here is some results.
>
>
> On 12/11/2016 12:03 PM, Padma Pillay-Esnault wrote:
>> Hello Bob
>>
>> Thanks for sending this formula.
>>
>> If we assume that the population is not necessarily a flat space and 
>> that it is in different separate instances, this could help reduce 
>> the collision.
>>
>> Per the ideas problem statement draft, we have discussed about the 
>> different types of devices and whether we could have some policy on 
>> what they can do. Eg. a camera should not be shopping for clothes or 
>> access bank accounts .... wouldn't it make sense to put them in 
>> different instances with different access policies?
>>
>> What are your thoughts on this?
>
> Let's work with a 24 bit ID space which is a population of 16,777,216
>
> Let's then assume a random distribution of 2^14 (16,384) devices. This 
> is a 99.97% probablity of a collision.  A sure thing.
>
> Now let's assume that there are 256 (2^8) classes of devices and that 
> the population is evenly divided.  That is 64 devices per class.  We 
> now have reduced our ID space to 16 bits (65,536), but the probablity 
> of a collision with only 64 devices is 3.08%!
>
> But if the division is not equal between the classes and one class has 
> 512 devices the probablity jumps up to 84.67%; again an almost sure thing.
>
> So we can see from this little exercise that dividing our ID space 
> into groupings can actually reduce the collision risk.  Two caveats 
> are clear though.  How many classes to use (you can see my effort at 
> this in draft-moskowitz-hierarchical-hip) and what if one class out 
> classes all the rest?
>
> You still need to manage collisions.  You just may be telling fewer 
> losers to select a new ID.
>
> Now if instead you are talking about reusing your ID space on the 
> assumption that these devices are NEVER interacting and thus never 
> 'seeing' the inevitable collisions, then, yes, reuse the ID space.  
> But we all know what assume also spells.
>
>
>> Padma
>>
>> On Thu, Nov 17, 2016 at 1:48 AM, Robert Moskowitz 
>> <rgm-ietf@htt-consult.com <mailto:rgm-ietf@htt-consult.com>> wrote:
>>
>>     Here is the equation:
>>
>>     probability of collision = 1 - e^{-k^2/(2n)}
>>
>>     Where n is your max population size (e.g. 2^64)
>>
>>     and k is your deployed population (e.g. 7B)
>>
>>     Bob
>>
>>     _______________________________________________
>>     Ideas mailing list
>>     Ideas@ietf.org <mailto:Ideas@ietf.org>
>>     https://www.ietf.org/mailman/listinfo/ideas
>>     <https://www.ietf.org/mailman/listinfo/ideas>
>>
>>
>
>
>
> _______________________________________________
> Ideas mailing list
> Ideas@ietf.org
> https://www.ietf.org/mailman/listinfo/ideas