Re: [core] Your cool presentation on friday core meeting
peter van der Stok <stokcons@xs4all.nl> Thu, 19 November 2015 07:43 UTC
Return-Path: <stokcons@xs4all.nl>
X-Original-To: core@ietfa.amsl.com
Delivered-To: core@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 302901A92B7 for <core@ietfa.amsl.com>; Wed, 18 Nov 2015 23:43:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.398
X-Spam-Level:
X-Spam-Status: No, score=0.398 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, BIGNUM_EMAILS=2.999, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ryviXLBY6YpM for <core@ietfa.amsl.com>; Wed, 18 Nov 2015 23:43:35 -0800 (PST)
Received: from lb2-smtp-cloud3.xs4all.net (lb2-smtp-cloud3.xs4all.net [194.109.24.26]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7570C1A92B4 for <core@ietf.org>; Wed, 18 Nov 2015 23:43:34 -0800 (PST)
Received: from webmail.xs4all.nl ([194.109.20.214]) by smtp-cloud3.xs4all.net with ESMTP id jKjY1r0024d84Ai01KjYPC; Thu, 19 Nov 2015 08:43:32 +0100
Received: from [2001:983:a264:1:5cbf:6d13:c88d:6742] by webmail.xs4all.nl with HTTP (HTTP/1.1 POST); Thu, 19 Nov 2015 08:43:32 +0100
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 8bit
Date: Thu, 19 Nov 2015 08:43:32 +0100
From: peter van der Stok <stokcons@xs4all.nl>
To: Alexander Pelov <a@ackl.io>
Organization: vanderstok consultancy
Mail-Reply-To: consultancy@vanderstok.org
In-Reply-To: <564C89C6.40903@ackl.io>
References: <82c7667140aeaa28efab31a778a26204@xs4all.nl> <564C89C6.40903@ackl.io>
Message-ID: <f7f13a18f7759f4c9b25c4e47b5d7412@xs4all.nl>
X-Sender: stokcons@xs4all.nl (P2l56B8SiGvttByq+qwGNGgvp1LocfBi)
User-Agent: XS4ALL Webmail
Archived-At: <http://mailarchive.ietf.org/arch/msg/core/niMiPb_fC96dhWq3U8wodxWKBiA>
Cc: Core <core@ietf.org>
Subject: Re: [core] Your cool presentation on friday core meeting
X-BeenThere: core@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: consultancy@vanderstok.org
List-Id: "Constrained RESTful Environments \(CoRE\) Working Group list" <core.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/core>, <mailto:core-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/core/>
List-Post: <mailto:core@ietf.org>
List-Help: <mailto:core-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/core>, <mailto:core-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 19 Nov 2015 07:43:38 -0000
Hi Alexander see between <pvds> and </pvds> Alexander Pelov schreef op 2015-11-18 15:23: > Hi Peter, > > I'm quite busy these days. I would like to send you a detailed > response, but unfortunately this will be impossible today. > > Please, see in-line for some initial remarks. > > Le 18/11/2015 09:09, peter van der Stok a écrit : >> Hi Alexander, >> >> I like to react to your presentation of CoMI during the Friday CoRE >> meeting. That is necessary for me to understand the underlying >> factual discussion. >> >> Your statement “A hash clash 5 years down the road can break your >> network” needs some clarification. I interpret that, out of the blue, >> a hash clash can occur and lead to problems in the network. This is >> simply not true. A new hash clash may occur when modules are changed >> and recompiled in a server. At that moment and before the server is >> made operational hash clashes are detected and remedial actions can be >> taken. > One of the specific cases I cited is, when you have your network > running for months, maybe several years, before updating a YANG > module, which leads to hash clash. This leads to unexpected protocol > exchanges (e.g. clash-file loading), unexpected memory allocations > (the clients must learn that some of its servers have these new > clashes, while others have some other clashes, and so forth), bugs > that have not been tested for long time, ... <pvds> I do not share your concern. hashes are generated before loading of the module, hash clashes are easily verified, and rehashes are also done before loading of the module. The rehash protocol concerns one client and one server and can be tested in isolation. Hash clashes are easily simulated. </pvds> > >> Your next statement “hashes are underspecified” also needs some >> clarification. I understood from you that the remark was motivated by >> the absence of a full-proof process defining the solution of hash >> clashes. Such a process is not necessary to assure inter-operability. >> Once solved, the server returns the unique rehashed values and there >> is no need to specify how the rehash values are reached. Nevertheless, >> the draft suggests that a tilde is prefixed to the YANG name, after >> which it is rehashed. There is a very small probability that the new >> hash clashes with another one. Actually, you said to be afraid that >> the hashing algorithm may continuously generate the same set of >> clashing values independent of the prefix. I have no idea if this is >> true, and do not propose to check it for murmur. > I've though on how to handle this specific question for a long time, > and unfortunately you cannot guarantee that re-hashing will no > guarantee new hash clash. Which could, in turn, generate new clashes. > Each clash leads to two new names that need to be rehashed. > >> >> I agree that for efficiency reasons the same rehash process should be >> followed in all servers such that all servers with the same set of >> module versions arrive at the same rehash values. An approach >> different from rehashing is to assign the lowest not assigned natural >> number to the clashing names in lexicographical order. That will >> generally mean that two colliding names get the values 1 and 2 >> assigned, and very rarely the values 3 and 4 may be used. > > I think that Structured IDs provide a way of handling this in a > reasonable manner. We've already discussed how in a structured ID you > can have hashes. <pvds> I showed that the problem above (continuous generation of hashes) can be easily avoided. </pvds> >> >> Last I should like to make a remark about probabilities. The whole >> world around us is based on probabilities. For example, there is a >> finite probability that a fatal fault will occur in an airplane during >> one hour of flight. Or that during digital transmission bits are >> toggled without detection by the checksum. These probabilities are >> calculated and should be smaller than a given probability value. This >> is a well-established engineering practice. Therefore, the clash >> probabilities are calculated in appendix E of the CoMI draft. It shows >> that for the targeted hash size and number of names, the probability >> that more than one clash occurs is 10^-3 smaller than the probability >> of one clash. These are quite small values. >> > What constitutes a "small probability" is a relative question. 10E-3 > is typically considered quite elevated (not to say - unacceptably > high) collision probability. Given that you can have hundreds of > entries, this probability is even worst. Given that each collision > requires re-hashing, which could provoke other collisions, we end up > with a problem I consider to be quite dangerous. <pvds> Appendix E cites different numbers than you do above. For a server with 1000 hashed YANG names and a hash of 30 bits, the probability of a single clash in a given server is 5*10^-4. That means that in an installation with 2000 servers containing disjunct sets of 1000 YANG names (total of 2 million YANG names), there is on average one server with one clash. There being more than one clash has a probability that is 10^-3 lower. In my opinion these are acceptably low probabilities, because they promise quite small rehash information in the clients (In this case: no rehash or one server with one clash) BTW Andy has hashed all YANG names in all to him known modules and no clash occurred. </pvds> > >> By the way, by relying on identifier assignment, there is also a >> finite probability that the same identifier is allocated to names on >> different modules (a hidden clash), due to power failures, undetected >> transmission errors, or simply copying mistakes. >> I recommend that in the security section of the identifier assignment >> draft, it is discussed how modules are detected with an identifier >> that has been assigned without going through the registration process. >> > That's a non-issue. <pvds> I regret that you take this issue so lightly. When there is no SDO that makes YANG name number allocation part of the certification process, YANG module writers are not very motivated to pass through a registry and assign the registered numbers. Consequently, servers may use identical numbers for different YANG objects. This means that the extension of your installation with a server runs the risk of duplicate identifiers. You need a procedure to test this, or at least write text that this a reasonable hazard. </pvds> > >> Hope this will stimulate further discussion. >> >> Peter > Thanks for the useful remarks. I am glad when we have constructive > discussion. Sorry, for the brevity. > > Best, > Alexander
- [core] Your cool presentation on friday core meet… peter van der Stok
- Re: [core] Your cool presentation on friday core … Alexander Pelov
- Re: [core] Your cool presentation on friday core … peter van der Stok
- Re: [core] Your cool presentation on friday core … Alexander Pelov