Re: [arch-d] Centralization or diversity

John Day <jeanjour@comcast.net> Wed, 08 January 2020 21:49 UTC

Return-Path: <jeanjour@comcast.net>
X-Original-To: architecture-discuss@ietfa.amsl.com
Delivered-To: architecture-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C1A3E1201EA for <architecture-discuss@ietfa.amsl.com>; Wed, 8 Jan 2020 13:49:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Level:
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=comcast.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vWcIuzBJ21g8 for <architecture-discuss@ietfa.amsl.com>; Wed, 8 Jan 2020 13:49:52 -0800 (PST)
Received: from resqmta-ch2-10v.sys.comcast.net (resqmta-ch2-10v.sys.comcast.net [IPv6:2001:558:fe21:29:69:252:207:42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A18FE12003E for <architecture-discuss@ietf.org>; Wed, 8 Jan 2020 13:49:52 -0800 (PST)
Received: from resomta-ch2-20v.sys.comcast.net ([69.252.207.116]) by resqmta-ch2-10v.sys.comcast.net with ESMTP id pIXGiO0edRHmIpJD5ipBti; Wed, 08 Jan 2020 21:49:51 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=20190202a; t=1578520191; bh=uME5qzaFTLhlYOXAMa6XL/H5U4cZaMWRBrkJWzCRHhI=; h=Received:Received:Content-Type:Mime-Version:Subject:From:Date: Message-Id:To; b=sydpshLQ1qnvr07sIb6rSkJkKXCDqIERDch1Q0gcpqoxTebPd5dlFM0Z1IUCPXk1H e596pim3OsgDGylv1fgvXux3FH4QoF2wgX5jJUL5Gzfwbya0Yk0rEtWYyytzWTQEbT Wa4dg02NPU68mAfLfO3vH5FKTkKft35aby4WyLkIYFlfN+Me8an2vrrZKq0clkYwMS Kl9cnYXuC7+IBaJW9Q6YT5YxsgzV1kiIStn7yZb8GRbNNm2WWVoLdoNc9GwnJr5gHl yIHJtO80SXqE9RgGk4WLY3nLhDXyQsDtijejr1uH7ftfpkRzhhDb6HgDEc+vcnAab1 gcVwDWqpIolTQ==
Received: from [10.0.1.32] ([24.34.101.155]) by resomta-ch2-20v.sys.comcast.net with ESMTPA id pJD2inb0i0kUupJD3i5Hcg; Wed, 08 Jan 2020 21:49:51 +0000
X-Xfinity-VAAS: gggruggvucftvghtrhhoucdtuddrgedufedrvdehkedgudehudcutefuodetggdotefrodftvfcurfhrohhfihhlvgemucevohhmtggrshhtqdftvghsihdpqfgfvfdppffquffrtefokffrnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpegtggfuhfgjfffgkfhfvffosehtqhhmtdhhtdejnecuhfhrohhmpeflohhhnhcuffgrhicuoehjvggrnhhjohhurhestghomhgtrghsthdrnhgvtheqnecuffhomhgrihhnpehivghtfhdrohhrghenucfkphepvdegrdefgedruddtuddrudehheenucfrrghrrghmpehhvghloheplgdutddrtddruddrfedvngdpihhnvghtpedvgedrfeegrddutddurdduheehpdhmrghilhhfrhhomhepjhgvrghnjhhouhhrsegtohhmtggrshhtrdhnvghtpdhrtghpthhtohepthhonhihrdhlihesthhonhihrdhlihdprhgtphhtthhopehtthgvsegtshdrfhgruhdruggvpdhrtghpthhtoheprghnughrvgifrdgtrghmphhlihhnghesgeduledrtghonhhsuhhlthhinhhgpdhrtghpthhtoheprghrtghhihhtvggtthhurhgvqdguihhstghushhssehivghtfhdrohhrghenucevlhhushhtvghrufhiiigvpedt
X-Xfinity-VMeta: sc=-100.00;st=legit
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: John Day <jeanjour@comcast.net>
In-Reply-To: <FDE60F88-DD22-4209-8415-76C6FB3C15F9@tony.li>
Date: Wed, 8 Jan 2020 16:49:46 -0500
Cc: Toerless Eckert <tte@cs.fau.de>, Andrew Campling <andrew.campling@419.consulting>, architecture-discuss@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <265A560E-F07F-4CB8-8D5F-6077D1419CE5@comcast.net>
References: <LO2P265MB0573A1353911BFDD554DE5C8C2760@LO2P265MB0573.GBRP265.PROD.OUTLOOK.COM> <CAKKJt-dtX4kceJqY4kaB-vg4rs0uEn01SyyzR4m+UAO_0bZ=7g@mail.gmail.com> <c2633b3d-d114-217a-efca-2002bca9a084@gmail.com> <2D994CF8-8C96-405B-A4EF-65672EF31031@tony.li> <20200108182816.GS8801@faui48f.informatik.uni-erlangen.de> <60CB9CB1-590D-4DCD-963C-E34B8F40EA64@tony.li> <20200108192553.GV8801@faui48f.informatik.uni-erlangen.de> <FDE60F88-DD22-4209-8415-76C6FB3C15F9@tony.li>
To: tony.li@tony.li
X-Mailer: Apple Mail (2.3445.104.11)
Archived-At: <https://mailarchive.ietf.org/arch/msg/architecture-discuss/wv_vdPHYE2GezRc-UAAJ1V8oiMo>
Subject: Re: [arch-d] Centralization or diversity
X-BeenThere: architecture-discuss@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: open discussion forum for long/wide-range architectural issues <architecture-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/architecture-discuss>, <mailto:architecture-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/architecture-discuss/>
List-Post: <mailto:architecture-discuss@ietf.org>
List-Help: <mailto:architecture-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/architecture-discuss>, <mailto:architecture-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Jan 2020 21:49:56 -0000

FWIW, there were also problems with keeping the test suites up-to-date with the errata.  Bugs could be fixed in the standards and incorporated into implementations faster than the people developing the test suites could keep up (or wanted to). Of course the fixes were in the implementations before they were in the standards. And what was worse, when pointed out to them that their test-suited didn’t conform to the spec, they didn’t care. They still insisted they were right.

For some standards, it was possible to develop utilities that did a specific function and used part of a standard to do that function but didn’t need the rest of the standard. But the people running the test suites couldn’t deal with it. If it didn’t implement the entire protocol, it didn’t pass. The utility couldn’t cause any of the rest of the protocol to be generated. It conformed to the standard for everything it did. They were using the standard as it had been intended. 

But the testers were effectively bean-counters and totally stupid.

Conformance fell under the ANSI committee I chaired and I remember writing them (some military base in Arizona whose name I have thankfully forgotten) nasty letters telling them to get their act together or get out. They were doing more harm than good. Wish we had known about the IS-IS issues as well to add to the list.

One needs some way to test implementations before trying them live.  But one has to treat them (and it has to be instilled in the testers) that the purpose is to help:  obey Postel’s rule, find problems, and then determine whether it is the tests, the ’standard’ or the new implementation that is at fault. And above all if it isn’t externally visible, it is none of the tester’s business.  Even if there is ‘formal specification’, one can’t be sure that a bug is in the prose, the formal spec, the tests, or the implementation. They are all suspect. (Often formal specifications are more complex than the code. And we know what that means!)

Years before the standards issues arose, I did a survey of testing methods for our own use. I expected to find that two or three were pretty effective, pick one or two.  What I found were 20 or 25 that were all pretty effective and the advice was pick 2 or 3 and it didn’t really matter which ones as long as they were different. (Not the result I was looking for.)  ;-)

Take care,
John

> On Jan 8, 2020, at 15:58, tony.li@tony.li wrote:
> 
> 
>> Yes, i was thinking of ISO PICS. But i lost track of those ISO standards
>> since the 90th. I guess i would have to see a successfull and still
>> currently used standard and how its PICS do or do not help. Just
>> not alot of those ISO specs left in wide deployments, right.
>> X.500 ? Maybe i can get more insight from the security community.
> 
> 
> You need not go that far.  Remember IS-IS?  Still thriving.  
> 
> And yes, my experience with PICs and test suites comes from that.
> 
> 
>>> What it does do is to encourage people to write ???conformance test suites???.  These then get sold to unsuspecting customers and end-users.  Unfortunately, the quality of such suites is so low that you spend way, way, way more time debugging the test suite and you never find bugs in your implementation.
>> 
>> This is just the worst possible outcome, and i am sure one can learn
>> from those bad experiences. Just think of taking a more organized
>> step towards being able to collect protocol implementation information
>> from the industry. Right now every WG who wants to raise the level of
>> an IETF protocol, e.g.: to(wards) full standard is coming up with a
>> questionaire in an ad-hoc fashion (we're just doing this in Multicast
>> for IGMP/MLD). 
> 
> 
> You can do that if you like, but the one proven, effective method is interoperability testing.
> 
> Running code trumps questionaires, conformance test suites, theory, and mandate.
> 
> 
>> My point was: If the only murphy you have is competitive pressure, it works not
>> it there is only limited competition, such as a total of maybe no
>> more than 3 competing national infrastructures. 
> 
> 
> And countries that stifle competition inflict vendor lock on themselves.
> 
> 
>> See above. The best i think we (IETF) can do is to educate better about
>> this by appropriate documents. Certainly some additional protocol work
>> will result from that. I for once had simple slides 10 years ago how to fixup
>> non-dual plane networks to be dual-plane. And maybe we can come up with
>> questionaires to get better numbers from actual deployments (the
>> PICS discussion).
> 
> 
> The IETF’s job is not education. It does not work as the education is not requested
> and not welcome. The IETF’s job is standards and it needs to focus on that.
> 
> 
>> Figuring out ideas for a complete ecosystem of monetization and
>> deployment between competition and regulation is better left for
>> an ongoing bar-bof with emphasis on bar.
> 
> 
> I concur. Let’s stop fooling ourselves into thinking that anything that the IAB/IETF writes has any impact
> on regulation or economics.
> 
> 
>> Actually one of the interesting conclusions was that dual-plane in
>> many cases is really free but deployments often just don't fully
>> utilize it.
> 
> 
> I’d agree that it’s close to free.  If you accept that you need a dual-plane approach for resiliency, then
> adding a second vendor to the mix is relatively easy. There’s additional cost because of decreased 
> purchasing volume. There’s additional costs in management and operations.
> 
> However, as compared to the costs of an unnecessary outage, it’s still trivial.
> 
> Tony
> 
> _______________________________________________
> Architecture-discuss mailing list
> Architecture-discuss@ietf.org
> https://www.ietf.org/mailman/listinfo/architecture-discuss