Re: [arch-d] draft-iab-rfc7500-bis-00 - new principle of measureability

Jay Daley <jay@daley.org.nz> Wed, 26 June 2019 23:16 UTC

Return-Path: <jay@daley.org.nz>
X-Original-To: architecture-discuss@ietfa.amsl.com
Delivered-To: architecture-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CB728120177 for <architecture-discuss@ietfa.amsl.com>; Wed, 26 Jun 2019 16:16:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rA4rv0cjFK_O for <architecture-discuss@ietfa.amsl.com>; Wed, 26 Jun 2019 16:16:15 -0700 (PDT)
Received: from 412514.vps-10.com (412514.vps-10.com [212.48.70.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 59E89120600 for <architecture-discuss@ietf.org>; Wed, 26 Jun 2019 16:15:23 -0700 (PDT)
Received: from wintermute.local (unknown [158.140.230.105]) by 412514.vps-10.com (Postfix) with ESMTPSA id 244CC4E0F98; Thu, 27 Jun 2019 11:15:17 +1200 (NZST)
Authentication-Results: 412514.vps-10.com; spf=pass (sender IP is 158.140.230.105) smtp.mailfrom=jay@daley.org.nz smtp.helo=wintermute.local
Received-SPF: pass (412514.vps-10.com: connection is authenticated)
From: Jay Daley <jay@daley.org.nz>
Message-Id: <6CB8D3FA-B4D2-4A49-9E0E-DE2FCA28F02B@daley.org.nz>
Content-Type: multipart/alternative; boundary="Apple-Mail=_56087ED7-56F6-4F5C-8BA6-BE6020003239"
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Date: Thu, 27 Jun 2019 11:15:14 +1200
In-Reply-To: <E7428B2B-2E90-4962-B5D1-75DD2E4ADD62@apnic.net>
Cc: "architecture-discuss@ietf.org" <architecture-discuss@ietf.org>, "iab@iab.org" <iab@iab.org>
To: Geoff Huston <gih@apnic.net>
References: <69EAE468-64DC-40C8-9D49-BEAEA0E7EDF3@daley.org.nz> <E7428B2B-2E90-4962-B5D1-75DD2E4ADD62@apnic.net>
X-Mailer: Apple Mail (2.3445.104.11)
Archived-At: <https://mailarchive.ietf.org/arch/msg/architecture-discuss/cgkx_eY9Wn7KS8YxyBRancGHTWQ>
Subject: Re: [arch-d] draft-iab-rfc7500-bis-00 - new principle of measureability
X-BeenThere: architecture-discuss@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: open discussion forum for long/wide-range architectural issues <architecture-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/architecture-discuss>, <mailto:architecture-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/architecture-discuss/>
List-Post: <mailto:architecture-discuss@ietf.org>
List-Help: <mailto:architecture-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/architecture-discuss>, <mailto:architecture-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 26 Jun 2019 23:16:18 -0000


> On 27/06/2019, at 11:02 AM, Geoff Huston <gih@apnic.net> wrote:
> 
> OK, I’ll bite.
> 
> What _exactly_ do you mean by "the performance of an IANA registry”?


Performance in this context is the measurement of processes and services provided by the IANA registry.  This is then assessed against expectations set in policies.

For example,

- Availability of the well known locations for protocol identifiers.  This could be measured by recording each outage. 

- Responsiveness of publication.  This could be measured by recording how long it takes from the publication of an RFC that includes a new protocol identifier for it to be registered and published.


cheers
Jay


> 
> 
>> On 27 Jun 2019, at 7:44 am, Jay Daley <jay@daley.org.nz> wrote:
>> 
>> I support the principles as documented but feel that there is one more that needs to be added:
>> 
>> * Measurability
>> The operations of IANA registries must include the collection and publication of sufficient data, in accordance with the other principles of openness and transparency, to allow their performance, both current and historical, to be measured and to assess whether they are  delivering their services in accordance with the other principles and the registry policies.
>> 
>> 
>> My reasoning to add this as a specific principle is because any assessment of the performance of an IANA registry requires data.  However, I have too often come across the situation where openness, transparency and even accountability are insufficient to ensure that this happens.  Common excuses include "we don’t collect the data", "the data belongs to our customers not us", "the data is unreliable", "the data we collect doesn’t measure our performance" and so on.  Adding this as an explicit principle ensures that data is collected and that it is fit for purpose and collected under the terms that allow it to be used for this purpose. 
>> 
>> 
>> As a background to this comment I should note that I was part of the team that designed the naming functions SLA for PTI and an inaugural member of the Customer Standing Committee within ICANN that assesses and reports on PTI’s delivery of the IANA naming function in accordance with the SLA.
>> 
>> cheers
>> Jay
>> 
>> 
>> -- 
>> Jay Daley
>> jay@daley.org.nz
>> +64 21 678840
>> skype: jaydaley
>> twitter: @jay_daley
>> 
>> _______________________________________________
>> Architecture-discuss mailing list
>> Architecture-discuss@ietf.org
>> https://www.ietf.org/mailman/listinfo/architecture-discuss
> 

-- 
Jay Daley
jay@daley.org.nz
+64 21 678840
skype: jaydaley
twitter: @jay_daley