Re: Thought experiment [Re: Quality of Directorate reviews]

Keith Moore <moore@network-heretics.com> Thu, 07 November 2019 21:15 UTC

Return-Path: <moore@network-heretics.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1BA881209B0 for <ietf@ietfa.amsl.com>; Thu, 7 Nov 2019 13:15:12 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a8t6EthKxy4M for <ietf@ietfa.amsl.com>; Thu, 7 Nov 2019 13:15:10 -0800 (PST)
Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AD2A11209C8 for <ietf@ietf.org>; Thu, 7 Nov 2019 13:14:58 -0800 (PST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id CB009209B5; Thu, 7 Nov 2019 16:14:57 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Thu, 07 Nov 2019 16:14:57 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=jexkWyoHwvz8/b8z4kP2GWyy+Tuw1zeCoEU31HcBV RY=; b=n1S/xrA53E/Q1Yk80NgmSP4IrgaFoUUPUAipzvBdyPNeja2ncaDTxagRU +0LsgnW6bSz/RyI5PfE89Jrl2/8XWtGcdl5yWSGdsMKPPZsyZAMxHO8j9te/MRfp fuYLFHZQw3FarPSMf04CtP1BTtaTXoULYYytmv4PkjoJ0cN1l5RffWgoRMuoxvW+ u5pF6s+HsLoarhiwt0ujjSyLxLce8uKi3yQk+tQU1crnKTIRAfnAqHQHyWpjgMu8 As+PDL6RZjtjGorq3fuitZi82E3gcBmYVE2O/+rGUyddvDgfUHzks3b32InnrhXq duLZtklGkDhABTu85DEohRCiEBR5Q==
X-ME-Sender: <xms:UInEXX8kRf_4Yxj1P_TcD9qbDBgLEi98T6MPPZWBZYnHHAw0wMB7QQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrudduledgudegjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuvfhfhffkffgfgggjtgfgsehtkeertddtfeejnecuhfhrohhmpefmvghi thhhucfoohhorhgvuceomhhoohhrvgesnhgvthifohhrkhdqhhgvrhgvthhitghsrdgtoh hmqeenucfkphepuddtkedrvddvuddrudektddrudehnecurfgrrhgrmhepmhgrihhlfhhr ohhmpehmohhorhgvsehnvghtfihorhhkqdhhvghrvghtihgtshdrtghomhenucevlhhush htvghrufhiiigvpedt
X-ME-Proxy: <xmx:UInEXcjgDqG_v3a7bggGxSf_V3Klo55YsIWLk5esB7mmJ-SOlTp3OA> <xmx:UInEXbWtEi3XIGSLAVYKJx1iX9sv4FECcJRcSY0NRG9l5GE7sAl61A> <xmx:UInEXaEJbQNBUODyq9uDU_M2NHzsEckjOokzLPvbO6_nQcRJvwx2hg> <xmx:UYnEXW_NN2iFLeHAVufcPE4gL7sqEMad_sGXe-iW-nU9DDCtfSPXDQ>
Received: from [192.168.1.97] (108-221-180-15.lightspeed.knvltn.sbcglobal.net [108.221.180.15]) by mail.messagingengine.com (Postfix) with ESMTPA id C9302306005B; Thu, 7 Nov 2019 16:14:55 -0500 (EST)
Subject: Re: Thought experiment [Re: Quality of Directorate reviews]
To: Nico Williams <nico@cryptonector.com>
Cc: ietf@ietf.org
References: <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <91686B28-9583-4A8E-AF8A-E66977B1FE13@gmail.com> <012b9437-4440-915c-f1f9-b85e1b0be768@gmail.com> <20191107014849.GC12148@localhost> <57465486-71b1-a87a-fa8c-bad7157f9025@gmail.com> <3caeb4cf-b92b-99fd-77df-7b1aef3e2979@network-heretics.com> <20191107194408.GF12148@localhost>
From: Keith Moore <moore@network-heretics.com>
Message-ID: <695f84a4-683d-ab48-3925-840c6c5134d3@network-heretics.com>
Date: Thu, 7 Nov 2019 16:14:54 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <20191107194408.GF12148@localhost>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/SVZO-rLq-SzFzpVzn7Me7S1LGRI>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2019 21:15:12 -0000

On 11/7/19 2:44 PM, Nico Williams wrote:

>> If we want there to be a prototype "just for testing" status, it should
>> probably be called something other than Proposed - the name has come to mean
>> something else in IETF context.   And we should deliberately change one or
> We have it: it's Experimental.  But it's essentially seen as a negative,
> thus underused.

Experimental doesn't imply that there's an intention to standardize a 
protocol, though this does happen in rare cases.

Though now that I think about it, the right way to denote prototype 
status is probably to designate specific Internet-Drafts rather than to 
publish them as RFCs.   RFC publication isn't designed for temporary 
documents.

>> more protocol elements to make the standard incompatible with
>> already-deployed implementations.   The trick would be to keep that
>> information from leaking out beforehand.    Maybe give the RSE /N/ options
>> for things to change that are selected at random, similarly to the way
>> nomcoms are selected.
> Are you proposing that there be a required backwards-incompatible change
> so as to force... what exactly?  That the old thing does not get
> deployed?  I assume it will mostly discourage use of the new track.

The purpose is to remove some of the incentives to deploy protocol 
implementations before they're approved, while still allowing 
implementations and meaningful interoperability tests. When things are 
prematurely deployed it puts pressure on the WG and/or IESG to accept 
specifications that have significant flaws. It can also harm IETF's 
reputation when implementations don't meet the specification or don't 
interoperate.

> The risk with deploying prototypes is that a backwards-incompatible
> change will be needed, so perhaps early reviews should focus only on
> identifying features at risk of such changes and requiring that the
> experimental protocol cover upgrades or reduce the risk.
IMO one of the primary reasons for early reviews is to learn as early as 
possible whether an intended direction or protocol design choice would 
create or exacerbate tussles that could be addressed, and/or have 
adverse effects on other interests that might compel a different 
direction or choice.    It goes beyond "identifying features" because 
it's not always individual features that cause the conflicts.
>> But my guess is that the only way to get buy-in for Prototype specifications
>> would be to convincingly promise faster development.
> Yes.  That includes faster RFC-Editor turnarounds.  If we remove the
> other bottlenecs, then RFC-Editor queue time will become the next
> bottleneck to address.

I was thinking earlier today that the biggest waster of time and energy 
might be when IETF invests years of volunteer effort on specifications 
that turn out to be not very useful.   Even worse might be when IETF 
increases complexity (of implementations, operations, etc.) that doesn't 
result in a tremendous benefit to the Internet user community.

(It's easy to pay more attention to things that are easily measured, 
than those that are not, even though the easily measured quantities 
might not be the most relevant ones.)

Keith