Re: Thought experiment [Re: Quality of Directorate reviews]

Keith Moore <moore@network-heretics.com> Thu, 07 November 2019 01:20 UTC

Return-Path: <moore@network-heretics.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4850B1208F4 for <ietf@ietfa.amsl.com>; Wed, 6 Nov 2019 17:20:22 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id alLw_rbOgzM4 for <ietf@ietfa.amsl.com>; Wed, 6 Nov 2019 17:20:20 -0800 (PST)
Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 27DF31208E6 for <ietf@ietf.org>; Wed, 6 Nov 2019 17:20:19 -0800 (PST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id AF828210B8; Wed, 6 Nov 2019 20:20:18 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 06 Nov 2019 20:20:18 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=0v6Q8kW04MqKnXmadmge7rRb/7oe10TU3OBqwho8P U4=; b=lGTWUgplaD2p83oCEg0GkxNRXGDjUJ8UUst6VJB+WsfHI6EKh6iOceYaB HbpsDmOA41NyJwLZ+awAf3fzKj1ucI+/rU0joCeaHIlHzhqaa57ko6n9PY0G19Sw S/0++fLrSatbRcrzB9dBKnkMw3U+5+D5vvMwFYwmgXexVUp8+tGxD8sIEkYwF9AX RrLTcHMkQfbVEOK0l/vigjsewFkDmk5AcJYZ7BE/CNDhvhi6PCphiSfuH9IqdAt8 Tny8gC+EVibzawTPy/C7FDWgE7T0CmKMlarCUMXjDzfKCfIc+8PL8ltgVDBCJxxv 1DUL80Lf5Abg4cWgfUK8raQNJ4+BQ==
X-ME-Sender: <xms:UXHDXXAFnAFqv33PKl1EroFMgl4N1dOsAsZ-VmGPzH-O9A-OgQcjMw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedruddukedgfedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgfgsehtke ertddtfeejnecuhfhrohhmpefmvghithhhucfoohhorhgvuceomhhoohhrvgesnhgvthif ohhrkhdqhhgvrhgvthhitghsrdgtohhmqeenucfkphepuddtkedrvddvuddrudektddrud ehnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmohhorhgvsehnvghtfihorhhkqdhhvghr vghtihgtshdrtghomhenucevlhhushhtvghrufhiiigvpedt
X-ME-Proxy: <xmx:UXHDXQIwhpMvpt7Mczl02GO9lWx5dM_p_Muu0VAHDWg9a1QE468pbw> <xmx:UXHDXSHrNbcP8B8pczfYf-AzGh7GPZbdGevO_p00YJynN4aS80cbHA> <xmx:UXHDXbG2JoMp-M8J8q6D4Wg_ueWkiWAeaVRKV4yKwdxiBgIl7LUsrw> <xmx:UnHDXTuFoG-0uK-qQknbODhqNlWolFReenSLm2UV1YsHBQfAAJpDew>
Received: from [192.168.1.97] (108-221-180-15.lightspeed.knvltn.sbcglobal.net [108.221.180.15]) by mail.messagingengine.com (Postfix) with ESMTPA id 67C9E306005F; Wed, 6 Nov 2019 20:20:16 -0500 (EST)
Subject: Re: Thought experiment [Re: Quality of Directorate reviews]
To: ietf@ietf.org
References: <157279399807.13506.13363770981495597049.idtracker@ietfa.amsl.com> <0EF64763-BA25-468A-B387-91445A61D318@gmail.com> <CAJU8_nUovmFmgNiYx0ez_1f+GPdU9xGViDYWfowEEomrn0pyDw@mail.gmail.com> <alpine.LRH.2.21.1911040841160.27600@bofh.nohats.ca> <CE06CC6D-E37F-4C90-B782-D14B1D715D4B@cable.comcast.com> <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <91686B28-9583-4A8E-AF8A-E66977B1FE13@gmail.com> <012b9437-4440-915c-f1f9-b85e1b0be768@gmail.com> <8b387d57-f056-7de3-9851-a431b17b67b4@network-heretics.com> <c97440e1-c6bb-8219-2970-357d9c45beb2@gmail.com>
From: Keith Moore <moore@network-heretics.com>
Message-ID: <fa406d85-babb-9821-047f-ad7efbdd23f5@network-heretics.com>
Date: Wed, 6 Nov 2019 20:20:15 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <c97440e1-c6bb-8219-2970-357d9c45beb2@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/aQk2AuKRePKW4UqXO0hFkfNERUQ>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Nov 2019 01:20:25 -0000

On 11/6/19 7:43 PM, Brian E Carpenter wrote:

> On 07-Nov-19 12:01, Keith Moore wrote:
>> On 11/6/19 5:54 PM, Brian E Carpenter wrote:
>>
>>> Here's a thought experiment.
>>>
>>> Update the standards process such that the approval of Proposed Standard
>>> RFCs, after an IETF last call including some specified cross-area review
>>> requirements, is done by the WG consensus process with the consent of the AD .
>> I don't think a typical WG chair is in a good position to review things
>> from a broad perspective.   The ADs are in a MUCH better position to do
>> that, precisely because they are exposed to everything that IETF does.
> But it doesn't scale, or so the ADs are telling us.

I believe them :)

Or to put it differently:  I don't think it scales so long as there is 
essentially no pushback on chartering any WG for which there appears to 
be community interest.

Note that there's not a contradiction there.  The ADs really are in the 
best position to review things from a broad perspective. AND with the 
current number of working groups and number of drafts that they're 
producing, there's too much work for the ADs to do.

But to me it looks like what doesn't scale is the community expectation 
that IETF should take on every WG for which there seems to be public 
interest, and those WGs should be able to produce as many drafts as they 
wish, with no page limit on those drafts.

(I'm not saying that there can't be some middle ground, some 
optimization of what's currently being done.   But I do believe that the 
organization needs to take a good hard look at its expectations for 
output and for "fairness" in what IETF takes on, and think about what 
the Internet needs from IETF rather than about what individuals and 
companies want IETF to do for them.)

It's long been clear to me that there's an inherent limit to the extent 
to which the IETF structure can scale, for several reasons other than 
just AD workload.   And to me that forces some hard questions about both 
the volume and scope of IETF's work.     IMO IETF should focus on 
producing fewer but higher quality and more relevant RFCs, rather than 
priding itself on the sheer number of RFCs produced.

> And my thought
> experiment wouldn't take ADs out of the loop; it would take them out
> of the detailed review work.

It would also take away the tremendously valuable perspective that they 
acquire as the result of doing (some of) that work.   We would soon be 
left without anyone who could really do a good job at broad review.

Or maybe the thing to do is to make directorates more formal and more 
recognized, and expand their roles a bit.   Maybe we should explicitly 
look at (some?) directorates as the places to cultivate broad expertise 
(others might still have narrower focus).   Then have the directorates 
tasked with not just reviewing documents and making recommendations 
about them, but also other things - e.g. analyses of problem spaces and 
tussles - that would inform AD decisions and also feed-forward to WGs.   
The buck would still stop with IESG but they'd have more explicit support.

>> Also, the WG chairs are properly concerned with the specific perspective
>> of their WGs; they know where the hard battles were fought.   Their WG
>> needs them to be in a position to defend the WG's work.   To expect them
>> to do both that and the broad review would put them in a conflicted
>> position, and it's probably the broad review that would get shortchanged.
> That's why the idea would be to make the WG chairs the *visible* approvers,
> which IMHO would significantly change their incentives.

Yes, but then they wouldn't be in a good position to support the WGs.   
A chair can't be both in the trenches in a WG AND be a reasonably 
objective reviewer from a broad perspective.   Either they're separate 
people, or we just give up on our notion of rough consensus because 
nobody would be in a good position to evaluate it.

Keith