RE: Qualitative Analysis of IETF and IESG trends (Re: Measuring IETFand IESG trends)
"Ross Callon" <rcallon@juniper.net> Wed, 25 June 2008 17:20 UTC
Return-Path: <ietf-bounces@ietf.org>
X-Original-To: ietf-archive@megatron.ietf.org
Delivered-To: ietfarch-ietf-archive@core3.amsl.com
Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AA3183A69E6; Wed, 25 Jun 2008 10:20:32 -0700 (PDT)
X-Original-To: ietf@core3.amsl.com
Delivered-To: ietf@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 181123A683B for <ietf@core3.amsl.com>; Wed, 25 Jun 2008 10:20:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.407
X-Spam-Level:
X-Spam-Status: No, score=-6.407 tagged_above=-999 required=5 tests=[AWL=0.192, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kzuzcFEVwDb6 for <ietf@core3.amsl.com>; Wed, 25 Jun 2008 10:20:29 -0700 (PDT)
Received: from exprod7og110.obsmtp.com (exprod7og110.obsmtp.com [64.18.2.173]) by core3.amsl.com (Postfix) with ESMTP id BD2673A6A21 for <ietf@ietf.org>; Wed, 25 Jun 2008 10:20:29 -0700 (PDT)
Received: from source ([66.129.228.6]) by exprod7ob110.postini.com ([64.18.6.12]) with SMTP; Wed, 25 Jun 2008 10:18:55 PDT
Received: from p-emlb02-sac.jnpr.net ([66.129.254.47]) by p-emsmtp02.jnpr.net with Microsoft SMTPSVC(6.0.3790.3959); Wed, 25 Jun 2008 10:19:23 -0700
Received: from emailwf1.jnpr.net ([10.10.2.33]) by p-emlb02-sac.jnpr.net with Microsoft SMTPSVC(6.0.3790.3959); Wed, 25 Jun 2008 10:19:22 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Subject: RE: Qualitative Analysis of IETF and IESG trends (Re: Measuring IETFand IESG trends)
Date: Wed, 25 Jun 2008 13:19:19 -0400
Message-ID: <3525C9833C09ED418C6FD6CD9514668C0424C555@emailwf1.jnpr.net>
In-Reply-To: <486267E0.8080704@qualcomm.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: Qualitative Analysis of IETF and IESG trends (Re: Measuring IETFand IESG trends)
Thread-Index: AcjW2me11VKE/UqiRYyvPukYBslWCwABjK/g
References: <20080624203548.D3A8D3A67FD@core3.amsl.com><48622DEB.7060403@piuha.net> <486267E0.8080704@qualcomm.com>
From: Ross Callon <rcallon@juniper.net>
To: Lakshminath Dondeti <ldondeti@qualcomm.com>, ietf@ietf.org
X-OriginalArrivalTime: 25 Jun 2008 17:19:22.0832 (UTC) FILETIME=[9C4FBD00:01C8D6E7]
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IETF Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/pipermail/ietf>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: ietf-bounces@ietf.org
Errors-To: ietf-bounces@ietf.org
I think that this is a very good partial summary of the trends, and I am also concerned about these. My personal opinion only: I think that we need stronger guidelines on what is a valid discuss, and what is not. We might need some mechanism for the WG or the IETF or the IAB to override a DISCUSS that does not represent consensus. As one example, personally I would prefer to not allow DISCUSS's for the purpose of motivating future or ongoing work -- there are lots of people associated with the IETF who would like to get their names on drafts, and if work is needed it should be possible to motivate it without stopping progression of some different related draft. Also, if work is being done motivated solely by the desire to get past a DISCUSS comment and without any intention of ever deploying the result, then there is a very real possibility that the work might not result in the most practical solution. I think that producing and coming to consensus on these guidelines might be a significant challenge. However, I think that this is needed. Ross -----Original Message----- From: ietf-bounces@ietf.org [mailto:ietf-bounces@ietf.org] On Behalf Of Lakshminath Dondeti Sent: 25 June 2008 11:45 To: ietf@ietf.org Subject: Qualitative Analysis of IETF and IESG trends (Re: Measuring IETFand IESG trends) Hi all, I am concerned by the following trends: * Number of outstanding Discusses is growing. (Thanks to Jari's data) * The extent of text changes as part of Discuss Resolution is increasing (I have only anecdotal evidence on this; perhaps others have statistics). * In some cases, members of the IESG are pretty much writing core parts of documents (or worse yet make authors iterate until the text is satisfactory), overruling WG consensus, based on personal opinions or citing solicited reviews. There are a number of derivative trends as well: * ADs who cannot convince working groups seem to use the threat of DISCUSS to get their way. * I would have thought document shepherds represent and defend working group consensus and shepherding ADs defend the completeness, accuracy, relevance of the drafts and defend their assessment of the IETF consensus. But, increasingly, it seems to fall on an AD holding a DISCUSS position, and authors to discuss and agree on text which becomes finalized without any other review. (Note: I am a guilty party in some of these cases, as document author and document shepherd. In all cases, I seem to have looked for an expedient way out rather than find a solution to what seems to be problematic). * One of the new trends seems to be to use DISCUSS to include applicability statements in current drafts to avoid potential DISCUSS positions on future drafts. That or some variation of that is status quo. It should not be that way. I would like to see a better definition of the roles of the WG, authors of a document, document shepherds, shepherding ADs and other ADs in our standards process. I would like to hear others' opinions (I was going to put together a draft with some ideas on how we might define these roles, but I want to hear others' thoughts before I do that) on this topic. thanks, Lakshminath On 6/25/2008 4:37 AM, Jari Arkko wrote: <deleted text> > > That being said, it is beneficial to understand what is happening and > what changes are occurring in the process. Or understand where > improvements might have a negligible vs. visible impact. > > One of the things that the IESG has been concerned about recently is > that the number of outstanding Discusses is growing. We talked about > this in our May retreat and identified some actions to help reduce this > problem. For instance, better tool support so that the ADs would better > see the different things that are waiting for their action, getting the > document shepherds better involved in the resolution process, informing > authors how they should respond to Discusses, using RFC Editor notes to > make small changes when the authors are slow in producing a new version, > better checks of documents before they come to telechats (e.g., to > ensure that formal syntax in a document is free of errors), etc. These > would be in addition to the usual things we'd do, like debate whether > the Discuss was within the Discuss criteria, whether the issue is real, > try to ensure that the AD and the authors are being responsive over > e-mail, etc. > > Another interesting area to think about is the time that our processes > take. For instance, documents that go through me take on the average > five months from publication request to approval. But there is a lot of > variation. This time includes everything from AD review, IETF Last Call > to IESG review and waiting for document revisions, etc. One interesting > piece of information is that documents that require no revision during > this process are very rare, but they go through the system about five > times as fast. If we look at the (unreliable) substates, they appear to > indicate that the IESG processing time is divided roughly 1:2:1 for > waiting on AD, authors, and mandatory process waits like last call periods. > > I am working to extend the analysis a little bit further by including > individual draft and WG document stages. You see some of the results of > this in the third URL above, but I'd like to understand what fraction of > the overall draft-smith-00 to RFC time is taken by the different stages > for all IETF work, and how the stages have developed over time. > > Comments and suggestions on what would be useful to measure are welcome. > > Jari > > _______________________________________________ > IETF mailing list > IETF@ietf.org > https://www.ietf.org/mailman/listinfo/ietf > _______________________________________________ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf _______________________________________________ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
- Appeal against IESG blocking DISCUSS on draft-kle… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… Lakshminath Dondeti
- RE: Appeal against IESG blocking DISCUSS on draft… Eastlake III Donald-LDE008
- RE: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… Brian E Carpenter
- Re: Appeal against IESG blocking DISCUSS on draft… Brian E Carpenter
- RE: Appeal against IESG blocking DISCUSS on draft… Eric Gray
- Re: Appeal against IESG blocking DISCUSS on draft… Pete Resnick
- Re: Appeal against IESG blocking DISCUSS on draft… Tony Hansen
- RE: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… Robert Elz
- Re: Appeal against IESG blocking DISCUSS on draft… TSG
- Re: Appeal against IESG blocking DISCUSS on draft… TSG
- Re: Appeal against IESG blocking DISCUSS on draft… Brian E Carpenter
- RE: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… Frank Ellermann
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Crocker
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Crocker
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Crocker
- Re: Appeal against IESG blocking DISCUSS on draft… Brian Dickson
- Re: Appeal against IESG blocking DISCUSS on draft… Simon Josefsson
- Re: Appeal against IESG blocking DISCUSS on draft… eburger
- Re: Appeal against IESG blocking DISCUSS on draft… David Kessens
- Re: Appeal against IESG blocking DISCUSS on draft… Lakshminath Dondeti
- Re: Appeal against IESG blocking DISCUSS on draft… Fred Baker
- Re: Appeal against IESG blocking DISCUSS on draft… Fred Baker
- Re: Appeal against IESG blocking DISCUSS on draft… Lakshminath Dondeti
- RE: Appeal against IESG blocking DISCUSS on draft… Debbie Garside
- Re: Appeal against IESG blocking DISCUSS on draft… Marshall Eubanks
- Re: Appeal against IESG blocking DISCUSS on draft… Steven M. Bellovin
- Re: Appeal against IESG blocking DISCUSS on draft… Robert Elz
- RE: Appeal against IESG blocking DISCUSS on draft… Eastlake III Donald-LDE008
- Re: Appeal against IESG blocking DISCUSS on draft… Spencer Dawkins
- RE: Appeal against IESG blocking DISCUSS on draft… Scott O. Bradner
- Re: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… Pete Resnick
- Re: Appeal against IESG blocking DISCUSS on draft… Marshall Eubanks
- Re: Appeal against IESG blocking DISCUSS on draft… Eliot Lear
- Re: Appeal against IESG blocking DISCUSS on draft… TSG
- example TLH (was: Appeal against IESG blocking DI… Frank Ellermann
- Re: Appeal against IESG blocking DISCUSS on draft… LB
- Re: Appeal against IESG blocking DISCUSS on draft… Marshall Eubanks
- Re: Appeal against IESG blocking DISCUSS on draft… Harald Alvestrand
- Re: Appeal against IESG blocking DISCUSS on draft… Simon Josefsson
- Limits of RFC 2606 (Was: Appeal against IESG bloc… Stephane Bortzmeyer
- Re: Appeal against IESG blocking DISCUSS on draft… Bob Hinden
- RE: Appeal against IESG blocking DISCUSS on draft… Debbie Garside
- RE: Appeal against IESG blocking DISCUSS on draft… Debbie Garside
- Re: Appeal against IESG blocking DISCUSS on draft… Robert Elz
- Re: Limits of RFC 2606 Frank Ellermann
- Re: Appeal against IESG blocking DISCUSS on draft… Frank Ellermann
- Re: Appeal against IESG blocking DISCUSS on draft… Robert Elz
- RE: Appeal against IESG blocking DISCUSS on draft… Dave Cridland
- RE: Appeal against IESG blocking DISCUSS on draft… Dave Cridland
- Re: Appeal against IESG blocking DISCUSS on draft… Frank Ellermann
- Re: Appeal against IESG blocking DISCUSS on draft… Ralph Droms
- Re: Appeal against IESG blocking DISCUSS on draft… Spencer Dawkins
- RE: Appeal against IESG blocking DISCUSS on draft… Dave Cridland
- Re: Appeal against IESG blocking DISCUSS on draft… Ned Freed
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- RE: Appeal against IESG blocking DISCUSS on draft… Debbie Garside
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- RE: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… Ted Hardie
- RE: Appeal against IESG blocking DISCUSS on draft… Pete Resnick
- Re: Appeal against IESG blocking DISCUSS on draft… Frank Ellermann
- Re: Appeal against IESG blocking DISCUSS on draft… Eliot Lear
- Re: Appeal against IESG blocking DISCUSS on draft… Ted Hardie
- Re: Appeal against IESG blocking DISCUSS on draft… Robert Elz
- Re: Appeal against IESG blocking DISCUSS on draft… SM
- RE: Appeal against IESG blocking DISCUSS on draft… Debbie Garside
- RE: Appeal against IESG blocking DISCUSS on draft… Debbie Garside
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- Re: Appeal against IESG blocking DISCUSS on draft… Randy Presuhn
- Re: Appeal against IESG blocking DISCUSS on draft… John Levine
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Cridland
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- Re: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Appeal against IESG blocking DISCUSS on draft… Pete Resnick
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- Re: Appeal against IESG blocking DISCUSS on draft… Bernard Aboba
- Re: Appeal against IESG blocking DISCUSS on draft… Harald Tveit Alvestrand
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- RE: Appeal against IESG blocking DISCUSS on draft… Bernard Aboba
- RE: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- Re: Appeal against IESG blocking DISCUSS on draft… Russ Housley
- Re: Appeal against IESG blocking DISCUSS on draft… Stewart Bryant
- Re: Appeal against IESG blocking DISCUSS on draft… Julian Reschke
- Measuring IETF and IESG trends (Was: Re: Appeal a… Jari Arkko
- Re: Appeal against IESG blocking DISCUSS on draft… Spencer Dawkins
- Re: Measuring IETF and IESG trends (Was: Re: Appe… Marshall Eubanks
- Qualitative Analysis of IETF and IESG trends (Re:… Lakshminath Dondeti
- Re: Qualitative Analysis of IETF and IESG trends … Melinda Shore
- RE: Qualitative Analysis of IETF and IESG trends … Ross Callon
- Re: Qualitative Analysis of IETF and IESG trends … Jari Arkko
- Re: Qualitative Analysis of IETF and IESG trends … Brian E Carpenter
- Re: Qualitative Analysis of IETF and IESG trends … Spencer Dawkins
- Re: Qualitative Analysis of IETF and IESG trends … John C Klensin
- Re: Qualitative Analysis of IETF and IESG trends … Lakshminath Dondeti
- Re: Qualitative Analysis of IETF and IESG trends … Dave Crocker
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Crocker
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Crocker
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Crocker
- Re: Appeal against IESG blocking DISCUSS on draft… Dave Crocker
- Re: Qualitative Analysis of IETF and IESG trends … Lakshminath Dondeti
- Re: Qualitative Analysis of IETF and IESG trends … Lakshminath Dondeti
- Re: Qualitative Analysis of IETF and IESG trends … Lakshminath Dondeti
- Re: Appeal against IESG blocking DISCUSS on draft… SM
- Re: Appeal against IESG blocking DISCUSS on draft… John C Klensin
- Re: Qualitative Analysis of IETF and IESG trends … Brian E Carpenter
- Re: Qualitative Analysis of IETF and IESG trends … John C Klensin
- Re: Qualitative Analysis of IETF and IESG trends … SM
- Re: Qualitative Analysis of IETF and IESG trends Frank Ellermann
- Re: Qualitative Analysis of IETF and IESG trends Paul Hoffman
- Re: Qualitative Analysis of IETF and IESG trends … Lakshminath Dondeti
- Re: Qualitative Analysis of IETF and IESG trends … SM
- Re: Qualitative Analysis of IETF and IESG trends … Lakshminath Dondeti
- Re: Qualitative Analysis of IETF and IESG trends … Russ Housley
- Re: Qualitative Analysis of IETF and IESG trends … Brian E Carpenter
- Re: Qualitative Analysis of IETF and IESG trends … Jari Arkko
- Re: Qualitative Analysis of IETF and IESG trends … Ted Hardie
- Re: Qualitative Analysis of IETF and IESG trends … Joel M. Halpern
- Re: Qualitative Analysis of IETF and IESG trends … Brian E Carpenter
- Re: Qualitative Analysis of IETF and IESG trends … Jari Arkko
- RE: Qualitative Analysis of IETF and IESG trends … Romascanu, Dan (Dan)