Re: Idea for a process experiment to reward running code...

Barry Leiba <barryleiba@computer.org> Sat, 01 December 2012 22:22 UTC

Return-Path: <barryleiba.mailing.lists@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6930F21E80AC for <ietf@ietfa.amsl.com>; Sat, 1 Dec 2012 14:22:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.977
X-Spam-Level:
X-Spam-Status: No, score=-102.977 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UzpY5moSZNSb for <ietf@ietfa.amsl.com>; Sat, 1 Dec 2012 14:22:09 -0800 (PST)
Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by ietfa.amsl.com (Postfix) with ESMTP id 56F7E21E803F for <ietf@ietf.org>; Sat, 1 Dec 2012 14:22:09 -0800 (PST)
Received: by mail-lb0-f172.google.com with SMTP id y2so1468809lbk.31 for <ietf@ietf.org>; Sat, 01 Dec 2012 14:22:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=kE4L6YlzWSmQx0hKJbSsT9qpeMjrO3o0/HSDwUGJdVI=; b=dUVr7GY8ivOeVUE59KJyUnmhIOIdRCMmwge7RI7B0KjZxxxtW5WgV9dCThZzFRMiff giptm+vhzoI3WaSMUjTpdQuFtI3jfyzxkFZXpMAylfAy5M4+l3pBvM4AFaQXGnOj29k4 3rOatPLati2rxQX1dqTIdtdh9whYjJaEbE/6K54gyxWp8KlG24DHImo6dW66EBq8P6Bi JkGaKH0VISze/URPgEyPamGHVlGiaP2hzyCkpiLEn/RtsRuyGZlKVe7FF4kvRn37c4kS zNR38bJJU7I0FrCEM/oZIExFhisU9YUkDrpZT0lWGEEWfFCqTq6s0W8G5bJ5lojLPvHc z8Ng==
MIME-Version: 1.0
Received: by 10.152.108.197 with SMTP id hm5mr5258302lab.45.1354400528301; Sat, 01 Dec 2012 14:22:08 -0800 (PST)
Sender: barryleiba.mailing.lists@gmail.com
Received: by 10.112.12.166 with HTTP; Sat, 1 Dec 2012 14:22:08 -0800 (PST)
In-Reply-To: <50BA64AB.3010106@cs.tcd.ie>
References: <50BA64AB.3010106@cs.tcd.ie>
Date: Sat, 1 Dec 2012 17:22:08 -0500
X-Google-Sender-Auth: JG7e_ZQkOCzJHrkT5uVJuphm10g
Message-ID: <CAC4RtVD0x3JsPR8qJ1BbVytU-yL96z-Sbr5hW9GetnEX-U3FQQ@mail.gmail.com>
Subject: Re: Idea for a process experiment to reward running code...
From: Barry Leiba <barryleiba@computer.org>
To: Stephen Farrell <stephen.farrell@cs.tcd.ie>
Content-Type: text/plain; charset=ISO-8859-1
Cc: IETF-Discussion <ietf@ietf.org>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 01 Dec 2012 22:22:10 -0000

> [1] http://tools.ietf.org/id/draft-farrell-ft

I have a serious problem with the premise of the proposal.

The short version is that if the process we're talking about is
useful, we should not shortcut it as a "reward" for anything.  And if
it's not useful, we should not be doing it, regardless of whether we
want to provide incentives or not.

In particular:

   1.  Working group last call (WGLC), IETF last call (IETF-LC) and Area
       Director (AD) review all run in parallel over the same two-week
       period.

Note that WGLC is not a part of our formal process, and that AD
Evaluation can take as little time as it takes for the responsible AD
to click the "Go directly to Last Call Requested" button in the
datatracker.  Therefore, making this step happen today is entirely up
to the working group chairs and the responsible AD.  Any chair who
thinks that a document has had enough review in the WG already can
send the document to the AD directly.  That chair can also ask the AD
to expedite her review and get Last Call started ASAP, and it's up to
the AD to decide whether that's appropriate.  We don't need a process
experiment for that.

That said, AD Evaluation often (VERY often) turns up things that would
otherwise come up in Last Call comments, including in directorate and
review team reviews.  It often corrects problems or gaps in the
protocol specification, issues with IANA requests, security issues,
and so on... which would take up a lot of other people's time as the
Last Call review goes on.  An AD who has been following the document
already might be happy going directly to Last Call Requested with just
a quick glance at the document (as I have with some documents), but
might have extensive comments about other documents, regardless of
whether there are implementations or not.

Kicking off Last Call is something that requires AD judgment, not some
sort of "reward" system.

   2.  Only comments that would be "DISCUSS-worthy" according to the
       IESG Discuss Criteria [DCRIT] need be handled during last call.
       Other comments can be handled or not, at the authors/editors
       discretion.

I'm puzzled by this.  First, a lot of things fall under the DISCUSS
criteria, but ADs decide not to give them DISCUSS status, depending
upon the situation.  I guess in item 3 you cover who decides this, but
as it's specified here it seems entirely too fluffy.  And it appears
to devalue the non-blocking comments more than they already are.
Again, if our process of making comments *of all sorts* has value,
then we shouldn't short-cut it.  It's already the case that the
responsible AD and the IESG as a whole get to decide which Last Call
comments "need to be handled" and which don't, so this really isn't a
process change either.  But I wouldn't want to say outright that
last-call comments that don't meet DISCUSS criterial will likely be
ignored in some "fast path" process that "rewards" a protocol for
having implementations.

   3.  After fast-track last-call, the document must either be returned
       to the WG, or else enter IESG review as soon as any changes
       required are made.  The relevant AD makes the decision as to
       whether changes are required and when those are completed
       sufficiently to move to the next stage.

This is no different to what we normally do: after Last Call ends, the
document goes into "Waiting for AD Go-Ahead" state, and the
responsible AD decides whether it's ready for IESG Evaluation or not.

I very strongly dislike the idea of saying that we'll "reward"
something by trimming (by a mere two or three weeks; what real value
does that have?) a process that we consider as adding value to our
documents.  And, again, if we think it doesn't add value, then let's
just do this as standard procedure, as we already can today, with no
process experiment at all.

Just tell all your WGCs that they can skip WGLC, and you'll send their
docs straight into Last Call as soon as you get them.

Barry