Re: [apps-review] Suggested changes for Applications Area Review Team

SM <sm+ietf@elandsys.com> Sat, 19 March 2011 19:08 UTC

Return-Path: <sm@elandsys.com>
X-Original-To: apps-review@core3.amsl.com
Delivered-To: apps-review@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E2D843A69B8 for <apps-review@core3.amsl.com>; Sat, 19 Mar 2011 12:08:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.769
X-Spam-Level:
X-Spam-Status: No, score=-102.769 tagged_above=-999 required=5 tests=[AWL=-0.170, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2Tw10kF0snCH for <apps-review@core3.amsl.com>; Sat, 19 Mar 2011 12:08:53 -0700 (PDT)
Received: from mail.elandsys.com (mail.elandsys.com [208.69.177.125]) by core3.amsl.com (Postfix) with ESMTP id 6C7B63A69BB for <apps-review@ietf.org>; Sat, 19 Mar 2011 12:08:53 -0700 (PDT)
Received: from SUBMAN.elandsys.com ([41.136.236.222]) (authenticated bits=0) by mail.elandsys.com (8.13.8/8.13.8) with ESMTP id p2JJA7vU028850; Sat, 19 Mar 2011 12:10:17 -0700
DKIM-Signature: v=1; a=rsa-sha1; c=simple/simple; d=elandsys.com; s=mail; t=1300561819; bh=N+w6E5btdiiw9jypBRu2pOcAhWE=; h=Message-Id:Date:To:From:Subject:Cc:In-Reply-To:References: Mime-Version:Content-Type; b=sDooqfvAgzav8rePpR8ymc6rQDIl+JCr55rcESk1F+ea5jzObuYhU15UMbLEGWY5t zg4rrk1oRmuVJyFQtkjF/z3soRsbNhX6vMksTTP9diPs7DveRlNuamTHePE34Eh9pQ hfXMepiPzkLv2BB1OvoaibTrhvqw5MJr1AlBR7vg=
Message-Id: <6.2.5.6.2.20110319095600.0bc0c0d8@elandnews.com>
X-Mailer: QUALCOMM Windows Eudora Version 6.2.5.6
Date: Sat, 19 Mar 2011 12:09:04 -0700
To: dcrocker@bbiw.net
From: SM <sm+ietf@elandsys.com>
In-Reply-To: <4D84D06B.2070600@dcrocker.net>
References: <6.2.5.6.2.20110318165117.0d43e6e0@elandnews.com> <4D84D06B.2070600@dcrocker.net>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format="flowed"
Cc: Pete Resnick <presnick@qualcomm.com>, apps-review@ietf.org
Subject: Re: [apps-review] Suggested changes for Applications Area Review Team
X-BeenThere: apps-review@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Apps Area Review List <apps-review.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/apps-review>, <mailto:apps-review-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/apps-review>
List-Post: <mailto:apps-review@ietf.org>
List-Help: <mailto:apps-review-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/apps-review>, <mailto:apps-review-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 19 Mar 2011 19:08:57 -0000

Hi Dave,
At 08:48 19-03-2011, Dave CROCKER wrote:
>I suggest that all assignments MUST receive an explicit 
>acknowledgment.  I'm not sure what the timeout should be for the 
>less urgent ones, but 24 hours seems fine for the IESG (relatively urgent) one.
>
>But a default ack is not a good protocol construct for critical functions...

I agree.  However, we are not working on a protocol here.  This is a 
people issue.  If I assign you a review, I know that you will send an 
ack within 24 hours.  If you do not do that, I would wait for a 
couple of days as I know that you generally do reply to emails.

If I go for MUST receive an explicit acknowledgement, I already have 
historical evidence that it will fail in at least 50% of cases even 
if I wait five working days for an answer.

I'll be candid.  The suggested changes are to get team members to 
actually perform the assignments instead of having an Applications 
Area Review Team in name only.

The latest review was assigned to Carsten.  It falls within an IETF 
period where he will be travelling to Prague and he will be busy at 
the meeting.  I do take that into account.    My choices are simple; 
I can tell Alexey and Peter that the review cannot be done within two 
weeks or I can discuss with Carsten to find an alternative that fits 
his schedule.  I prefer the latter.  That is why I told him that he 
can take more than two weeks.  From past experience, I know that he 
will deliver on the review.

I prefer not to comment on specific cases of reviews that have not 
been performed.  Alexey and Peter probably know about them even 
though I did not discuss the cases with them.

>I usually follow a review format that has:
>
>    1. Summary of my understanding of the purpose and content of the 
> document. This is a common reviewing technique and it establishes a 
> base of factual understanding, before launching into the my 
> opinions about the quality of the content. If I have any 
> misunderstanding of the basics, it's better to surface that at the 
> outset.  I also find that the discipline of formulating the factual 
> summary helps to organize my thoughts about the draft.

Thanks for sharing this.  It would help the Apps Area ADs if the 
summary clearly states whether the document is ready for publication.

>    2. Summary of major issues
>
>    3. Inline detailed comments.  For extended documents, the 
> detailed feedback is typically extensive and not subject to 
> summarization.  I frankly do not know how to break it out into 
> clean distinctions of major/minor/nits without dramatically more 
> work.  In addition, my view of major vs. minor vs. nit is likely 
> not to match the authors'...
>
>Adding an explicit "rating" at the beginning seems like an excellent 
>idea; I'll try to remember to add one.

The clean distinctions are certainly more work and it is not always 
clear what should be major or minor.  And as you said, it may not 
necessarily match the author's or even the Apps Area ADs' view.

I'll use the review of apps-team review of 
draft-ietf-netconf-4741bis-07 as an example ( 
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg02268.html 
).  As Joshua mentioned, the I-D is an update to RFC 4741 and has 
therefore "stood the test of time".  If the reviewer gives a "Do Not 
Publish" in such a case, it won't stop publication.

Best regards,
-sm