Re: Quality of Directorate reviews

Benjamin Kaduk <> Mon, 18 November 2019 10:53 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 9FADB1208F2 for <>; Mon, 18 Nov 2019 02:53:09 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.2
X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 2LQIgOVKiaPd for <>; Mon, 18 Nov 2019 02:53:08 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id C8C781200DB for <>; Mon, 18 Nov 2019 02:53:07 -0800 (PST)
Received: from ([]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by (8.14.7/8.12.4) with ESMTP id xAIAr3cW021143 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Nov 2019 05:53:05 -0500
Date: Mon, 18 Nov 2019 02:53:03 -0800
From: Benjamin Kaduk <>
To: Michael Richardson <>
Subject: Re: Quality of Directorate reviews
Message-ID: <>
References: <> <9182.1573147520@localhost> <> <> <> <> <> <> <> <>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <>
User-Agent: Mutt/1.12.1 (2019-06-15)
Archived-At: <>
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 18 Nov 2019 10:53:10 -0000

Hi Michael,

On Sun, Nov 17, 2019 at 10:58:43AM +0800, Michael Richardson wrote:
> Benjamin Kaduk <> wrote:
>     > My understanding is that most directorates have a secretary that does
>     > the assignments (secdir does, at least).
> yes, that's my understanding.
> I'd like to see more coordination between ADs (particularly Sec-ADs) and
> directorates so that the security review process can occur earlier, and so
> that any loop with the SecADs can happen earlier.
> In the case of draft-ietf-anima-bootstrapping-keyinfra, I'd have liked to get
> more attention from Christian,Jari and Russ (reviewers) and the various ADs
> earlier.  The significant reviews were done a year ago, and we are just
> finishing now.
> That's a big investment of time among the 6 or 7 people involved.

I'm not sure that I understand what you're looking for here.
In the case of,
my conclusion from looking at the review is that "this document is not
ready for publication as-is and should go back to the WG before it comes
before the IESG".  That is, I would essentially ignore the document until
the secdir reviewer is satisfied [or the authors' responses give some
indication that the reviewer is incorrect], to make more efficient use of
my time.  But it sounds like you're suggesting that I should see that
review and take it as a signal to get *more* involved with the document.
Am I missing something?

>     > By the time an AD is looking
>     > at the review next to the document it might only be a few days before
>     > the telechat where the document is up for approval, which is not really
>     > enough time to get another review in without deferring the document.
> It seems that we doing these early secdir reviews, but someone this is not
> feeding up to the ADs well enough, who then do their own review.  That's just
> not leveraging the secdir well.

In a similar vein, I'm not sure what you would see as "leveraging the
secdir well".  I don't see myself as beholden to accept the secdir review
as-is -- to me the secdir reviews are a tool that I can take advantage of
as I perform my AD duties, but it's not the only tool at my disposal, and
if I am concerned that a given document may have broad impact or contain
subtle security issues, I am generally going to do an in-depth review
myself, in addition to any other reviews that have been done.

>     > Maybe we should go get that extra review and try to remove the stigma
>     > against deferring documents; I don't have a sense for how the community
>     > would feel about that.
> I'm okay with this, but maybe the sponsoring AD and WG chairs need to be more
> active in chasing down reviewers.

Just to check: this is "seeking enough reviews from the relevant area(s)"
as opposed to "ensuring that people who did reviews respond to the updates
made because of their reviews"?

> Again, I'd like more offocial acknowledgement of the work reviewers do.
>     > And yes, the AD should look at the directorate review when it arrives,
>     > but looking only at the review and not the document being reviewed is
>     > not always enough to tell whether additional review would be valuable.
> Agreed.
> What if the Shepherd write up was had more ways to flag things?

>From a technical point of view, I think this may only be relevant when the
shepherd is not a WG chair for the WG in question -- my understanding is
that the WG chair can click a button in the datatracker to request a
directorate review at any time, which might be more helpful than just a
note in the writeup.