Re: Quality of Directorate reviews

[Benjamin Kaduk <[e-mail]>]

Hi Michael,

On Sun, Nov 17, 2019 at 10:58:43AM +0800, Michael Richardson wrote:
> 
> Benjamin Kaduk <[e-mail]> wrote:
>     > My understanding is that most directorates have a secretary that does
>     > the assignments (secdir does, at least).
> 
> yes, that's my understanding.
> 
> I'd like to see more coordination between ADs (particularly Sec-ADs) and
> directorates so that the security review process can occur earlier, and so
> that any loop with the SecADs can happen earlier.
> 
> In the case of draft-ietf-anima-bootstrapping-keyinfra, I'd have liked to get
> more attention from Christian,Jari and Russ (reviewers) and the various ADs
> earlier.  The significant reviews were done a year ago, and we are just
> finishing now.
> That's a big investment of time among the 6 or 7 people involved.

I'm not sure that I understand what you're looking for here.
In the case of
https://datatracker.ietf.org/doc/review-ietf-anima-bootstrapping-keyinfra-16-secdir-lc-huitema-2018-09-29/,
my conclusion from looking at the review is that "this document is not
ready for publication as-is and should go back to the WG before it comes
before the IESG".  That is, I would essentially ignore the document until
the secdir reviewer is satisfied [or the authors' responses give some
indication that the reviewer is incorrect], to make more efficient use of
my time.  But it sounds like you're suggesting that I should see that
review and take it as a signal to get *more* involved with the document.
Am I missing something?

>     > By the time an AD is looking
>     > at the review next to the document it might only be a few days before
>     > the telechat where the document is up for approval, which is not really
>     > enough time to get another review in without deferring the document.
> 
> It seems that we doing these early secdir reviews, but someone this is not
> feeding up to the ADs well enough, who then do their own review.  That's just
> not leveraging the secdir well.

In a similar vein, I'm not sure what you would see as "leveraging the
secdir well".  I don't see myself as beholden to accept the secdir review
as-is -- to me the secdir reviews are a tool that I can take advantage of
as I perform my AD duties, but it's not the only tool at my disposal, and
if I am concerned that a given document may have broad impact or contain
subtle security issues, I am generally going to do an in-depth review
myself, in addition to any other reviews that have been done.

>     > Maybe we should go get that extra review and try to remove the stigma
>     > against deferring documents; I don't have a sense for how the community
>     > would feel about that.
> 
> I'm okay with this, but maybe the sponsoring AD and WG chairs need to be more
> active in chasing down reviewers.

Just to check: this is "seeking enough reviews from the relevant area(s)"
as opposed to "ensuring that people who did reviews respond to the updates
made because of their reviews"?

> Again, I'd like more offocial acknowledgement of the work reviewers do.
> 
>     > And yes, the AD should look at the directorate review when it arrives,
>     > but looking only at the review and not the document being reviewed is
>     > not always enough to tell whether additional review would be valuable.
> 
> Agreed.
> What if the Shepherd write up was had more ways to flag things?

>From a technical point of view, I think this may only be relevant when the
shepherd is not a WG chair for the WG in question -- my understanding is
that the WG chair can click a button in the datatracker to request a
directorate review at any time, which might be more helpful than just a
note in the writeup.

-Ben