Re: Quality of Directorate reviews

[Keith Moore <[e-mail]>]

On 11/16/19 2:08 AM, Benjamin Kaduk wrote:

> My understanding is that most directorates have a secretary that does the
> assignments (secdir does, at least).  By the time an AD is looking at the
> review next to the document it might only be a few days before the telechat
> where the document is up for approval, which is not really enough time to
> get another review in without deferring the document.  Maybe we should go
> get that extra review and try to remove the stigma against deferring
> documents; I don't have a sense for how the community would feel about
> that.

IMO there's always going to be some conflict between an AD doing his/her 
job and "what the community wants", because nobody thinks his/her own 
document is lowering the quality of IETF standards. So the pressure to 
approve documents despite serious flaws, and/or to approve documents 
without delay (even though some documents really do require more time to 
review and more eyeballs to look at them), will always be there.   There 
should not be a "stigma" associated with deferring documents if there's 
any reason at all to subject them to more scrutiny.

Neither IESG nor the RSE is the source of most delays, after all; most 
delays are in the working groups.   And yeah, people compile stats on 
the IESG and the RSE to see how well they "perform" but those stats 
don't measure the most relevant things, like the quality of IETF's 
output, or the extent to which IETF standards are utilized, or the 
extent to which they're followed (this is a different thing), or 
(especially) the degree to which IETF standards meet the Internet's 
needs.   There's an unfortunate tendency these days to pay more 
attention to quantities that are easily measured, than to those that 
actually matter.

> And yes, the AD should look at the directorate review when it arrives, but
> looking only at the review and not the document being reviewed is not
> always enough to tell whether additional review would be valuable.

Entirely agree.   I don't think IESG should presume that directorate 
reviews will always be sufficient, and to do so IMO is tantamount to 
IESG abandoning its duty.

Keith