Re: Things that used to be clear (was Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)

[Nico Williams <[e-mail]>]

On Fri, Jul 05, 2019 at 01:02:18PM -0400, Ted Lemon wrote:
> On Jul 5, 2019, at 12:31 PM, Nico Williams <[e-mail]> wrote:
> > At IETF we have commitment reviews, naturally, but we only have
> > inception reviews for WGs and their initial work items, but no inception
> > reviews for subsequent work items, and no interim reviews.
> 
> Isn’t a call for adoption an inception review?   If not, it should be.

I was referring to review from *outside* the WG.  Calls for adoption are
typically intra-WG only.

Reviews by a shepherd/AD, directorates, the IESG, and IETF (IETF LC) are
commitment reviews.

The WG spinup process is an inception review for the WG and its initial
work item slate.

We have no formal external-to-WGs interim review process.

I have seen ADs ask for interim reviews by experts outside a WG, so we
do have an informal interim review process.  And maybe that should be
good enough provided we engaged it *often* -- currently we don't.

> We actually kind of suck at commitment reviews, in a lot of cases: the
> work is done, after all, and everybody who thought it was a good idea
> at adoption time is tired of looking at it.   So it’s not uncommon
> (and I’ve seen it on documents I’ve finished) that the document is
> done and polished, and then there’s no response at all to the WGLC.

I'd say this is actually us being bad at interim reviews.  Our
commitment review process is very good, but very heavy-duty and thus
slow and painful.

By reserving all that pain for the very end of the process -precisely
when the WG participants are out of steam- the end result sucks.  The
problem isn't that we're bad at commitment reviews, rather the opposite
combined with the lack of interim reviews.  All the _surprises_ come
at the end, when authors/WGs least want them.

> [...]
>
> The problem the IETF has is that unlike at your former job, there is
> no way to enforce this process other than by killing documents, and
> that often only impacts the people who are working on them and the
> people who would benefit from publication, and not all the people who
> thought it was a good idea at adoption time, but have wandered off
> during the course of the work.

Ok, so let's fix that.

My take is that a) directorates and shepherds/ADs should be expected to
provide interim reviews as requested, b) WG chairs should request them
when the ether the chairs believe, or the WG has consensus that, "now is
a good time for interim review.

Interim reviews should also be announced on the [e-mail] list, but
review feedback should be on WG list.  This invites interim reviews from
the wider community as well.

Now, I don't know how to incentivize the wider community to provide
interim reviews.  Reviewing I-Ds is really time-consuming and energy
sapping for me personally -- the biggest problem is making the time and
finding a way to get it funded when it's a lot of time.

I suspect that the directorates are short on cycles too.  I get that
what I'm proposing may not be feasible at this time.

Nico
--