Re: [Icar] Input based on SIRS experience

["Spencer Dawkins" <[e-mail]>]

Dave,

Thanks for providing this input. Have I ever thanked you and Brian
publically for taking a swing at SIRS? It was an omission. Thank you.

I have a couple of comments inline.

Spencer

----- Original Message ----- 
From: "Dave Crocker" <[e-mail]>
To: "Icar (E-mail)" <[e-mail]>
Sent: Saturday, January 10, 2004 11:08 AM
Subject: Re: [Icar] Input based on SIRS experience


> Adding my own thoughts to the archive, since the topic came up, and
with the
> hope it can added to the discussion:
>
> It was not much effort to get a core of reviewers signed up.
>
> This suggests a good core of community motivation to help.  We have
not needed
> more, so we do not know whether an effort like this can attract
_enough_
> qualified reviewers.

Yeah, Margaret is also coming up with 200-300 reviewers needed, and we
still have no idea how many we can get. :-{

>
> I am not sure how to interpret the low number of review requests.
>
> It is easy to say that it is due to lack of community awareness or
interest or
> trust in the SIRs effort.  Certainly we have not done as much
marketing as we
> might have wished. However this is view is confounded by the very
slow metabolic
> rate of the IETF and the fact that the effort began at the beginning
of a
> summer.

I concur completely with each point.

>
> No doubt there probably are other factors.  Cliches like 'tipping
point' and
> 'crossing the chasm' were created for just this issue. For example,
it may be
> that it is more difficult to change existing working group efforts
than it is to
> get integrated in with new(er) ones. Certainly we have not done any
serious,
> sustained effort to get working groups to request reviews.

Yes.

>
> On doing the administrative side of this effort, there are some
things that I
> suspect are strategic, besides the relatively obvious fact that this
really does
> take some ongoing, administrative effort, more than I had expected.
(But, then,
> I always underestimate these things.)
>
> It needs to be easy to have a record of the review process for a
document and,
> by derivation, an ability to see the whole set of review efforts.
There should
> be a central point for tracking review efforts and a central point
for
> accessing reviews.

I agree, and reviews should be tightly tied to documents (easy to see
what reviewers think).

>
> Besides the obvious, local and tactical benefit to the individual
working group,
> this permits the community to get the ability to review the _set _of
reviews and
> get a sense of the effort and result of doing reviews.  In other
words, it has
> marketing value.  It makes it easy for others to see that there is
review
> activity and it makes it easy for them to evaluate the nature and
quality of
> those reviews.  This will make them more comfortable getting reviews
for their
> own efforts and will help them decide on what reviewers to ask for.
Whether this
> needs to be a permanent, formal archive, like RFCs, an ephemeral,
formal
> archive, like I-Ds, or an unofficial, ephemeral archive like a WG
mailing list,
> has not been resolved.
>
> My own thought is that formal publication (RFC and even I-D) is far
too much
> overhead.  The only existing mechanism is the mailing list. Creating
a new
> mechanism strikes me as too much effort, and added effort is a very
large
> barrier to adoption of this process.

This may be correct, but it's tragic, if so. Most of our archives
aren't that tractable, and some are REALLY grim (whether because of
spam or simply because of posting volume).

>
> Opening up the IETF Tracker to working group chairs, and adding the
review
> construct and/or allowing free-form entries would help this
enormously.
> (Free-form means that the IETF does not have to formally adopt the
thing being
> tracked.  This means reviews -- and other activities -- could be
tracked much
> sooner than would be possible if we first have to get IETF-wide
approval of the
> review process.

I guessed that by "free-form" you were thinking of  using the
mechanism to track reviews of documents that hadn't been adopted as WG
documents yet. That's also a question.

>
> I am not sure whether selection of reviewers, by requesters,  needs
to be
> different than we set up.
>
> A web page with a list of bios, and a mailing list for sending
requests to is
> probably fine.

The mailing list violates a concept Ted Hardie talked about in
Minneapolis - that appeals to individuals work better than appeals to
groups in the IETF. I think Ted is right. Requesters will really have
to be bothered to wonder who needs to review a document, rather than
posting a request and wishing reviewers would materialize.

>
> The public discussion about reviewing shows that the community is
still working
> on clarifying what types of reviews are needed, when, and how to use
them.
>
> There is also the small matter of the SIRS reviews that have been
done.
>
> Have they been they type that are needed?  Should they be done
differently?  We
> have not tried to assess this. Certainly we cannot expect regular,
widespread
> use of a review process until working groups know what to ask for,
or at least
> what to expect they will get and how they should use the results.

You are such a social scientist. How inconvenient that you should ask
a question when we have no idea what the answer is!


_______________________________________________
Icar mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/icar