Re: Issues around sponsoring individual documents

[John C Klensin <[e-mail]>]

Lisa,

We've discussed aspects of this topic fairly extensively
off-list, starting many months ago.  I will try to avoid
repeating that discussion here, but the summary is that I
believe the growth of work being done in nominal ISs is a
healthy adaptation to some evolution in the Applications Area.
Some of our subject matter has gotten fairly specialized and we
are seeing input from fewer generalists who have really put in
the work to understand subject matter and context.   At least
some of the IS work has developed because of a perception that
the WG process can actually produce less useful review because
it introduces a lot of noise and opinions from people who don't
know much about the subject matter and who are unwilling to
actually read and study drafts (rather than, e.g., responding
loudly to what they perceive as happening on mailing list
threads).  

The review effort for 2821bis and 2822bis is fairly symptomatic
in this regard, with a number of comments early in the process
from people who had either not bothered to study the drafts, had
not understood or accepted the constraints on an effort to move
existing documents to Draft, or wanted to use the revision
effort as a platform for completely revising the Internet email
model.  While the level of attention being paid to the work has
waxed and waned (something that can also happen in a WG), I
believe that we have actually gotten better input and review
than might have been the case with a WG, especially had that WG
drifted off into needing to educate a larger population or
dispose of irrelevant concerns.

I think that, if we have a problem, it is not having more
variations than either "WG" or single-model Individual
Submission process.   But, even today, we don't have a single
model for the latter: we have IS efforts that are really
individual work; efforts that are the result of a self-selected
but closed design team; efforts that are widely publicized to
the IETF community and discussed on public, IETF-based, lists
but not organized as WGs, and so on.  I also believe that the
true measure of the size of the Applications Area is better
reflected in the amount of work we are getting out than in a
count of WGs.

Some comments on your specific questions below, but I'm largely
in agreement with John Leslie and Martin Duerst and won't repeat
their remarks.


--On Thursday, 06 September, 2007 17:44 -0700 Lisa Dusseault
<[e-mail]> wrote:

>... 
> (BTW, I'm sure I can follow the advice of "Use your judgement"
> if anybody decides to say that, but it doesn't really inform
> that judgement does it?  )

"Use your judgment" is, in fact, probably the right answer.  I
hope this discussion provides you the information and
calibration you need.

> Is an IS that defines a new protocol for the Standards Track
> fine in general?
> Is an IS that extends a standard protocol developed in a WG
> fine in general?
> Is an IS that obsoletes a standard protocol developed in a WG
> fine in general?

In each case, I don't think there is an "in general".  I think
you need to monitor the situations and the amount and quality of
review.   I think it is also important that we not automatically
assume that having a WG implies extensive and competent review
from diverse parties.  Often that is that case, and perhaps it
is a reasonable presumption in the absence of evidence to the
contrary.  But there have certainly been WGs --and particular
documents coming out of otherwise well-behaved WGs-- in which
there has been evidence that no careful, competent, and
disinterested review occurs.

> Do IS's suffer from less review?  Is that a problem?
> Whose responsibility is it to get sufficient review?

The responsibility ought to lie with the authors and proposers.
I believe that it is their responsibility to convince you, the
IESG, and whatever portion of the community is interested and
relevant that there is sufficient interest and review to justify
standardization.  Regardless of how it is developed, one of the
requirements for an IETF standards-track document must be that
people --other than the developers and, where relevant, the
firms for which they work-- actually care whether there is a
standard in the area and what it contains.  One key difference
between WG-produced and individual submission documents is that,
because WGs go through a formal qualification and chartering
process, there is a presumption that their work is relevant,
appropriate for standardization, and has been reviewed.  None of
those presumptions automatically apply to an IS document; the
authors of such documents need to convince the community that
those criteria are met.   On the other hand, none of them are
automatically true of a WG product, but it is up to those who
are convinced that one or more criteria have not been met to
demonstrate that.

> Is it mostly well-connected individuals that can use the IS
> track, knowing an AD to do the sponsoring?  Or mostly
> individuals with a lot of time on their hands?

I hope not.  John's and Martin's comments are relevant here, but
I would also point out that people who are well-connected also
often have a much easier time getting WGs approved and through
the system.  Unfortunately, there is little or no substitute for
experience in the IETF (not the same thing as being
well-connected, but highly correlated often the more important
of the two) as an aid to getting things through the system.

>...
> Should I limit time spent sponsoring IS's? [1]  IESG work plus
> IS work could consume 30 hours a week, or 40, or 50...
> 
> Assuming I limit the potentially endless amount of work
> devoted to IS's, do I limit it algorithmically (e.g. first
> come first served),  as a matter of pure taste, or other?
> How should I prioritize IS sponsoring work? Which documents
> get my attention first?  [2]

I think you have to look at subject matter and depth of support.
My observation above about a variety of mechanisms being
subsumed under "individual submissions" is important here.  In
most cases, if some individual shows up and says "I have this
pet idea and I think it should be an IETF standard", you should
send him or her away until and unless they can demonstrate
support in the community for both the idea and for standardizing
it.   Put differently, use your own judgment, but remember that
no idea has a right to be standardized, whether as an individual
submission or via a WG.  For this, and for several of your other
questions, I think the burden is on the proposer to convince you
that the idea is appropriate for standardization, that there is
sufficient interest, that adequate review has occurred, and that
an appropriate degree of consensus has been achieved.  

> How would appeals against IS documents affect answers to these
> issues? (Usually individual ADs take the first stab at
> handling appeals, formal or informal, against WG chair
> decisions.  When there's no WG chair involved with a doc, I
> guess the appeal would go straight to the whole IESG...)

No, the individual AD still takes the first stab.  Agreeing to
sponsor an IS doesn't automatically make you an advocate of any
issue involving it.  And the second step, according to the
procedures, is a discussion with the IETF Chair.

> Does sponsoring many IS documents give an AD, and the IESG as
> a whole, too much power?

Not in and of itself.  Consistent bad judgment might, but this
is not the only topic around which bad judgment can result in a
concentration of power in the IESG.  And, if the community
doesn't have a way to deal with bad judgment by ADs, the problem
to be solved isn't one of too many IS documents.

> Are we discouraging legitimate WGs by encouraging IS's?

Occasionally, probably so.  We are also discouraging
illegitimate WGs, would-be WGs that waste a lot of time and
never get organized, WGs with such a high noise:signal ratio
that it is impossible to get work done or evaluate consensus,
and so on.  The bottom line is that either the individual
submission process or the WG one can be abused.

>...

    john