Re: [ietf-nomcom] Changing the candidate selection model

Obviously this was sent in error... my apologies to all.

Spencer

> Sorry that I disappeared just before you sent this reply.
>
> On balance I think that it was well-written and (even more exciting to me)
> addresses MY issues that we've discussed, even the ones I think may very 
> well not apply to you on corporate support.
>
> If it seems to you that a positive statement would be helpful, please let 
> me know, and I'll say so onlist. If it seems that I'd be more likely to 
> wake up one or more flakes, please let me know, and I'll be quiet!
>
> Thanks,
>
> Spencer
>
>
>> Sorry to have taken so long to get back to this.  Other work...
>>
>> Joel, at the risk of being heretical, the job of the Nomcom is
>> not exclusively to "pick the best candidate".  It doesn't have
>> enough information to do that, partially because (with one
>> qualification that I'll discuss below) incumbents are a known
>> quantity while non-incumbent nominees are not.  So the job, in
>> practice, is to evaluate the strength of an incumbent's
>> performance, the advantages and disadvantages of long tenure,
>> and then to make a judgment about whether an alternate nominee
>> would be worth the risk than any non-incumbent might constitute.
>> Only if there is no incumbent does the Nomcom, in practice,
>> actually get to make a "best candidate" decision with all of the
>> Nominees evaluated on an equal basis.
>>
>> If you tell me that isn't what happens, I'll have to believe you
>> somehow because I don't have the inside information that you do,
>> but what I know about organizational behavior and selection/
>> election processes causes me to doubt that any other formulation
>> is plausible, even though "pick the best candidate" sounds good
>> in theory.   Even if that were the only factor, if there is an
>> incumbent who is willing to serve another term, a Nomcom
>> realistically has to balance the qualities and record of that
>> incumbent against the calculated risks posed by other nominees.
>>
>> Another job of the Nomcom --and the rules that enable it-- is to
>> ensure that the maximum number of qualified people are willing
>> to offer themselves as nominees for a given role.  If there are
>> impediments that discourage people from being nominated who
>> might be willing and able to serve, we should seek to eliminate
>> or reduce those impediments.
>>
>> Consider, for example, that we keep pushing the date at which
>> nominees are expected to tentatively commit themselves further
>> and further back. The RFC5078 timeline implies about eight
>> months between the first announcement of open positions and call
>> for nominees and the date on which decisions are announced.  Now
>> let's assume that we've got some potential nominees who have to
>> go to their management in June and say
>>
>> "I'd like to sign up for a two-year position with the
>> IETF.  It requires that you commit, right now, to
>> dedicating two years of my time, starting next March, to
>> this role which I may or may not get.   That commitment
>> implies that I shouldn't be put on projects in the next
>> eight months which would be in trouble if I had to drop
>> them in March with, if we are lucky, a month's notice,
>> so I'm on reduced functionality for eight months and
>> then may largely disappear for two years.
>>
>> "Incidentally, while the rules would permit us to back
>> out in February if circumstances change,
>> draft-dawkins-nomcom-openlist will make that much more
>> obvious, and possibly an embarrassment to the company,
>> especially if I'm the only non-incumbent nominee.  And,
>> since, at least statistically, the Nomcom routinely
>> returns incumbents, especially those who have served
>> only one previous term, the odds of my actually being
>> selected are low."
>>
>> That scenario is _not_ a way to build large nominee pools.
>> Worse, our current expectation is essentially that the same
>> people will not only do it once but, after not being selected
>> that year, will come back to their managers three or four months
>> after the selections are announced and say "Remember what we
>> went through last time?  I think we should do it again.".
>> Perhaps large companies who are willing to donate a certain
>> fraction of their staffs to the IETF don't care.  Perhaps people
>> who work for such companies but who are in a position to
>> allocate very significant fractions of their time however they
>> like are unaffected.
>>
>> I think the IETF would be better off with much more diversity in
>> the nominee pools -- while I'm grateful for the dedication of
>> the companies involved, I don't believe that it is a good sign
>> that, of the membership on the IESG and IAB, three companies
>> support three people each and another one supports two.   I'm
>> not concerned about conspiracy theories here but about the
>> observation that, with nearly 25% of the Nomcom-appointed IESG
>> and IAB positions in the hands of four organizations, how much
>> trouble we would be in if one or more of them got into serious
>> trouble and withdrew support.
>>
>> I do not believe that this proposal will solve the problem of
>> concentration of slots in the hands of a few organizations (if
>> indeed once considers it a problem).   But I think anything we
>> can do that broadens the Nomcom's effective range of choices
>> would be a good idea.
>>
>>>From my point of view, your reaction to the proposal is based on
>> an assumption that the current model has no adverse
>> consequences.  I think it does and am suggesting that those
>> consequences are severe enough that it is time to make the
>> tradeoffs somewhat differently.
>>
>> Some specific comments below.
>>
>> --On Wednesday, May 27, 2009 10:39 -0400 "Joel M. Halpern"
>> <jmh@joelhalpern.com> wrote:
>>
>>> Having read the draft, it seems to me that this is a bad idea.
>>>
>>> Firstly, I am not sure what problem it solves.  There is
>>> apparently some concern about whether the nomcom is returning
>>> too many or too few incumbents.
>>
>> Actually, that is not a concern or problem we were trying to
>> solve in this draft.  Instead, we started with the knowledge
>> that, whether it was reasonable or not, about half were returned
>> each time (as you note).  We examined the "ask the boss"
>> scenario outlined above and some similar ones and added in the
>> concerns about "running against" someone (especially an
>> incumbent) from  the same or a related organization that have
>> been expressed in conjunction with discussions of
>> draft-dawkins-nomcom-openlist and concluded that, on balance, we
>> would have much higher odds of having a good nominee pool when
>> it was needed if we separated incumbent evaluation from
>> consideration of new Nominees.
>>
>> We also believe (although we could be wrong, especially given
>> the tendency for work to expand to fill the available time) that
>> the separation would reduce the total workload, and time
>> requirements, on the Nomcom.  If it did and the effect of that
>> were to broaden the pool of Nomcom volunteers, that would also,
>> in our opinion, be good for the IETF.
>>
>>>...
>>> At the same time, this proposal seems to actually make it
>>> harder for the nomcom to do a good job.  The goal is to
>>> appoint the best available person to the job.
>>> Choosing to re-appoint or not re-appoint an incumbent without
>>> comparison with considering who else may be available and
>>> qualified is not, it seems to me, able to meet that goal.
>>
>> Our disconnect here lies in your use of "best available".
>> Certainly, that is correct if what you intended to say is "best
>> available among those who volunteered to be nominees.  But the
>> current model is almost certainly reducing the number of
>> available nominees.  In some ways, that makes the Nomcom's job
>> easier.  But encouraging people to nominate themselves against
>> incumbents who, statistically, will be returned anyway does not
>> give you a good pool of "available" nominees when the Nomcom
>> needs them.
>>
>> If you don't consider it a breach of confidentiality (since you
>> would not need to discuss any particular candidate, I think it
>> does not), it would help in this discussion if you would reveal
>> the number of plausible nominees you had available for each slot
>> in the last year.
>>
>>> It seems to me that the general notion of how many incumbents
>>> ought to be returned in any given cycle depends heavily on how
>>> many good alternatives there are. So I would hate to require
>>> the committee to make that decision without having that
>>> information.
>>
>> It seems to me that, if a Nomcom starts making decisions about
>> how many incumbents to return, as a count, we are in deep
>> trouble.  Fred's rumored examples, in which "all of the
>> incumbents were deemed good to return but the nomcom felt
>> obliged to sacrifice one, and did so, with quite a bit of hurt
>> feelings around the community. [...] And one could just as
>> easily imagine a case in which the nomcom felt that none should
>> be returned." both strike me as serious mistakes.
>>
>> Indeed, while nothing in either the current system or the
>> proposed one could stop a Nomcom from going down some sort of
>> "quota of incumbents to be returned" path if it decided it
>> wanted to, part of the intent of the proposal is to focus
>> attention away from that approach.   There are a few implicit
>> assumptions in this model which I will try to clarify in the
>> next draft.  One of them is that incumbents should be evaluated
>> on performance --both theirs and that of the body to which they
>> are contributing-- and returned there are signs of
>> non-performance, bad behavior, or burnout.  Of course, that
>> doesn't obligate those incumbents to be available for additional
>> terms.  But the read-in costs and uncertainties of putting in a
>> new person are sufficiently high that, if an incumbent is
>> serving well and effectively in a body that is working well and
>> wants to continue doing so, we should "let" him or her.
>>
>> If we don't believe that, then this is a bad proposal.  But,
>> again, if one judges from the statistics, that is, with rare
>> exceptions, exactly how Nomcoms are behaving.  The difference is
>> that they are behaving that way after exerting more or less
>> considerable efforts to collect other nominees (possibly making
>> those people unavailable when they are really needed, as
>> discussed above) and after spending lots of their time to see if
>> they can do better.
>>
>>> More important than the abstract failure mode, there are
>>> multiple specific failure modes that would do a disservice to
>>> the community and the nomcom.  If the nomcom has many
>>> qualified incumbents, if doing a stand-alone evaluation it may
>>> choose to return rather more incumbents than it would if it
>>> knew that some of those seats had good nominees available.
>>
>> You can't have it both ways.  One goal is well-qualified people
>> with minimum risk that they cannot or will not do the job.  If
>> that is it, then the discussion about incumbents above is
>> relevant because "good nominees" are always going to be a risk,
>> if only because we've had ample examples of the Peter Principle
>> playing out in the IETF as well as in other organizational
>> settings.  A different goal is to insure turnover by preferring
>> "good nominees" to incumbents.  If that is your preference, the
>> way to accomplish it is to be explicit about it and impose
>> rotation rules (term limits, more or less) with an exception
>> procedure.
>>
>>> Similarly, if the nomcom wishes to choose among the
>>> incumbents, because it feels that the good ones would still
>>> represent returning too many, the nomcom really needs to know
>>> who else is available.  Conversely, it is quite possible that
>>> the nomcom would, in a stand-alone evaluation, decide that
>>> some incumbent was not quite as good as they would like, and
>>> choose not to appoint him. Only to discover that said
>>> incumbent is significantly better than any of the nominees
>>> they evaluate.
>>
>> Here is where we have a significant difference of opinion that,
>> I believe, would make a real difference.  Let's assume that we
>> have an incumbent who is not very good but sort of getting by
>> despite several liabilities, and potential other nominees who
>> are unlikely to be any better.  At that point, it seems to me
>> that you are suggesting that, under the current system, the
>> Nomcom should select, not a "good" or "best" candidate (because
>> there isn't one) but rather pick the least bad one.  The
>> proposed system would certainly make that harder.
>>
>> By contrast, I believe that, if an incumbent is not performing
>> well, returning him or her is almost always a mistake.  It
>> weakens the IETF and the relevant body to have someone who is
>> underperforming continue.  Perhaps worse, we've had a number of
>> cases in which someone has been returned to a body in spite of a
>> record of bad behavior, only to have that person behave as if
>> reappointment constitutes endorsement of the behavior and
>> increase it.  We've also found that comments from Nomcoms or
>> others are of no help in preventing further inappropriate
>> behavior.
>>
>>>From that perspective, if the Nomcom makes a decision that
>> someone is not performing adequately, I believe that person
>> should be retired or removed.   If the Nomcom cannot then find a
>> nominee in whom it has high confidence, I believe that any of
>> the three alternatives are better than going back to the
>> incumbent: the Nomcom searches harder and gets more creative,
>> the Nomcom decides to take the risk with a Nominee who has at
>> least not already proven to be inappropriate for the role, or
>> the Nomcom explicitly decides to not fill the position because
>> there are no plausible candidates.  In the latter case, I'd
>> expect the relevant body to be forced to either run short-handed
>> or restructure and that either of those options would be better
>> for the IETF that proceeding with weak leadership because some
>> earlier structure required it.
>>
>> The proposed model makes the "retire the underperforming
>> incumbent" scenario a little more efficient and likely to
>> succeed, but the comments above apply even under the current
>> model: if a Nomcom deliberately puts a "least unsatisfactory"
>> nominee (whether incumbent or not) into a job, it is doing the
>> community no favors.    And, as I have suggested above, the
>> proposed model is reasonably likely to give the Nomcom a broader
>> range of choices and thereby lower the odds of getting into that
>> situation.
>>
>>> The document seems to suggest that the answer to some of these
>>> issues is to assume that there are always enough good
>>> nominees, and that if there are not, maybe the seat should go
>>> unfilled.  This is sufficiently far from my understanding of
>>> the realities that it seems a dangerous assumption.
>>
>> Again, my observation of the realities is that selection of the
>> "best available" person, even if that person is known to be weak
>> as a candidate, is not merely a danger, but an almost-certain
>> problem.
>>
>>> There is another dimension in which I have serious concerns
>>> with this proposal.  One of the things the nomcom evaluates,
>>> for all bodies, is the balance of members.  That balance may,
>>> in different years, be reflective of different aspects of
>>> skills, knowledge, contacts, etc...   But That kind of
>>> balancing has been seen as important in may cases.
>>> Pre-selecting which incumbents will remain, and which will
>>> not, means that such balancing can not be used as part of the
>>> selection.
>>
>> Again, I don't think so.  Resignations and other rare and
>> exceptional cases notwithstanding, a Nomcom does not get the
>> option of replacing (or considering replacing) more than half of
>> a given body in a given year.   Given that, any attempted
>> "balancing" has to consider balance with at least half the
>> membership of the relevant body as a given -- the balance must
>> consider the incumbents who are not up for renewal, a group whom
>> the Nomcom cannot affect.   So the question is whether choices
>> of which incumbents to return can significantly aid or interfere
>> with balance given that established base.  I believe that, if
>> the body is sufficiently out of balance that special
>> considerations of incumbents are needed, that lack of balance
>> will usually show up as disfunction and make it clear which
>> incumbents should be removed, regardless of the order in which
>> things are done.  Conversely, if the perceived imbalance is
>> dictated by principles too subtle to show up in that way, I'm
>> willing to sacrifice giving the Nomcom an option that would seem
>> to put "balance" ahead of "best candidate".  YMMD.
>>
>>> Also, it is my opinion that this would significantly increase
>>> the time for the selection process.  The selection process is
>>> not linear.  As such, while the number of seats to be filled
>>> has some effect on the time, it is not a significant factor.
>>> (Issues of collecting input, and whether particular choices
>>> are easy or hard, make more difference.) While this work could
>>> possibly be overlapped with some portion of the nomination
>>> selection, it clearly would need to be completed, confirmed,
>>> and announced before nominations were considered usable, since
>>> nominations would clearly need these results if this process
>>> were to take place.  As such, I believe it would add several
>>> months to the process.
>>
>> Here, I am at a disadvantage because I don't have your in-Nomcom
>> experience.   But my reading of the situation is that a review
>> of, at most, a dozen people, without any need to compare them to
>> others, is not going to take very long as compared to an
>> in-depth review of several people per open slot.  And, by
>> eliminating slots that need to be considered, and for which
>> nominees need to be collected, one can further reduce workload.
>> I don't see how it could be otherwise unless there is a desire
>> to drag things out.
>>
>>> It is also the case that the differences in kinds of seats
>>> makes some difference.  Even if it were somehow concluded that
>>> this made sense for the IESG, I can not see why one would
>>> apply it to the IAB.  In the case of the IAB, one is not
>>> nominated against any one individual, but rather nominated for
>>> a seat in a pool.  As such, none of the arguments presented
>>> about the open nominee list seem to apply to the IAB.  And
>>> even more than the IESG (where some balancing issues do apply)
>>> the community consensus has seemed to me quite clear that
>>> balance of various kinds are a set of very important factors
>>> in selecting IAB members.
>>
>> When we started with this, we started with an "IESG only"
>> assumption.  While it seemed useful to explore whether it could
>> (and should) also be applied to the IAB and any other
>> Nomcom-selected positions, I'd be happy to go back to "IESG
>> only" if that proposal made sense.
>>
>>>...
>>
>> best,
>>  john
>>
>> _______________________________________________
>> ietf-nomcom mailing list
>> ietf-nomcom@ietf.org
>> https://www.ietf.org/mailman/listinfo/ietf-nomcom
>>
>
>
> _______________________________________________
> ietf-nomcom mailing list
> ietf-nomcom@ietf.org
> https://www.ietf.org/mailman/listinfo/ietf-nomcom
> 