Re: RSE Bid Process

[Ted Hardie <[e-mail]>]

Hi Mike,

Some comments in-line.

On Mon, Jul 8, 2019 at 6:11 PM Michael StJohns <[e-mail]> wrote:

> On 7/8/2019 12:53 PM, Ted Hardie wrote:
>
> Hi Mike,
>
> Apologies for the delay in replying; I was off for the July 4th long
> weekend.  Some thoughts in-line.
>
>
> No worries - I was on a break too.
>
>
> On Thu, Jul 4, 2019 at 6:12 AM Michael StJohns <[e-mail]>
> wrote:
>
>> Top posting and in line:
>>
>> Hi Ted -
>>
>> Thanks for this outline.  However, I'm not sure that going forward with
>> the RFP before fixing the structural issues with the relationships between
>> the IAB, RSOC, RSE and LLC makes sense.  Patrick's note is on point.
>> Unfortunately, figuring out how to fix those relationships,and getting
>> agreement is probably not possible in the next 2-3 months.
>>
>
> If the community prefers adjusting RFC 6635 before re-running the bid
> process, that's probably possible, but we will likely lose the transition
> period between Heather and the new RSE as a result.  That's a pretty
> important trade-off, and one I'd want folks to discuss.
>
> That actually wasn't the point of the paragraphs.  I too don't believe its
> possible to fix 6635 in time for getting an RFP and transition out, but I
> do believe that it is possible to put 6635 aside for now as "broken", and
> work on a more standard business basis.  In other words, just an RFP,
> contracting entity (e.g. the LLC), and the successful bidders using mostly
> of the documentation and SOW work that we already had in place for Heather
> and making sure that 6635 is not included by reference in the ultimate
> contract.
>
>
> The 2106 SOW is here:

https://iaoc.ietf.org/documents/RSE-descr-2016-r6-Clean.htm

as you can see, it references RFC 6635 and it incorporates a good bit of
language from it as well.

One of the main ways to get the process expedited is to cleave as closely
as possible to the previous process, and to re-use as much as possible of
what was previously done.  If you put it aside, reusing this won't work.
That will add delay, rather than remove it, as we will have to get
community agreement, legal review, and LLC board approval of new language.
If RFC 6635 has not been superseded, there's also a risk that someone who
prefers that language and method will appeal any effort to route around
it.

So we appear to agree that we cannot supersede RFC 6635 and use those new
processes to have a transition that includes the current RSE, but we
disagree on the consequences of that.  Not speaking for anyone but myself,
I think that means either we use the current processes or accept a delay.


> Given the small size of the community of editor/publishers we are going to
>> try and draw from and our current missteps, my estimation is that we're
>> unlikely to get more than one bid if that, and that any contract terms
>> we're offered by a given bidder are going to be much tighter and much more
>> restrictive on things like IAB/RSOC "management" and "oversight".   There's
>> going to want to be more assurance by the bidders that independent
>> contractor means "independent".  I would also expect that bidders will want
>> more assurances against arbitrary termination of the contract.
>>
>> Considering all of the above, let me make an alternate proposal.  First
>> that we ask back the previous RSOC in place of the current group and ask
>> them to serve until we resolve the issues with the various relationships
>> (e.g. the next publication of 6635 as a community - not IAB - document).
>> We give them the responsibility for running the bid process and for the
>> ultimate selection of the final candidate with the advice and agreement of
>> the LLC.
>>
>
> As my note pointed out, 3 of the 4 current members of the RSOC were
> serving before the refresh, with tenure up to 5 years.  So "asking back"
> the previous RSOC would continue to include them.  That seems to limit, in
> my eyes, the impact of this change.
>
> This is a bit disingenuous.   In Sept of 2018 I think I'm correct in
> saying that the IAB told the entire RSOC that they had to reapply for their
> positions and also opened it others.
>
The call is here:
https://mailarchive.ietf.org/arch/msg/ietf-announce/-9XsyFQsp-4fybaKssEttVINCQE
It went out in May, rather than September.  September was the date that
process concluded, after community feedback and interviews were done.   The
September change also saw Christian Huitema replacing Martin Thomson (both
being IAB members).

The call prior to that was in 2014.  As I noted in my mail asking for
feedback on establishing set terms for RSOC members, RFC 6635 provides no
guidance on when or how to refresh the membership of the RSOC, so the IAB
does it according to the process for refreshing other programs.  I have so
far only heard positive responses on the change to three year staggered
terms for RSOC members , with the understanding that opening RFC 6635 may
result in changes that make that adjustment O.B.E. (e.g. if the next RFC
Model Document provides for direct appointment of the RSOC by the NomCom).
If you have thoughts on that proposal, please do share them.

>   The RSOC prior to that point consisted of 6 appointed members (including
> one IESG member and the chair), and two IAB members plus the RSE and an
> IAOC rep as non-voting members.  The RSOC after that point consists of 4
> appointed members, of which three were continued (include the IESG member
> and the chair), and one new member.  E.g. 1/2 of the old RSOC was let go or
> didn't reapply.  The three members not continuing represented about 24
> years of cumulative experience.  The three retained members have a total
> cumulative experience of 11 years.   Asking the three non-continuing
> members to rejoin would almost triple the cumulative experience.  It would
> also restore the I* member vs community member ratio back to where it was
> previously.
>

> Further, I'm not sure that the precedent this would set is in line with
> the preferences expressed in RFC 6635 or your note.  Until there is an
> adjustment of the process, the only way to change the RSOC membership is
> for the IAB to do it.  This would establish as precedent that the IAB could
> dismiss the sitting RSOC as a body if it did not like the outcome of their
> work.  There's a great deal of both text and established practice that
> makes the IAB hands-off to the work of the RSOC, and this would seem to run
> directly against that intent.
>
> I'm sure there are at least a few of us that would be willing to draft an
> emergency RFC to adjust the process for an interim period.  I don't see
> that as a deal breaker.
>
> I'm sure you could, but I am a good bit less sure of getting community
support for it.  It changes the relationship between the two bodies in a
very fundamental way and it goes directly contrary to RFC 6635:

   The RSOC will act with authority delegated from the IAB: in general,
   it will be the RSOC that will approve consensus policy and vision
   documents as developed by the RSE in collaboration with the
   community.  While it is expected that the IAB will exercise due
   diligence in its supervision of the RSOC, the RSOC should be allowed
   the latitude to do its job without undue interference from the IAB.
   Therefore, it is expected that the IAB will accord RSOC reports and
   recommendations the benefit of the doubt.

The one exception given is the following paragraphs, which notes how
decisions that impact the RSE as an individual are handled:

For all decisions that affect the RSE individually (e.g., hiring and
   firing), the RSOC prepares recommendations for the IAB, but the final
   decision is the responsibility of the IAB.  For instance the RSOC
   would do the following:

   o  perform annual reviews of the RSE and report the result of these
      reviews to the IAB.

   o  manage RSE candidate selection and advise the IAB on candidate
      appointment (in other words, select the RSE subject to IAB
      approval).

In other words, the IAB's part of the process occurs after the
recommendation occurs.  That part of the process did not occur in this case
because Heather's decision came before it could.  Please see the timeline
posted.

> With respect to the IAB taking a hands off approach - I'm not sure that's
> the correct view of the reality of the situation.  If it is, then we need
> to ask whether there's a failure of supervision in addition to other
> possible problems.  There's this old adage - you can delegate authority,
> but you can't get away with delegating all the responsibility.
>
>
> I don't think the IAB means to get away with delegating its
responsibility.  But it has tried to exercise its responsibilities here in
the way the RFCs prescribe.  In this case, that's a serial function where
that step didn't occur.  Future models may make it concurrent to avoid this
or may take other tacks, but this is the model we are operating under now.


>
> As an additional point, you use the term above "arbitrary termination".  I
> hope you understand that termination was not invoked here.  The contract
> does include provisions for termination with a much shorter notice, and it
> includes provisions for not renewing the contract.  The RSOC's
> recommendation to the IAB was for renewal of the contract for the available
> term.  I've discussed in other messages how that intent likely went astray
> because the message included other elements, but the upshot is that she was
> not terminated.  The portion of the process that completed was a
> recommendation that her contract be renewed.  She has decided not to take
> up a renewal.  As noted elsewhere in this message, we regret that, but we
> also need to be clear it was not a termination.
>
> Could you please post the email that the RSOC sent to the RSE?
>

I believe the words "recommend" or "recommendation" were not part of that
> message.  Also, given the "hands off", and the presence of IAB members on
> the RSOC it would be naive in extreme to expect there to be any pushback
> from the IAB to such a recommendation.  So "decision" or "decided", not
> "recommend" or "recommendation" is more correct I believe.
>

Sarah has already posted it, and I have posted the one sent to the IAB
list, of which she is a member.  The first message, as you note, did not
include the word "recommend", the second one did.  I have confirmed with
Heather that she received both at essentially the same time.  Since she has
been through this process multiple times before, I assume she understood
that IAB approval was required.


> With respect to "termination" - to terminate is to end, to cut something
> short, to bring to a close.   Informing the RSE that the first renewal
> option was being exercised, but that the second option would never be
> exercised is as much a termination as providing a 30 day notice.   It set a
> finite termination date fairly far in the future, but it set it none the
> less.    You might not like the use of the word, but it is accurate.
>
> In employment, terminate is a specific term of art.  See
https://www.thebalancecareers.com/termination-from-employment-2060505 for
an example definition in this domain.   Arbitrary termination is also a
term of art, and your assertion that it is the case here does not match the
facts as Heather has presented them.  Her choice not to renew the contract
was based on a belief that her goals and those of the IETF leadership had
diverged.  While I regret that conclusion and decision, it was clearly hers
to make. Further, all the parties to the contract had agreed to the forms
of renewal.  She was within her rights to terminate the contract without
taking up the extensions, and the RSOC was doing the job set out for it by
our processes in performing this review at this time.  And I will repeat,
as it appears not to have been clear, that they recommended a renewal of
the contract.  While they also recommended working on the bid process over
the following two years, they stressed in both their message to Heather and
to the IAB that this was not related to Heather's performance.

> In every contract that I've been involved in of the form, "base plus
> option years", the renewals have been mostly pro-forma - the expectation on
> both sides was that unless there was a problem and as long as performance
> was acceptable, the contract would be renewed.  Stability was a desired
> characteristic on both sides.   In the RSE case - pro-forma renewal was my
> assumption based on a need to continue to produce a world class document
> series with a lot of historical baggage for a community with diverse needs
> and sometimes troublesome wants.    This is why this action on the part of
> the RSOC without any reference to the community is both so surprising, and
> so indefensible.    My comment on "Arbitrary termination" in this case is
> with respect to making sure that both sides agree on the expectations for
> the renewals, and each keep their parts of the bargain irrespective of
> changes in the people involved
>
>
> I believe that the type of relationship you describe is better
accomplished by a contract of a longer term, and I suggest that you put
forward your reasons for that as comments to the SOW.


>
>
>> We use the existing SOW to the greatest extent possible.  We mostly
>> divorce the IAB from the RSOC/RSE until the relationship issues are
>> resolved - note that may require a contract modification with the RSE at
>> some point.
>>
>>
> The previous RSOC does have experience with this bidding process which
>> should mitigate some possible time sinks in getting up to speed.
>>
>> As Heather pointed out some time ago, the previous bidding process
> resulted in a single bid, by the incumbent.  Relying on re-running that
> same process may not be the best strategy.
>
> I'm doubtful that the problem was the bidding process.  I haven't seen
> even the first suggestion that there is a plan for ensuring more
> bidders.      The typical ways you get more bidders are "increase the
> offered value of the contract" or "decrease the scope or quality of work
> desired".  Not sure either of these make sense for us.
>
>
> As noted below, one possible approach is to use a search firm.  Other
suggestions for how to ensure a good pool of bidders are welcome from you,
or any other member of the community.


>
>
>> More inline - applies even if we don't take the above suggestion.
>>
>> On 7/3/2019 3:02 PM, Ted Hardie wrote:
>>
>> As folks are aware, there is ongoing discussion on Heather’s decision not
>> to continue as RFC Series Editor (RSE) and about how the situation has been
>> handled. Firstly, we’d like to express our regret at how things have
>> transpired and our willingness to have an open discussion with the
>> community about where we can all best go from here. This mail sets out our
>> view of how recent events unfolded, and our understanding of the process
>> for finding the next RSE.
>>
>>
>> The RSE currently has a two year contract which began in 2018 and which
>> permitted two extensions of up to two years each.  As part of their duties
>> under RFC 6635, the RFC Series Oversight Committee (RSOC) reviewed the
>> contract in May of this year.  On June 6th, they notified Heather Flanagan,
>> the current RSE, that they would recommend the contract be extended for two
>> years.  At the same time, they noted an intent to prepare a Request for
>> Proposals during that two year period, in order to address some concerns
>> that were raised about response rates after the last bid process.
>> Immediately afterwards, the RSOC notified the IAB both of their
>> recommendation to renew the contract in 2020 and their intent to prepare an
>> RFP for 2022.
>>
>> Initial discussions within the IAB would normally have been held at the
>> following IAB meeting, on June 10th.   On June 7th, however, the RSE
>> notified the IAB, the IETF LLC Board, and RSOC that she did not intend to
>> renew the contract.  The IAB is grateful that she provided early notice and
>> that she has agreed during the remaining months of the existing contract to
>> help the RSOC and IETF community to consider the requirements needed for
>> the next RFC Series Editor.
>>
>> Discussions on the requirements are ongoing on the IETF list.  While we
>> expect additional community discussions of future processes or models, we
>> also want to take advantage of the time Heather’s courtesy provided as best
>> we can.  As a result, we believe we should begin the RFP process as set out
>> in RFC 6635 now, with an aim to getting an RSE and a contract in place with
>> sufficient transition time.  Since the last running of this process, the
>> IAOC has concluded and the IETF LLC taken shape, so we wanted to lay out to
>> the community our understanding of that process with the IETF LLC in
>> place.  As with other aspects of the IAOC to LLC transition, the RSOC and
>> IAB will aim for minimal change to the current process, and zero unexpected
>> changes. That would give the following steps:
>>
>>
>> 0) The RSOC prepares a short paper on how a succession plan for the RFC
>> editor might work, and how recruiting in this space might work.   That
>> paper needs to cover what happens if the RFP fails to receive even one
>> bidder.
>>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> No comment on this?  Any ideas even?
>
It wasn't clear to me whether this was a succession plan under the current
process or between the two processes.  If it is between the current process
and an RFC Editor model 3 developed before hiring the next RSE, I can see
the need for this, though it might end up being a community driven effort.
If it is within this process, then this seems to add a step that slows
things down without much useful addition.

Your proposal appears to be that there be an RFC Editor (interim model)
that fits neither the current process nor is the result of a community
process.  A plan might well be needed there, but it depends a great deal on
how we come to community consensus on both the need for an interim model
and what it is.


>
>> 1) The RSOC prepares a statement of work based on RFC 6635 and previous
>> RFPs <https://iaoc.ietf.org/documents/RSE-RFP-4August2017.pdf>.
>>
>> 2) The RSOC sends the SOW out for public comment.
>>
>> If IAB members, the IAB, or the IETF LLC board have comments, they occur
>> during this period.  There is no separate review for them.
>>
>> 3) After the RSOC finalizes the SOW, it notifies the IAB, which requests
>> the IETF LLC issue an RFP.
>>
>> Instead, the RSOC provides the SOW to the LLC which turns it over to the
>> lawyers.
>>
>>
>> I believe this is functionally the same, since the IAB does not approve
> the SOW, simply forwards it.
>
>
>> Note: this is not an approval step, just the mechanics of who has the
>> token to turn this crank.
>>
>> 4) The RFP is issued by the IETF LLC.
>>
>> 5) The RSOC, or a committee formed from within the RSOC, reviews the
>> proposals and interviews candidates.  Depending on the number of candidates
>> this could involve steps for creating a short list before running
>> interviews.
>>
>> This is the one that bothers me the most.  Given the missteps its unclear
>> this is the right group of people to be picking the next RSE.  Instead,
>> pick a search committee (as described I think in the previous SOW),
>> consisting of folks that have at least a little knowledge in the
>> publication/editing space.  These don't actually need to be IETF
>> participants especially if the SOW is tightly written.
>>
>
> I believe you're right that this could be a search committee formed by the
> RSOC, rather than totally within it. As an example, the RSOC could decide
> that it needs to use a search firm to reach appropriate candidates.  Under
> the current process, however, step 6 remains the same:  the RSOC must make
> the recommendation.
>
>
> And I'm suggesting that it's possible this RSOC may be fatally tarnished
> and we would be less likely to receive quality bids if it's just "business
> as usual".  If we're shown as trying to remediate some of the issues
> identified by the RSE prior to issuing the RFP it may increase our chances
> of actually getting someone who "gets us".
>
This RSOC, like every RSOC before it, is a set of community volunteers
doing their work as best they can under the processes that have been set
out.  I have repeatedly suggested that the root of this was a structural
issue in the two different modes of interaction specified by RFC 6635; you
evidently do not agree.  I continue to believe that to be the case,
however, and I personally believe that the best thing we can do is be clear
in our desire to remedy that structural issue, whether before or during the
next RSE's term.  Even if we were to desire to have an RSOC whose
membership was permanent and thus never lost state, peoples' jobs and lives
necessitate change.  Both Joe Hildebrand and Robert Sparks, who served on
the RSOC both before and during their time on the IAB and whose terms ended
with the change in RSOC, are good examples of this.


>
>>
>> 6) The RSOC recommends an appointment to the IAB.
>>
>> 7) The IAB approves the appointment, subject to contracting.
>>
>> Here's another possible issue.  It's unclear that the community and the
>> IAB/RSOC are aligned with what they want as an RFC editor.    I'm not sure
>> that will matter in the end especially if we get only one or two bidders.
>>
> I believe the IAB and RSOC are both following these discussion closely and
> I am sure that we will continue to get more input from the community in the
> form of comments on the SOW.
>
>
>>
>>
>> 8) The IETF LLC negotiates a contract with the appointed RSE.
>>
>> 9) After timing is agreed, the new RSE is announced and a transition
>> begins.
>>
>> Once again, we regret how things have transpired and will be engaging
>> with the community on the longer term handling of how the IAB, RSE and IETF
>> relate to one another. That may include a new version of RFC 6635.  Until
>> those conclude, however, we believe we need to move forward with the
>> current process. As part of that, we want to confirm with the community our
>> understanding of the process required under the existing model and
>> procedures.
>>
>> If you have comments on this point, please send them by replying to this
>> mail or to [e-mail] and [e-mail].  Note that both lists include
>> the current RSE as a member.  If you have comments on the SOW, e.g., on
>> appropriate length or renewal intervals, please send them to RSOC after the
>> SOW has been published for community review. For comments on the broader
>> situation, the [e-mail] list is, as always, available.
>>
>> As an immediate structural change, we need either the IAB or the RSOC,
>> not both, to have their fingers in the RSE pie.   If that's the RSOC, then
>> it should probably be selected by the community, not by the IAB.
>>
> As I noted above, the RSOC generally operates independently of the IAB.
> The current documents call out specific times when the IAB must approve the
> actions of the RSOC, particularly this:
>
> For all decisions that affect the RSE individually (e.g., hiring and
> firing), the RSOC prepares recommendations for the IAB, but the final
> decision is the responsibility of the IAB.  For instance the RSOC
> would do the following:
>
>    o  perform annual reviews of the RSE and report the result of these
>       reviews to the IAB.
>
>    o  manage RSE candidate selection and advise the IAB on candidate
>       appointment (in other words, select the RSE subject to IAB
>       approval).
>
>
> The IAB treats these approvals as it does confirmations of the IESG slate
> for NomCom appointments.  We don't re-run the process, we confirm that the
> process ran as described.
>
>
> And this is possibly the saddest statement of all.  We have an RSOC that
> is beholden to the IAB, not the community.  The only way the community can
> enforce accountability on the RSOC is by firing the IAB.   After the
> various emails from the folk involved, I'm saddened that response from the
> IAB is that it's all on the RSOC.
>
I regret that you have come to this conclusion from the emails exchaged.
The IAB takes its stewardship of the series quite seriously.  The focus on
the RSOC's role has been because Heather's decision to resign occurred
between its recommendation and the IAB's actions.  That does not mean in
any way that we believe that we are not the ones accountable to the
community.

> In any event, during the times I was on the IAB, the IAB was not a rubber
> stamp for the Nomcom.  It was as much concerned with the correctness of the
> result as it was the correctness of the process.  With your above
> statement, I'm unclear what the value add of the IAB is for either the RSOC
> or the Nomcom.
>
The IAB also takes its role as confirmation body seriously, and I apologize
if my language inappropriately led you to believe it is a rubber stamp.
But confirmation is not the same as selection, and its role is to make sure
that the process has run correctly.  That involves a great deal of
information flowing between the NomCom and the IAB about the candidates, as
well as the NomCom chair providing full rationales.  This is why this, too,
is under the same sort of confidentiality we extend to our NomCom process.

regards,

Ted Hardie


> Mike
>
>
>
>
> regards,
>
> Ted
>
>> Later, Mike
>>
>>
>>
>> Regards,
>>
>> Ted Hardie
>>
>> for the IAB
>>
>>
>>
>>
>