Re: RSE Bid Process

[Alissa Cooper <[e-mail]>]

Hi Ted,

> On Jul 8, 2019, at 11:59 PM, Ted Hardie <[e-mail]> wrote:
> 
> Hi Mike,
> 
> Some comments in-line.
> 
> On Mon, Jul 8, 2019 at 6:11 PM Michael StJohns <[e-mail] <mailto:[e-mail]>> wrote:
> On 7/8/2019 12:53 PM, Ted Hardie wrote:
>> Hi Mike,
>> 
>> Apologies for the delay in replying; I was off for the July 4th long weekend.  Some thoughts in-line.
> 
> No worries - I was on a break too.
>> 
>> On Thu, Jul 4, 2019 at 6:12 AM Michael StJohns <[e-mail] <mailto:[e-mail]>> wrote:
>> Top posting and in line:
>> 
>> Hi Ted -
>> 
>> Thanks for this outline.  However, I'm not sure that going forward with the RFP before fixing the structural issues with the relationships between the IAB, RSOC, RSE and LLC makes sense.  Patrick's note is on point.   Unfortunately, figuring out how to fix those relationships,and getting agreement is probably not possible in the next 2-3 months.  
>> 
>> If the community prefers adjusting RFC 6635 before re-running the bid process, that's probably possible, but we will likely lose the transition period between Heather and the new RSE as a result.  That's a pretty important trade-off, and one I'd want folks to discuss.
> That actually wasn't the point of the paragraphs.  I too don't believe its possible to fix 6635 in time for getting an RFP and transition out, but I do believe that it is possible to put 6635 aside for now as "broken", and work on a more standard business basis.  In other words, just an RFP, contracting entity (e.g. the LLC), and the successful bidders using mostly of the documentation and SOW work that we already had in place for Heather and making sure that 6635 is not included by reference in the ultimate contract.
> 
> 
> The 2106 SOW is here:
> 
> https://iaoc.ietf.org/documents/RSE-descr-2016-r6-Clean.htm <https://iaoc.ietf.org/documents/RSE-descr-2016-r6-Clean.htm>
> 
> as you can see, it references RFC 6635 and it incorporates a good bit of language from it as well.  

The SOW does reference RFC 6635, but the way in which it does is not necessarily inconsistent with what Mike may be suggesting. That is, given that it is a statement of _work_, the first three references to RFC 6635 are focused on the responsibilities and work of the person entering into the contract. The fourth reference is informational ("Additional information on the RFC Series Editor position …”). And, while the version you link to above mentions the IAB, the version incorporated as Attachment I in the current RSE contract <https://www.ietf.org/documents/232/RSE-2018-Independent-Contractor-24Oct17-Public.pdf> does not.

Furthermore, the current RSE contract also says the following in the body of the contract itself (not the SOW):

"1. SERVICES:

a. Contractor shall serve as RFC Series Editor (“RSE”) and shall perform such services related to that function as are set forth in Attachment I, the RSE Services (the “Services”) and additional services as needed and requested by ISOC that are reasonably related to the function and responsibility of the RSE. Contractor’s designated point of contact within ISOC shall be the IETF Administrative Director, and/or such other person as designated by ISOC, as determined from time to time.

b. As these Services are incorporated in an RFC, RFC 6635, the specific Services to be performed hereunder are subject to change. The parties will work together cooperatively to effect any changes to the Services that are required by subsequent versions of such Internet Draft and/or any resulting RFC.”

While the LLC will need to create a new contract for a new contractor, I imagine that substantially similar contract language could be used to provide a means to amend the SOW after contracting as a result of community discussions. The upshot of all of this may be that: 

(1) Most of the relationships Mike references above could change after a new contract is signed even if the exact same SOW is used this year as was used in 2017, because the only firm statement the SOW makes about the RSE’s reporting relationships is "The RSE reports to the RFC Series Oversight Committee (RSOC)” and because the contract itself needs to be generated anew anyway. That is, as long as there is something called an RSOC for the RSE to report to, the SOW would not be incorrect even if the RSOC<->IAB or IAB<->LLC or RSE<->IAB or RSE<->LLC relationships change.

(2) The new contract could provide flexibility to amend the SOW to align with changes the community agrees on in the future even in the absence of a new RFC that replaces RFC 6635, just as the current contract does.

Offering all of this as observations only.
Alissa

> 
> One of the main ways to get the process expedited is to cleave as closely as possible to the previous process, and to re-use as much as possible of what was previously done.  If you put it aside, reusing this won't work.  That will add delay, rather than remove it, as we will have to get community agreement, legal review, and LLC board approval of new language.  If RFC 6635 has not been superseded, there's also a risk that someone who prefers that language and method will appeal any effort to route around it.  
> 
> So we appear to agree that we cannot supersede RFC 6635 and use those new processes to have a transition that includes the current RSE, but we disagree on the consequences of that.  Not speaking for anyone but myself, I think that means either we use the current processes or accept a delay.  
> 
> 
>> 
>> Given the small size of the community of editor/publishers we are going to try and draw from and our current missteps, my estimation is that we're unlikely to get more than one bid if that, and that any contract terms we're offered by a given bidder are going to be much tighter and much more restrictive on things like IAB/RSOC "management" and "oversight".   There's going to want to be more assurance by the bidders that independent contractor means "independent".  I would also expect that bidders will want more assurances against arbitrary termination of the contract.
>> 
>> Considering all of the above, let me make an alternate proposal.  First that we ask back the previous RSOC in place of the current group and ask them to serve until we resolve the issues with the various relationships (e.g. the next publication of 6635 as a community - not IAB - document).  We give them the responsibility for running the bid process and for the ultimate selection of the final candidate with the advice and agreement of the LLC. 
>> 
>> As my note pointed out, 3 of the 4 current members of the RSOC were serving before the refresh, with tenure up to 5 years.  So "asking back" the previous RSOC would continue to include them.  That seems to limit, in my eyes, the impact of this change.
> This is a bit disingenuous.   In Sept of 2018 I think I'm correct in saying that the IAB told the entire RSOC that they had to reapply for their positions and also opened it others.
> 
> The call is here: https://mailarchive.ietf.org/arch/msg/ietf-announce/-9XsyFQsp-4fybaKssEttVINCQE <https://mailarchive.ietf.org/arch/msg/ietf-announce/-9XsyFQsp-4fybaKssEttVINCQE>
> It went out in May, rather than September.  September was the date that process concluded, after community feedback and interviews were done.   The September change also saw Christian Huitema replacing Martin Thomson (both being IAB members).
>  
> The call prior to that was in 2014.  As I noted in my mail asking for feedback on establishing set terms for RSOC members, RFC 6635 provides no guidance on when or how to refresh the membership of the RSOC, so the IAB does it according to the process for refreshing other programs.  I have so far only heard positive responses on the change to three year staggered terms for RSOC members , with the understanding that opening RFC 6635 may result in changes that make that adjustment O.B.E. (e.g. if the next RFC Model Document provides for direct appointment of the RSOC by the NomCom).  If you have thoughts on that proposal, please do share them.
>   The RSOC prior to that point consisted of 6 appointed members (including one IESG member and the chair), and two IAB members plus the RSE and an IAOC rep as non-voting members.  The RSOC after that point consists of 4 appointed members, of which three were continued (include the IESG member and the chair), and one new member.  E.g. 1/2 of the old RSOC was let go or didn't reapply.  The three members not continuing represented about 24 years of cumulative experience.  The three retained members have a total cumulative experience of 11 years.   Asking the three non-continuing members to rejoin would almost triple the cumulative experience.  It would also restore the I* member vs community member ratio back to where it was previously.
> 
>> 
>> Further, I'm not sure that the precedent this would set is in line with the preferences expressed in RFC 6635 or your note.  Until there is an adjustment of the process, the only way to change the RSOC membership is for the IAB to do it.  This would establish as precedent that the IAB could dismiss the sitting RSOC as a body if it did not like the outcome of their work.  There's a great deal of both text and established practice that makes the IAB hands-off to the work of the RSOC, and this would seem to run directly against that intent.   
> 
>> I'm sure there are at least a few of us that would be willing to draft an emergency RFC to adjust the process for an interim period.  I don't see that as a deal breaker.   
> 
> I'm sure you could, but I am a good bit less sure of getting community support for it.  It changes the relationship between the two bodies in a very fundamental way and it goes directly contrary to RFC 6635:
> 
>    The RSOC will act with authority delegated from the IAB: in general,
>    it will be the RSOC that will approve consensus policy and vision
>    documents as developed by the RSE in collaboration with the
>    community.  While it is expected that the IAB will exercise due
>    diligence in its supervision of the RSOC, the RSOC should be allowed
>    the latitude to do its job without undue interference from the IAB.
>    Therefore, it is expected that the IAB will accord RSOC reports and
>    recommendations the benefit of the doubt.
> The one exception given is the following paragraphs, which notes how decisions that impact the RSE as an individual are handled:
> 
> For all decisions that affect the RSE individually (e.g., hiring and
>    firing), the RSOC prepares recommendations for the IAB, but the final
>    decision is the responsibility of the IAB.  For instance the RSOC
>    would do the following:
> 
>    o  perform annual reviews of the RSE and report the result of these
>       reviews to the IAB.
> 
>    o  manage RSE candidate selection and advise the IAB on candidate
>       appointment (in other words, select the RSE subject to IAB
>       approval).
> In other words, the IAB's part of the process occurs after the recommendation occurs.  That part of the process did not occur in this case because Heather's decision came before it could.  Please see the timeline posted.
> With respect to the IAB taking a hands off approach - I'm not sure that's the correct view of the reality of the situation.  If it is, then we need to ask whether there's a failure of supervision in addition to other possible problems.  There's this old adage - you can delegate authority, but you can't get away with delegating all the responsibility.
> 
> 
> I don't think the IAB means to get away with delegating its responsibility.  But it has tried to exercise its responsibilities here in the way the RFCs prescribe.  In this case, that's a serial function where that step didn't occur.  Future models may make it concurrent to avoid this or may take other tacks, but this is the model we are operating under now.
>  
> 
>> 
>> As an additional point, you use the term above "arbitrary termination".  I hope you understand that termination was not invoked here.  The contract does include provisions for termination with a much shorter notice, and it includes provisions for not renewing the contract.  The RSOC's recommendation to the IAB was for renewal of the contract for the available term.  I've discussed in other messages how that intent likely went astray because the message included other elements, but the upshot is that she was not terminated.  The portion of the process that completed was a recommendation that her contract be renewed.  She has decided not to take up a renewal.  As noted elsewhere in this message, we regret that, but we also need to be clear it was not a termination.
> Could you please post the email that the RSOC sent to the RSE? 
> 
> 
> I believe the words "recommend" or "recommendation" were not part of that message.  Also, given the "hands off", and the presence of IAB members on the RSOC it would be naive in extreme to expect there to be any pushback from the IAB to such a recommendation.  So "decision" or "decided", not "recommend" or "recommendation" is more correct I believe.
> 
> 
> Sarah has already posted it, and I have posted the one sent to the IAB list, of which she is a member.  The first message, as you note, did not include the word "recommend", the second one did.  I have confirmed with Heather that she received both at essentially the same time.  Since she has been through this process multiple times before, I assume she understood that IAB approval was required.  
>  
> 
> With respect to "termination" - to terminate is to end, to cut something short, to bring to a close.   Informing the RSE that the first renewal option was being exercised, but that the second option would never be exercised is as much a termination as providing a 30 day notice.   It set a finite termination date fairly far in the future, but it set it none the less.    You might not like the use of the word, but it is accurate.
> 
> In employment, terminate is a specific term of art.  See https://www.thebalancecareers.com/termination-from-employment-2060505 <https://www.thebalancecareers.com/termination-from-employment-2060505> for an example definition in this domain.   Arbitrary termination is also a term of art, and your assertion that it is the case here does not match the facts as Heather has presented them.  Her choice not to renew the contract was based on a belief that her goals and those of the IETF leadership had diverged.  While I regret that conclusion and decision, it was clearly hers to make. Further, all the parties to the contract had agreed to the forms of renewal.  She was within her rights to terminate the contract without taking up the extensions, and the RSOC was doing the job set out for it by our processes in performing this review at this time.  And I will repeat, as it appears not to have been clear, that they recommended a renewal of the contract.  While they also recommended working on the bid process over the following two years, they stressed in both their message to Heather and to the IAB that this was not related to Heather's performance.
> In every contract that I've been involved in of the form, "base plus option years", the renewals have been mostly pro-forma - the expectation on both sides was that unless there was a problem and as long as performance was acceptable, the contract would be renewed.  Stability was a desired characteristic on both sides.   In the RSE case - pro-forma renewal was my assumption based on a need to continue to produce a world class document series with a lot of historical baggage for a community with diverse needs and sometimes troublesome wants.    This is why this action on the part of the RSOC without any reference to the community is both so surprising, and so indefensible.    My comment on "Arbitrary termination" in this case is with respect to making sure that both sides agree on the expectations for the renewals, and each keep their parts of the bargain irrespective of changes in the people involved
> 
> 
> I believe that the type of relationship you describe is better accomplished by a contract of a longer term, and I suggest that you put forward your reasons for that as comments to the SOW.
>  
> 
>>  
>> We use the existing SOW to the greatest extent possible.  We mostly divorce the IAB from the RSOC/RSE until the relationship issues are resolved - note that may require a contract modification with the RSE at some point.
>>  
>> The previous RSOC does have experience with this bidding process which should mitigate some possible time sinks in getting up to speed.
>> 
>> As Heather pointed out some time ago, the previous bidding process resulted in a single bid, by the incumbent.  Relying on re-running that same process may not be the best strategy.
> I'm doubtful that the problem was the bidding process.  I haven't seen even the first suggestion that there is a plan for ensuring more bidders.      The typical ways you get more bidders are "increase the offered value of the contract" or "decrease the scope or quality of work desired".  Not sure either of these make sense for us.
> 
> 
> As noted below, one possible approach is to use a search firm.  Other suggestions for how to ensure a good pool of bidders are welcome from you, or any other member of the community..
>  
> 
>>  
>> More inline - applies even if we don't take the above suggestion.
>> 
>> On 7/3/2019 3:02 PM, Ted Hardie wrote:
>>> As folks are aware, there is ongoing discussion on Heather’s decision not to continue as RFC Series Editor (RSE) and about how the situation has been handled. Firstly, we’d like to express our regret at how things have transpired and our willingness to have an open discussion with the community about where we can all best go from here. This mail sets out our view of how recent events unfolded, and our understanding of the process for finding the next RSE.
>>> 
>>> The RSE currently has a two year contract which began in 2018 and which permitted two extensions of up to two years each.  As part of their duties under RFC 6635, the RFC Series Oversight Committee (RSOC) reviewed the contract in May of this year.  On June 6th, they notified Heather Flanagan, the current RSE,  that they would recommend the contract be extended for two years.  At the same time, they noted an intent to prepare a Request for Proposals during that two year period, in order to address some concerns that were raised about response rates after the last bid process.  Immediately afterwards, the RSOC notified the IAB both of their recommendation to renew the contract in 2020 and their intent to prepare an RFP for  2022.  
>>> 
>>> Initial discussions within the IAB would normally have been held at the following IAB meeting, on June 10th.   On June 7th, however, the RSE notified the IAB, the IETF LLC Board, and RSOC that she did not intend to renew the contract.  The IAB is grateful that she provided early notice and that she has agreed during the remaining months of the existing contract to help the RSOC and IETF community to consider the requirements needed for the next RFC Series Editor.  
>>> 
>>> Discussions on the requirements are ongoing on the IETF list.  While we expect additional community discussions of future processes or models, we also want to take advantage of the time Heather’s courtesy provided as best we can.  As a result, we believe we should begin the RFP process as set out in RFC 6635 now, with an aim to getting an RSE and a contract in place with sufficient transition time.  Since the last running of this process, the IAOC has concluded and the IETF LLC taken shape, so we wanted to lay out to the community our understanding of that process with the IETF LLC in place.  As with other aspects of the IAOC to LLC transition, the RSOC and IAB will aim for minimal change to the current process, and zero unexpected changes. That would give the following steps:
>> 
>> 0) The RSOC prepares a short paper on how a succession plan for the RFC editor might work, and how recruiting in this space might work.   That paper needs to cover what happens if the RFP fails to receive even one bidder. 
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> No comment on this?  Any ideas even?
> 
> It wasn't clear to me whether this was a succession plan under the current process or between the two processes.  If it is between the current process and an RFC Editor model 3 developed before hiring the next RSE, I can see the need for this, though it might end up being a community driven effort.  If it is within this process, then this seems to add a step that slows things down without much useful addition.
> 
> Your proposal appears to be that there be an RFC Editor (interim model) that fits neither the current process nor is the result of a community process.  A plan might well be needed there, but it depends a great deal on how we come to community consensus on both the need for an interim model and what it is.
>  
> 
>> 
>>> 
>>> 1) The RSOC prepares a statement of work based on RFC 6635 and previous RFPs <https://iaoc.ietf.org/documents/RSE-RFP-4August2017.pdf>.
>>> 
>>> 2) The RSOC sends the SOW out for public comment.
>>> 
>>> If IAB members, the IAB, or the IETF LLC board have comments, they occur during this period.  There is no separate review for them.
>>> 
>>> 3) After the RSOC finalizes the SOW, it notifies the IAB, which requests the IETF LLC issue an RFP.
>> Instead, the RSOC provides the SOW to the LLC which turns it over to the lawyers.
>> 
>> I believe this is functionally the same, since the IAB does not approve the SOW, simply forwards it.
>> 
>> 
>>> 
>>> Note: this is not an approval step, just the mechanics of who has the token to turn this crank.
>>> 
>>> 4) The RFP is issued by the IETF LLC.
>>> 
>>> 5) The RSOC, or a committee formed from within the RSOC, reviews the proposals and interviews candidates.  Depending on the number of candidates this could involve steps for creating a short list before running interviews. 
>> This is the one that bothers me the most.  Given the missteps its unclear this is the right group of people to be picking the next RSE.  Instead, pick a search committee (as described I think in the previous SOW), consisting of folks that have at least a little knowledge in the publication/editing space.  These don't actually need to be IETF participants especially if the SOW is tightly written.
>> 
>> I believe you're right that this could be a search committee formed by the RSOC, rather than totally within it. As an example, the RSOC could decide that it needs to use a search firm to reach appropriate candidates.  Under the current process, however, step 6 remains the same:  the RSOC must make the recommendation. 
> 
> And I'm suggesting that it's possible this RSOC may be fatally tarnished and we would be less likely to receive quality bids if it's just "business as usual".  If we're shown as trying to remediate some of the issues identified by the RSE prior to issuing the RFP it may increase our chances of actually getting someone who "gets us".
> 
> This RSOC, like every RSOC before it, is a set of community volunteers doing their work as best they can under the processes that have been set out.  I have repeatedly suggested that the root of this was a structural issue in the two different modes of interaction specified by RFC 6635; you evidently do not agree.  I continue to believe that to be the case, however, and I personally believe that the best thing we can do is be clear in our desire to remedy that structural issue, whether before or during the next RSE's term.  Even if we were to desire to have an RSOC whose membership was permanent and thus never lost state, peoples' jobs and lives necessitate change.  Both Joe Hildebrand and Robert Sparks, who served on the RSOC both before and during their time on the IAB and whose terms ended with the change in RSOC, are good examples of this.
>  
> 
>> 
>>> 
>>> 6) The RSOC recommends an appointment to the IAB.  
>>> 
>>> 7) The IAB approves the appointment, subject to contracting.
>> Here's another possible issue.  It's unclear that the community and the IAB/RSOC are aligned with what they want as an RFC editor.    I'm not sure that will matter in the end especially if we get only one or two bidders.
>> 
>> I believe the IAB and RSOC are both following these discussion closely and I am sure that we will continue to get more input from the community in the form of comments on the SOW. 
>>  
>> 
>>> 
>>> 8) The IETF LLC negotiates a contract with the appointed RSE.
>>> 
>>> 9) After timing is agreed, the new RSE is announced and a transition begins. 
>>> 
>>> Once again, we regret how things have transpired and will be engaging with the community on the longer term handling of how the IAB, RSE and IETF relate to one another. That may include a new version of RFC 6635.  Until those conclude, however,  we believe we need to move forward with the current process.  As part of that, we want to confirm with the community our understanding of the process required under the existing model and procedures.  
>>> If you have comments on this point, please send them by replying to this mail or to [e-mail] <mailto:[e-mail]> and [e-mail] <mailto:[e-mail]>.  Note that both lists include the current RSE as a member.  If you have comments on the SOW, e.g., on appropriate length or renewal intervals, please send them to RSOC after the SOW has been published for community review. For comments on the broader situation, the [e-mail] <mailto:[e-mail]> list is, as always, available.
>> As an immediate structural change, we need either the IAB or the RSOC, not both, to have their fingers in the RSE pie.   If that's the RSOC, then it should probably be selected by the community, not by the IAB.
>> 
>> As I noted above, the RSOC generally operates independently of the IAB.  The current documents call out specific times when the IAB must approve the actions of the RSOC, particularly this:
>> 
>> For all decisions that affect the RSE individually (e.g., hiring and
>> firing), the RSOC prepares recommendations for the IAB, but the final
>> decision is the responsibility of the IAB.  For instance the RSOC
>> would do the following:
>> 
>>    o  perform annual reviews of the RSE and report the result of these
>>       reviews to the IAB.
>> 
>>    o  manage RSE candidate selection and advise the IAB on candidate
>>       appointment (in other words, select the RSE subject to IAB
>>       approval).
>> 
>> The IAB treats these approvals as it does confirmations of the IESG slate for NomCom appointments.  We don't re-run the process, we confirm that the process ran as described.
> 
> And this is possibly the saddest statement of all.  We have an RSOC that is beholden to the IAB, not the community.  The only way the community can enforce accountability on the RSOC is by firing the IAB.   After the various emails from the folk involved, I'm saddened that response from the IAB is that it's all on the RSOC.
> 
> I regret that you have come to this conclusion from the emails exchaged.  The IAB takes its stewardship of the series quite seriously.  The focus on the RSOC's role has been because Heather's decision to resign occurred between its recommendation and the IAB's actions.  That does not mean in any way that we believe that we are not the ones accountable to the community.
> 
> In any event, during the times I was on the IAB, the IAB was not a rubber stamp for the Nomcom.  It was as much concerned with the correctness of the result as it was the correctness of the process.  With your above statement, I'm unclear what the value add of the IAB is for either the RSOC or the Nomcom.
> 
> The IAB also takes its role as confirmation body seriously, and I apologize if my language inappropriately led you to believe it is a rubber stamp.  But confirmation is not the same as selection, and its role is to make sure that the process has run correctly.  That involves a great deal of information flowing between the NomCom and the IAB about the candidates, as well as the NomCom chair providing full rationales.  This is why this, too, is under the same sort of confidentiality we extend to our NomCom process.
> 
> regards,
> 
> Ted Hardie
>  
> Mike
> 
> 
> 
>>  
>> regards,
>> 
>> Ted
>> Later, Mike
>> 
>>> 
>>> Regards,
>>> 
>>> Ted Hardie
>>> for the IAB
>>> 
>>> 
>> 
>