Re: Extended: Consultation on IETF LLC Draft Strategic Plan 2020

[Jay Daley <[e-mail]>]

> On 13/05/2020, at 9:06 AM, Stephen Farrell <[e-mail]> wrote:
> 
> So part of my problem with this is that I think parts of
> what you propose as LLC strategy needs IETF consensus so
> that in turn means I don't think it ought be you, as the
> LLC exec dir, who decides the timeline and the rest of
> the process.

As you will see below, where there are things that we agree need consensus and that consensus does not exist then I have added an issue to either remove or amend them, to follow the intent of this not being a consensus document but a "this is how we are going to deliver consensus" document.

> 
> But, I promised a review of the actual text, that's below
> but overall I think the text is overlong and overreaches
> in ways that could lead to the LLC setting policies for the
> IETF. That isn't ok. Sorry to be so negative but I think
> this would be best refactored down to something much
> smaller and that very carefully doesn't accidentally put
> the LLC into a position of controlling what the IETF does.
> I make specific comments below, but would suggest those not
> be used for wordsmithing, but rather that this document be redone
> starting afresh.

Until such time as it becomes clear that there is strong support for that view I will treat your comments as individual issues.

> 
> Cheers,
> S.
> 
> - Many people will not comprehend that "Strategic Plan 2020"
>  refers to the LLC and not the IETF. Many people will have no
> clue that the LLC is not the IETF.  This exercise needs to be
> properly scoped down so that confusion is much less likely to
> arise. I suggest a long and boring, but accurate, title such as
> "2020-2025 plan for IETF LLC administrative actions."

Added as issue https://github.com/ietf-llc/strategy-2020-consultation/issues/21

> 
> - The introduction needs to make it clear what the LLC is and
>  what it is not.  As we've seen from on-list comments, people
> will confuse plans for the LLC and IETF. That text might be
> something like: "The IETF Administration LLC (IETF LLC) provides
> the corporate legal home for the IETF, the Internet Architecture
> Board (IAB), and the Internet Research Task Force (IRTF). The
> IETF LLC exists to serve the needs of the IETF community,

Added as https://github.com/ietf-llc/strategy-2020-consultation/issues/22

> and
> does not 'lead' the IETF comminity."

Unnecessary.

> 
> - Mission. I suggest reversing the community/leadership, e.g.:
> 
>    OLD:
> 
>    The best possible support corporation to the IETF/IRTF/IAB
>    and their communities
> 
>    NEW:
> 
>    The best possible support corporation to fullfil the
>    administrative needs of the IETF and IRTF communities and
>    leadership (IESG/IRSG/IAB).
> 
> - I think a similar change to put community first instead of
>  "IETF/IRTF/IAB" should be done throughout.

Looking at this again from the context of our comments I think the "and their communities" is superfluous.  Added as https://github.com/ietf-llc/strategy-2020-consultation/issues/23

> 
> - Evidence - I think that'd be better as "Evidence led: In
>  following the IETF community consensus, the IETF LLC will use
> evidence, both qualitative and quantitative, as the primary
> driver of its decision making."

The Responsive value earlier says "The LLC will act consistently with the documented consensus of the IETF" and that doesn’t need to be restated everywhere else.

> 
> - "The LLC will focus solely on its defined role and within its
>  defined mandate." I think that needs references, to make it
> understandable.

Updated this issue https://github.com/ietf-llc/strategy-2020-consultation/issues/11 which is about adding references to include a reference to RFC 8711 section 4.3 "General IETF LLC Responsibilities".

> 
> - "As the IETF LLC is a support organisation to the
>  IETF/IRTF/IAB, the strategic goals should ideally reflect
> their strategic goals." As written, this makes no sense to me,
> given the IETF lacks strategic goals.

It does say "ideally" and it goes on to say "However, as those are neither well understood nor well defined, some assumptions are required.".  I’ve added an issue based on Pete’s suggestion to change that second sentence to something like "However, while those are neither well understood nor well defined, these are extrapolated from IETF/IRTF/IAB expressed administrative needs and desires".

> 
> - "predominant" - I don't believe there is IETF consensus to
>  domiate the world in the way that's implied here.

Already captured as https://github.com/ietf-llc/strategy-2020-consultation/issues/13

> 
> - "To secure long-term funding for the LLC/IETF/IRTF/IAB that is
>  more than sufficient to meet their plans." What plans? AFAIK
> we don't and won't have a community plan for what budget we'll
> need in 2025. I think that'd be better as some kind of
> extrapolation from e.g. the current budget that the community
> have had a chance to consider. (Not that we do;-) Or, the LLC
> board might establish whether or not the community would like to
> plan further ahead. If there were consensus on that, then it
> might be appropriate to state this goal.

As Pete said, there are plans for meetings, tools etc.  i understand that you doubt the wisdom of continuing the same way and if that becomes a consensus view then we will adjust our funding targets in response.

> 
> - "To ensure the value of the IETF is well understood by
>  participants and key stakeholders beyond." I don't believe
> this is a goal for the LCC. I think it would be overreach for
> the LLC to try represent the views of the IETF to
> "stakeholders". As an AD, I was always careful to never do that
> unless there was an RFC or BCP or similar to point at.
> 
> - The term "stakeholders" is IMO inappropriate.

This breaks down into a number of parts

- first is recognising the leadership role the EMO Directorate has, which is already captured in https://github.com/ietf-llc/strategy-2020-consultation/issues/5

- second is being more specific than "stakeholder", which is something like "participants, supporters and the broader community" and is already captured as https://github.com/ietf-llc/strategy-2020-consultation/issues/14

> 
> - "To rapidly mature the IETF LLC into an organization..." that
>  reads to me like a statement I'd see in the usual commercial
> "take over the world" kind of plan.  I hope we have as a goal
> that the LLC be lean, and not bloated. Where is that represented
> here?

The full goal is "To rapidly mature the IETF LLC into an organization that fully meets its values and has strong community support and approval", which is explicit in how success is measured and links directly to the community.

Can you point me to the appropriate IASA2 document which specifies that the LLC must be lean and not bloated and how that is defined?

> 
> - "To deliver a modern and universally well regarded toolchain
>  supporting all the steps of the standards development
> process." Almost all the terminology here is undesirable and
> this doesn't recognise the volunteer aspect that has and I hope
> will continue to be important.

The narrative to the Tools Transformations later on states:

"The area of IETF tools has grown organically over many years and due largely to the contribution of a small number of people.  This has delivered valuable functionality and we have a well supported, productive and featured-packed toolchain."

It is worth noting that over the last few years (not sure how long as this predates me) a number of key volunteers have become contractors.

> "deliver" implies "here's what
> I'm giving you"

The more common implication of "deliver" is "delivering what people ask us to deliver".  

> "modern" isn't meaningful, "univerally well
> regarded" is not a thing that happens in the IETF, and "all the
> steps" is wrong - a number of things that are needed (e.g.
> thinking, talking, coding) don't need such tooling.

Partly already captured as https://github.com/ietf-llc/strategy-2020-consultation/issues/6 to which I’ve added removing "all the steps"

> 
> - "The focus currently is very much on developing capability and
>  capacity" that reads to me as if there is a desire for a
> bigger LLC and not for a lean LLC.

See my response above.

> 
> - Transformation#1 seems to me to imply the LLC tells the IETF
>  what to do.  That isn't acceptable. I don't believe it's the
> LLC's job to try "fix" the IETF in terms of (our lack of)
> strategy. The stated result is not possible, as none of the
> IESG, IRSG  nor IAB can set strategic objectives for 5 years
> time and the community has not given those bodies any such
> strategic objectives.

As Pete says, this is pretty clearly about ensuring that the LLC strategy follows and aligns with the strategy of the IESG/IRSG/IAB as RFC 8711 section 4.6 says 

  "However, the nature of the Board's
   work involves treating the IESG, IRTF, and IAB as major internal
   customers of the administrative support services."

> 
> - "Policy set complete, fully bedded in and signed off by ISOC."
>  The term "Policy set" needs to be defined, so that it is clear
> that it only refers to policies to which the LLC ought adhere
> and that ought be approved by ISOC.  There are other policies
> that the community ought control e.g. the output from
> meeting-venue that don't fit here.

Added as https://github.com/ietf-llc/strategy-2020-consultation/issues/25

> 
> - "Clear, strongly supported and well articulated value
>  proposition for the IETF/IRTF/IAB that supports all our
> engagements." That reads to me like marketing gibberish (hey,
> I'm an academic:-) but if such a thing is needed, it is not up
> to the LLC to drive that process.

Without a value proposition we cannot expect to broaden our set of funders beyond those few large companies that are already heavily engaged in the IETF.  For example, if we want to engage with private foundations to get the kind of support that ISOC provides us then we will need to very clearly articulate the value proposition of the IETF.

The transformations need to be seen in terms of what the LLC needs to do its job and will attempt to carry out but not that it has to take the lead or try to define the content of the output.  This is one where ideally the leadership and content would come from the community but it is not clear to me if there are enough interested participants who do not see this as "marketing gibberish" to make that happen and so in the narrative I’ve erred on the side of the LLC needing to produce an interim while the community does its thing:

"As the form of the value proposition is unlikely to be something many participants are familiar with, achieving acceptance let alone positive agreement could be challenging and the LLC may need to use an interim value proposition for those areas, such as funding, which would be significantly delayed otherwise."

I’ve added https://github.com/ietf-llc/strategy-2020-consultation/issues/26 to make it clearer in that paragraph above that this will be ideally be led by the community.

> 
> - "participant journey" - what is that? If it means how people
>  engage with the IETF and then fall away from that then I don't
> accept that the LLC ought drive what we want there.

This is explained in the text:

"Closely linked to a value proposition, is a participant journey (aka customer journey) which documents the different stages of participation (e.g. newcomer, leadership), how people enter one of those stages and how they transition between them.  This is used as the basis for our communications and engagement (i.e. targeting specific stages). "

The sentence after that is the contentious one "A defined and measured participant journey is crucial to ensuring we have a long term pipeline of participants" and rather than explore that I will simply remove it  https://github.com/ietf-llc/strategy-2020-consultation/issues/27

> 
> - "Full set of high quality content that enables us to deliver a
>  compelling demonstration of the value of the IETF to any key
> stakeholder." I think that is a job for the IESG. They may ask
> for LLC help, but it is not a job to be lead by the LLC.

This is already planned to change to "participants, supporters and the broader community" and will indeed be done within the auspices of the IESG as it has been for some years I believe.  I’ve added that explicitly to https://github.com/ietf-llc/strategy-2020-consultation/issues/5

> 
> - "cutting edge" - where is the IETF consensus statement on
>  that?  I don't believe it exists. There have been many cases
> of so-called "cutting edge" technology proposals that were
> nonsensical crap.

Added as https://github.com/ietf-llc/strategy-2020-consultation/issues/28

> 
> - A "compelling case" seems like more marketing terminology.
> 
> - "As the form of the value proposition is unlikely to be
>  something many participants are familiar with, achieving
> acceptance let alone positive agreement could be challenging and
> the LLC may need to use an interim value proposition for those
> areas, such as funding, which would be significantly delayed
> otherwise." I have no idea what that means, but I think I
> dislike it:-) It sounds a bit like "we'll tell you what's good
> about you even if you don't agree with that."

See above.

> - "A defined and measured participant journey is"... not the
>  LLC's business.  If the community change the standards process
> to do away with WG chairs or ADs, that's not a thing in which
> the LLC ought have a say. Making such statements here either
> implies that the LLC will do this or that the IETF needs to get
> off its ass and do this. Even if that last is true, it is not
> the LLC's job to deliver that kick.

See above.

> - What does "(for LLC services)" mean? Whatever it means
>  precisely, I think it needs to be clear throughout and not a
> parenthetical aside near the end of the document.

Without the parenthetical it would not be clear of the scope.  For others the scope is clear.

> - "a major programme of surveys" - I hope not! But you might be
>  right.  Where is the evidence that this will be useful?

i would hope the basis for data collection is self-evident.

> 
> - " benchmarked against comparable organisations including other
>  SDOs" I don't believe that we want to be like all other SDOs.
> I'd be surprised if there were consensus for my position, or for
> the one stated in the document here.

Benchmarked does not mean following, it only means comparing like for like.  There have already been multiple requests for comparable data, such as the cost of document production compared to other SDOs.

> 
> - A goal "for a zero carbon IETF" is one I tihnk I'd agree with,
>  but it needs to be an IETF consensus and not an LLC policy.
> That also means defining it well. That ought be much more
> clearly brought up with the community and not introduced near
> the end of a document like this.

Yes you’re right, the scope needs to be limited to data collection for the community to then decide the next steps.  Added as https://github.com/ietf-llc/strategy-2020-consultation/issues/29

> 
> - "a modern architecture and modern user experience" I don't
>  think the LLC ought set the UX guidelines for IETF tools, nor
> decide that the architecture ought be "modern." Those are IETF
> community decisions to make, after which the LLC ought set about
> delivery.  (Whether or not it's a good plan to migrate from
> bespoke tooling to today's fashion needs to be an IETF community
> discussion, ever one of the many times we need to have that
> discussion.)
> 
> - "increasingly unrealistic and inefficient" where is the
>  evidence of that? It may be true, but I don't think I've seen
> it.
> 
> - "a world of cloud apps, APIs and web services" - meh, that's
>  more marketing IMO. Those web based APIs can be turned off and
> that's a major downside that needs to be recognised.

This not about shifting from bespoke tooling but about how those tools interact with other tools and how people interact with them, and you are assuming that means third party commercial apps rather than bespoke apps running in our self-managed containers and connected by APIs that we define.  

I could ask you, have the community stated they don’t want a modern architecture and modern user experience? and if not then why should we not aim for that?  Having said that, now that there is the new Tools Architecture and Strategy team in place this transformation can be rewritten as implementing their requirements.

https://github.com/ietf-llc/strategy-2020-consultation/issues/30

> 
> - "A new group to look at the big picture, set out a vision and
>  user requirements" I don't like big pictures. The LLC is not
> the body that should set any "vision" for IETF tools.

That’s what the TAS is and that can now be replaced with a reference to implementing their requirements.  Added to the issue above.

Thanks for taking the time for such a detailed review.

Jay

-- 
Jay Daley
IETF Executive Director
[e-mail]