Re: [Pesci-discuss] My Notes on draft-davies-pesci-initial-considerations-00.txt

On 10/19/2005 15:11 PM, Spencer Dawkins allegedly wrote:
> Dear Pesci Design Team,

Dear Spencer ... half the pesci group is in bed already.  I'll give my
personal opinions.  If I don't say something, it's because the issue
is substantive or belongs to someone else, and I'm waiting for them.

> 1.  Introduction
> 
> SD: The following text makes "continual debate" sound a lot more
> positive than it really is. I'd suggest something like "The IETF almost
> continually debates its own process. While this is a positive indication
> of "openness", "closure" also matters, and IETF process debates are
> often inconclusive and often revisited."
> 
>   The IETF almost continually debates its own process and this is in
>   many ways a healthy sign of its openness.  However, the debate is
>   often inconclusive.

The "often revisited" bit feels useful, but I think "closure matters"
is already implied.

> 1.1.  Principles, Policy, Process and Procedure
> 
> SD: I have heard the phrase, "principles don't change, but procedures
> can change frequently" - I'm not sure if that's a helpful observation
> here or not, but if it is, it might give a smaller number of principles
> (do we really have 32? I thought us us as an unprincipled lot, most days).
> 
>   Before going on to discuss process change we should be clear about
>   what we mean by "process", and some of the other "P" words which will
>   be used in the document.

That's one of the distinctions we were trying to imply.  I agree that
we have too many principles listed.  We were in a bit of a hurry and
only did a couple passes sorting them out.  On the other hand imho
most IETFers are very principled :-).

> 2.  Background reading
> SD: Sadly, this probably should be "An earlier phase" (there were more
> than one).
> 
>   The early phase of the current round of process discussions led to a
>   problem statement [RFC3774].

ok

> 
> 3.  Short problem analysis
> 
> SD: I would suggest "Insufficent cross-area review early in the process
> (an unsolved issue)"
> 
>   3.  Variable quality of output from WGs
>       *  Unsolved issue of adequate cross-area review earlier in the
>          process.

ok

> 
>   5.  Lack of a "career path" for IESG members - "dropped in,
>       overworked and chopped off"
> 
> SD: s/loose/lose/
> 
>       *  potential to loose both experience and capability of ex-ADs
> 
> SD: Expand "techspec" on first usage...
> 
>   Issues that are to be considered under the "techspec" banner are out
>   of scope for PESCI.

ok

> 
> 4.1.2.  Change Management Guidelines
> 
> SD: I'm not AGAINST quantitative measures, but this comes up every time
> we discuss significant process change, just before EKR gets up and rails
> about how noisy the data is and how we change too many things at once to
> figure out what changed output can be assigned to each changed input,
> and we've got to be expecting two or three years of transition and
> retraining for significant process changes anyway. I'd drop this one for
> now, as much heartburn as it causes the engineers among us.
> 
>   G9.  Try to explicitly estimate the impact of changes before making
>        them, and try to measure whether the expectations were met after
>        making the change.

I didn't think quantitative measures were intended.  "Explicitly
estimate".  I often try to wean people away from "physics envy" in
administration and back to allowing human judgment.  Since you thought
they were, there's work to do, so we should put in "not necessarily
quantitative".

> 
> SD: It might be worth putting in an explicit pointer to RFC 3933 here,
> so that it's more clear that our existing BCPs would encourage this
> approach.
> 
>   G10. Acknowledge that some refinement of the initial proposals may be
>        needed after trials.  To this end try to work expeditiously to
>        provide a nearly right solution that delivers most of the gains
>        rather than refining the solutions endlessly before any
>        implementation (in line with the IETF's usual way of developing

dunno ... others?

> 
> 4.2.1.  Development and Authorisation of the Changed Processes
> 
> SD: I got lost on "the development process" - is this actually "the
> process of developing process"? I know what I think a development
> process is, and this isn't it.
> 
>   P3. The development process must incorporate taking advice from
>       *  the IESG, the IAB, the IAOC, and the Working Group chairs
>       *  legal advisors

Yes, it's supposed to be more of P2 ("the process for developing and
agreeing these changed processes"), focusing on the "developing" bit.
 I think.

> 
> 4.2.2.  The Content of the Changed Processes
> 
> SD: I would append "according to common sense, and relying on the
> appeals process to handle the breakdown of common sense in unusual cases".
> 
>   P7.  New process rules should be flexible enough to allow unusual
>        cases to be handled according to common sense.

no objection

> 
> 4.2.3.  Management and Leadership in the Changed Processes
> 
> SD: if we have to talk about miscarriages, could we at least call them
> "miscarriages of process"?
> 
>   The IETF prides itself on being a technically led organization (i.e.,
>   our leaders are primarily technologists, rather than managers) with
>   open processes which can, whenever possible, be challenged by
>   participants if they feel there has been a miscarriage.

ok :-)

> 
> SD: I know what you're trying to say, but I think it's more like
> "processes should ALLOW that people in leadership positions remain
> knowledgeable", and even this seems snaky - I believe I remember Bradner
> explaining to me that ADs chair working groups just to make sure they
> don't turn into complete process weenies (to be fair, this was a number
> of years ago, and I could have dreamed the incident - but either way, I
> think we're only asking that the processes don't require a lobotomy, or
> a cessation of learning).
> 
> P13. Management and leadership of the standards process should be
>        predominantly carried out by people who are technically
>        competent in the area in which they are leading, and the
>        processes should ensure that people in leadership positions
>        remain knowledgeable about the state of the art (and beyond) in
>        their areas of responsibility.

ok

> 
> SD: "valuable" is an overstatement - "We should try to minimize the
> amount of "on the job" training required for leaders to become
> productive as quickly as possible after they are appointed."
> 
>   P14. The processes should ensure that it is possible for IETF
>        participants to acquire or hone the particular skills needed for
>        management and leadership positions in the IETF before taking on
>        these positions, for example by forms of apprenticeship.  "On
>        the job" training is valuable but we should try to give it in
>        such a way that leaders are as productive as possible as soon as
>        they are appointed.

ok

> 
> SD: I would suggest "a period to contribute using their skill" - or are
> we really saying that long-term ADs can't maintain their technical
> skills? I thought not.
> 
>   P16. The processes should ensure that, so far as is possible, the
>        pool of skills and experience developed by those in leadership
>        and management positions remains available to the IETF and
>        continues to be seen to be valued whilst giving these leaders a
>        period to refresh their skill in the technical arena.

Well, that's my experience.  You have your choice, you can go for
breadth or depth, and your reach in both directions decreases after
40.  It's certainly true that we want to allow a period for them to
*increase* their skill.  Maybe they won't degrade (much), in which
case it's not just about refreshing it.  Others?

> 
> 4.2.6.  Areas
> 
> SD: The majority of the IETF membership likely has little clue about
> "Kobe", except as a type of steak - it would be nice to provide any
> pointer as background for this point of first use (not everyone spends
> hours on IETF archeology, fascinating as it often is). Also - "the
> central focus of ... technical expertise in the IETF" seems slightly
> overblown - I'd go for "the central focus of ... cross-area technical
> expertise in the IETF", but at least one adjective seems indicated.
> 
>   Areas have become a fundamental structuring mechanism for IETF work
>   since the Kobe reorganization.  Area Directors (ADs) are at the
>   moment the central focus of management and technical expertise in the
>   IETF.

(1) nnngg, let's not try to describe the Kobe thing in a nutshell.  Do
we have something we can reference?

(2) I do believe ADs are the central focus of technical expertise.
Maybe they don't have it in the specialties, but it all comes together
at the ADs at this time.

> 
> SD: "ambit" is a perfectly lovely word, but I'm a 51-year-old native
> (American) English speaker and had never seen it used before. Perhaps,
> "sphere of influence" (from www.webster.com)?
> 
>   P25. Besides experts in particular areas, it is important that the
>        IETF also takes an architectural view and encourages
>        participants who are able to work at the overall architectural
>        level both to direct the standards development work in the IESG
>        ambit; and to set technical guidelines internally, monitor
>        technical developments in the research and commercial world out
>        side the IETF and interact with external groups.

Elwyn?

> 
> 4.2.7.  Working Groups
> 
> SD: I could be misspeaking here, but thought that we were headed toward
> a world where WG chairs are ALSO responsible for shepherding
> deliverables through the approval and publication process - perhaps
> including this as a principle is premature, but I needed to ask. Which
> also makes me wonder about using one continuous index across multiple
> sections for principle numbers - you'd like to add a P29 here, but then
> you have to renumber the rest of the way down.
> 
>   P27. WG Chairs are responsible for ensuring that WGs execute their
>        charters, meet their milestones, and produce deliverables that
>        are ready for publication.
>   P28. WGs should be primarily responsible for the quality of their
>        output, and for obtaining and responding to cross-area and other
>        external review.

We were trying hard not to be *too* premature.

> 
> 4.2.8.  Extended Review
> 
> SD: A number of us were SIRS volunteers, but I think the statement that
> "The SIRS experiment ... [has] shown the value of extended, cross-area
> review of documents" is grossly generous. SIRS showed that reviewers
> would volunteer (for some number of "reviewers"), that the IETF has the
> attention span of a hamster when it comes to evaluating process change,
> and that WGs had no appetite for additional review steps that didn't
> bypass review steps later on. IMO. I would also note that I've gotten
> pushback on technical objections based on how late in the approval cycle
> Gen-ART reviews happened - "haven't these guys suffered enough?", and
> "jeez, they deployed this stuff four years ago anyway, and it sort of
> works". Which leads me to suggest a P33 - "Technical show-stoppers are
> always relevant, at any point in the approval process".
> 
>   The SIRS experiment and the General Area review team have shown the
>   value of extended, cross-area review of documents.  Currently most
>   such review is carried out too late in the document life cycle for
>   best effectiveness.

ok ... but what would you say?

> 
> 5.1.  Components Not Considered for Major Change in this Document
> 
> SD: "This process" - the PESCI process? or something else?
> 
>   ISOC      ISOC acts as IETF's legal parent and guarantor.  This
>             process is not at liberty to change the relationship
>             between ISOC and IETF.

PESCI

> 
> SD: I cannot get from the following IAB writeup to the conclusion that
> there is no reason to change the IAB. The second paragraph seems
> justification enough to include IAB changes as in-scope, without
> reference to anything else..
> 
>   IAB       The main function of the IAB is architectural oversight of
>             developments in the Internet.  This includes monitoring
>             standards development work to ensure that it does not
>             adversely impact the existing Internet, setting guidelines
>             for architectural aspects of development, helping to
>             determine what new working groups should be chartered,
>             keeping a watching brief on technical developments outside
>             the IETF, and providing statements on such developments as
>             appropriate.
> 
>             The IAB currently acts as the second line of appeal for
>             decisions of the IESG on standards development and other
>             matters.  This function is not an ideal fit with the
>             general architectural remit of the IAB: the competencies of
>             the IAB members do not necessarily equip them to deal with
>             the process appeals that come to them from the IESG.  Some
>             of the members of the IAB may consider them a distraction
>             from their architectural role.

There's plenty of reason to change the IAB, but not this round.

> 
> 5.2.  Components Considered for Major Change in the Document
> 
> SD: I would append "for final cross-area review, only for ensuring that
> final cross-area review has been performed".
> 
>   IESG      The IESG is central to the processes of standards
>             development in the IETF.  The steering role of the IESG in
>             acceptance of new work, formation, and chartering of WGs,
>             monitoring of WGs, and final stages of document processing
>             seems to be appropriate, as it is essential to coordinate
>             these functions.  This does not imply that the IESG must be
>             solely responsible for final cross-area review.

but they are not, at this time.

> 
> SD: It may be worth noting that IESG decisions can be appealed, but that
> this is not a fast or cheap process.
> 
>             Another area where process changes might lead to greater
>             confidence in the standards development process is the
>             degree of self-regulation which the IESG exercises: in many
>             process related disputes the IESG is expected to act as
>             judge and jury on actions in which it is itself involved.

ok ... I thought we did that somewhere.

> 
>  Working Groups
> 
> SD: It may be worth appending, "appears to be difficult to achieve,
> especially since so many working groups disband when their
> specifications are published as proposed standards:.
> 
>             The main areas for concern here are ensuring high quality
>             output and maintaining momentum over the period of
>             development, particularly in the later stages when the
>             creative work is essentially finished and the work is
>             mainly polishing.  Active management by supervisors
>             together with more rigorous and earlier review of documents
>             seems to be required.  Taking work through from initial
>             standardisation to full standard appears to be difficult to
>             achieve,

ok

> 
>   NOMCOM    The NOMCOM provides the means of selecting leaders and the
>             volunteer managers for the IETF.  The process by which this
>             is performed is time consuming and could be considered
>             onerous for participants, but it is vital that the process
>             remains as open as possible and endeavours to keep the IETF
>             as free of undue external influences as possible.
> 
> SD: If you're scoping out NOMCOM process change parameters, please
> include a note about the desire to get better community input on
> non-incumbent candidates - discussed on IETF Discussion List, I believe?
> 
>             Changes to these processes should ensure that overlaps of
>             role (as regards those performing the selection and the
>             leaders being selected) do not arise and that the workload
>             of the NOMCOM does not get out of hand.

I dunno, that's just one of many factors.  I would be careful about
calling it out especially since things that are tend to become
shibboleths.

> 
> 6.1.  Change Process Proposal
> 
> 
> SD: It bothers me that this document handwaves approval of process
> change ("the IESG (and ISOC Board of Trustees) will respect that
> consensus and approve it") - who actually DECLARES the consensus that
> the IESG and ISOC BoT then respect? Note that I haven't got a better
> answer, but the default now looks like "Brian declares consensus (or
> lack thereof) based on [fill in the blank here], and if no one
> successfully appeals to IAB/ISOC BoT, we know where we're going" - and
> if this isn't the default, what is?
> 
>   The teams should function with an open discussion list, in the same
>   way that the PESCI committee has done.  The result of the committee
>   should be tested against the IETF consensus in the normal fashion; we
>   believe that if there is clear IETF consensus that the proposal makes
>   sense, the IESG (and the ISOC Board of Trustees) will respect that
>   consensus and approve of it.

I'm stepping back from this one because I'm out of time.  I look
forward to others' opinions.

Thanks ... Scott

_______________________________________________
Pesci-discuss mailing list
Pesci-discuss@ietf.org
https://www1.ietf.org/mailman/listinfo/pesci-discuss