RE: [Gen-art] Re: Gen-art review ofdraft-hartman-mailinglist-experiment-01.txt

["Margaret Wasserman" <[e-mail]>]

There are certainly some major weaknesses in the current IETF mailing list
management procedures, and those weaknesses are very well-described in this
document.  I agree that we need to address those weaknesses.  However, I am
not sure that I agree with the fix proposed in this document.

This document defines an RFC3933 experiment in which we would temporarily
give the IESG the authority to create new mailing list management procedures
and enact them.  The only hard limitations on this authority are that
posting suspensions cannot run beyond the timeframe of the experiment (18
months), nor can the procedures prevent anyone from reading an IETF mailing
list.  The document does not even limit the type of action that the IESG can
take -- while it only talks about posting rights suspensions, this document
would allow the IESG to define an enact other types of mailing list control
at their discretion.  It explicitly does not require that we use the same
procedures on all IETF mailing lists, as it explicitly allows the IESG to
define different procedures for different lists.  

This document does not define any principles that the IESG should follow in
determining mailing list management procedures, nor does it require any type
of community review or consensus to enact them.  Although it encourages an
IETF Last Call for new procedures, it does not require an IETF Last Call or
IETF consensus to establish a new procedure.  It doesn't even require that
the IESG send a new procedures for community review before enacting it
and/or that they explicitly notify the mailing lists to which the new
procedure would apply.  It merely requires that the new procedure be
published on a public web site.

At first, I thought it was the purpose of this document to allow the IESG to
try out different mailing list management procedures on different IETF
mailing lists for a short period of time, with the goal of picking some
successful procedures that would later be discussed by the community and
potentially reflected in our BCPs.  In other words, I thought that this was
a temporary measure to address the weaknesses in our current procedures and
get some experience with alternatives.  I still thought that the goal would
be to settle on well-defined, community-consensus-based procedures by then
end of this 18 months.  At that time, I supported this document, because I
saw it as a better alternative than living with our broken procedures until
the community could fix them.  I thought that the IESG could end this
experiment if/when we had community consensus on a new set of procedures.  

However, at the last Gen Area meeting, it became clear that if this
experiment is successful, the goal would be to permanently delegate this
level of discretion to the IESG, rather than publishing a specific set of
mailing list management procedures in BCPs.  Our mailing list management
procedures would permanently be defined in a web page that would be
maintained and modified by the IESG, rather than in BCPs that are determined
through a community consensus process.  

It is the nature of RFC 3933 experiments that they are intended to be
adopted as permanent procedural changes if they are found to be successful.
However, this document does not define any criteria for success.  RFC 3933
says:  "At or before the end of the "sunset" timeout, the IESG would either
revise (or cause to be revised) the document into a BCP RFC or the procedure
would expire..."  So, I _think_ what would happen at the end of this
18-month period is that the IESG would decide whether delegating this
authority to themselves was a useful procedural change and, if so, they
would publish a revision of this document as a BCP, permanently delegating
this authority to the IESG.

So, there are two questions I think we need to answer:  

(1) Is the community comfortable with the idea of giving the IESG such wide
discretion to establish and modify our mailing list management procedures? 

(2) Is the proposed experiment well-enough documented that we will be able
to clearly understand whether or not it was successful?

Personally, I would not be comfortable delegating this level of authority to
the IESG.  The former IESG's decisions in this area did not align with my
own principles, and I am not certain that the new IESG would balance
expediency and openness in a way that would be acceptable to me.  I am also
uncomfortalbe with the idea that something as important as our mailing list
management procedures would be documented on a web site that could be
modified without community review and approval.  I understand that I may not
be representative of the larger community in this regard, but I would much
prefer that we fix the problems with our current BCPs through a community
consensus process, resulting in updated BCPs that define our mailing list
management procedures.

I also have a smaller, but more concrete issue with this document.  IMO,
this document doesn't define a well-formed experiment.  In particular, it
doesn't define any criteria or method that that the IESG should use to
determine if the experiment has been successful.  Without such criteria, how
would the IESG decide whether or not to delegate this authority to
themselves on a permanent basis?  It has been recent IESG policy not to
publish experimental RFCs without well-defined success criteria, and I do
not think that this document meets that requirement.

Margaret 



> -----Original Message-----
> From: Sam Hartman [mailto:[e-mail]] 
> Sent: Monday, May 08, 2006 2:00 PM
> To: Harald Alvestrand
> Cc: IETF Discussion
> Subject: Re: [Gen-art] Re: Gen-art review 
> ofdraft-hartman-mailinglist-experiment-01.txt
> 
> >>>>> "Harald" == Harald Alvestrand <[e-mail]> writes:
> 
>     >> 
>     >> The primary reason you want to encourage 
> meta-experiments is that a
>     >> lot of the hypotheses you want to test involve delegation.  For
>     >> example I want to test the hypothesis that the right 
> way to solve the
>     >> mailing list mess is to delegate it to the IESG.  I 
> could delegate it
>     >> to the IESG as part of an experiment along with a 
> initial procedure.
>     >> But if I do that I'm testing a different hypothesis: 
> is delegating
>     >> something to the IESG with the whole IETF designing the initial
>     >> conditions a way to solve the problem.  As you might 
> imagine that's a
>     >> different hypothesis.
>     Harald> I do not believe that is a correct statement.
>     Harald> An RFC 3933 experiment by its very nature 
> involves three different
>     Harald> sub-experiments:
>     Harald> - Whether you can get the IETF to agree to 
> running this experiment
>     Harald> - Whether the bodies that administer the 
> experiment can perform the
>     Harald> experiment so that we have something to measure
>     Harald> - Whether the experiment produces useful results
> 
> I agree with this characterization.
> 
> I don't see how this characterization is in conflict with mine.
> 
> 
>     Harald> I think what you are proposing will lead to 
> failure of the *first*
>     Harald> experiment, which means that you won't get to the 
> other two.
>     Harald> If you say "give the IESG the power to produce 
> procedures, we're not
>     Harald> going to give you even a hint as to what these 
> procedures might be",
>     Harald> this is what I would characterize as selling a 
> pig in a poke, and it's
>     Harald> perfectly reasonable for the IETF community to 
> ring up a "no sale" on
>     Harald> the consensus register.
> 
> The IETF certainly may do that.  You clearly believe the IETF 
> should do that.  I believe the IETF should not do that.
> Ultimately Brian will have to make a consensus call.
> 
> I think it is reasonable to delegate comuing up with 
> procedures to the IESG.  I think that if we're going to 
> delegate that then the procedures should not be discussed in 
> this draft.
>  If the community wants more oversight, here are some things 
> I think would be reasonable to ask for:
> 
> * Add criteria the IESG will use for evaluating procedures or for
>   creating procedures.
> * Requiring proponents of this experiment to submit candidate 
> procedures that demonstrate procedures are possible
> * Adding explicit review requirements for procedures that the 
> IESG proposes.
> * Establishing principles the IESG should use
> 
> If you believe that one of these options would help, I invite 
> you to propose text or direction.
> 
> You could also propose an experiment to run with specific procedures.
> I would not object to that experiment but I would also not be 
> interested in contributing to it.
> 
> 
>     Harald> The theory that there is a different hypothesis 
> being tested if the
>     Harald> IESG sets the initial procedures than if the 
> initial procedures are
>     Harald> part of the experiment also seems doubtful to me; 
> in both cases what I
>     Harald> think will happen is that the IESG will start off 
> with procedures
>     Harald> designed by Sam Hartman, and the big difference 
> is that the community
>     Harald> will have seen the initial procedures before 
> deciding to run the
>     Harald> experiment.
>     Harald> The whole IETF has never designed anything and 
> never will; all design
>     Harald> is done by specific people, and (hopefully) 
> reviewed by some
>     Harald> large-enough fraction of the IETF. That's how 
> EVERYTHING in the IETF
>     Harald> works.
>     >> Since I'm on the IESG I'm actually in a
>     >> reasonably good position to negotiate an initial 
> procedure that the
>     >> IESG will be happy with and that would be similar to a 
> procedure the
>     >> IESG would come up with on its own.  However we want 3933
>     >> experiments--even experiments delegating things to the 
> IESG--to be
>     >> documents that anyone can write.  So we should require 
> that authors of
>     >> 3933 experiments demonstrate stakeholder buy-in for 
> experiments but
>     >> not require that they take actions as if they were the 
> stakeholders.
>     >> 
>     Harald> I do not believe this theory either.
> 
> So, the main point of the paragraph here is that I don't 
> believe only IESG members should be able to propose 
> experiments that change things for the IESG.  I hope we can 
> agree on that.
> 
>     Harald> RFC 3933 talks about the IESG and the community. 
> The "stakeholder"
>     Harald> concept is of your introduction, and I believe it 
> serves no benefit.
>     Harald> An RFC 3933 proposal author HAS to do three things:
> 
>     Harald> - Write up what he's proposing in enough detail 
> that it can be evaluated
>     Harald> - Convince the IESG that the experiment has 
> enough merit to warrant a
>     Harald> Last Call
>     Harald> - Generate enough review and comment in the 
> community that the IESG
>     Harald> can confidently say that there's community 
> consensus to run the
>     Harald> experiment
> 
> I proposed the concept of stakeholder buy-in as an attempt to 
> solve the first criteria.  I think that the IESG should 
> almost always last call experiments when the proponets can 
> demonstrate it is possible to do the work and when those 
> effected are interested in doing the work.
> I think that using this criteria will help resolve 
> long-standing questions of where to focus process reform 
> energy and of what people need to do in order to move their 
> proposals forward.  I think that the IESG should focus on 
> process reform where the stakeholders are interested in the 
> work over process reform where the stakeholders are not interested.
> 
> I do agree that when the community really desires some reform 
> and the stakeholders are not interested, we need some 
> resolution that allows the reform to proceed possibly with 
> resignations.  I think the bar in that case needs to be much higher.
> 
> But my concept of stakeholders is more about establishing a 
> criteria for forward progress than about blocking things.
> 
>     Harald> Certainly, a community feedback that says "I'm 
> one of the people
>     Harald> required to do work under this experiment, and 
> I'm not going to do it"
>     Harald> is a strong statement that needs consideration 
> when assessing whether
>     Harald> there's IETF consensus to run the experiment. But 
> sometimes that's
>     Harald> just part of the "rough" in rough consensus, even 
> when deciding to run
>     Harald> the experiment will cause someone to resign his 
> role. And sometimes
>     Harald> it's not.
>     >> The second reason that you want to allow 
> meta-experiments is that we
>     >> want to encourage RFC 3933 as the first step in process change.
>     >> Process change often results in BCPs.  You want the 
> 3933 experiment to
>     >> be reasonably similar to a BCP so that when 
> appropriate you can easily
>     >> convert a successful experiment into a BCP.  You would probably
>     >> replace any evaluation criteria with results of the evaluation,
>     >> replace the sunset clause with something else.  
> However you want the
>     >> operative language to remain the same.  A significant 
> result of the
>     >> mailing list discussion is the concern that our BCPs 
> are too specific
>     >> and encode operational details.  If you require that 
> meta-experiments
>     >> are not allowed you strongly push us in the direction of
>     >> overly-specific BCPs.  I think that would be a very bad idea.
>     >> 
>     Harald> Here I agree, but disagree that this document 
> does an appropriate job of it.
>     Harald> I think the separation of "principle" and 
> "process" is a good thing -
>     Harald> the principle of this document being (I think) 
> that the IESG sets
>     Harald> mailing list management procedures, and the 
> process being the process
>     Harald> by which such procedures are set, and the third 
> level being the
>     Harald> procedures themselves.
> 
>     Harald> However, just stating the principle alone at an 
> abstract level is not
>     Harald> enough to either start or evaluate the experiment.
>  I disagree.  If you'd provided more detail on why you don't 
> think that this document is sufficient, I could respond to 
> it.  However I think it might be useful to see what the 
> consensus position is before we spend a lot of time on this issue.
> 
> 
> 
> --Sam
> 
> _______________________________________________
> Ietf mailing list
> [e-mail]
> https://www1.ietf.org/mailman/listinfo/ietf
> 


_______________________________________________
Ietf mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/ietf