the methodology (was: So, where to repeat? (was: Re: management granularity))

[ext Andrew Sullivan <[e-mail]>]

Ok, I know I said that I'd shut up, but this is a clarifying question,
so I'll answer.

On Tue, Aug 07, 2012 at 05:28:33PM +0200, Sprecher, Nurit (NSN - IL/Hod HaSharon) wrote:
> Why the survey should limit it to the last five meetings...

That's what the oblique reference to Arrow-and-other-issues was about.
It turns out that if you give groups of people choices, the
preferences in a set are not necessarily transitive in the aggregate
even if they are in the individual cases.  (This is why preferential
ballot systems and so on have paradoxical effects.)  Similarly, if you
give people too many options, you overwhelm their ability to answer
and you get answers that are not consistent with the answers under
more narrowly constrained choices.

Survey methodology is one of my hobby horses.  The approach I
suggested is far from perfect, but given the lousy sample (our sample
is always self-selected) it seemed likely to me to produce better data
than other approaches.  I can justify this at greater length off list
if you really want.

> In the long history we experienced additional good places....

I picked recent ones where I heard people say good things and I didn't
hear very many complaints; also, I included only things people have
quite recently praised (that's why Minneapolis is on there even though
the last visit was some years ago).  I should note that I haven't
really found anything too bad about _any_ meeting I've been to, so I
don't have a horse in this race.  Anaheim was my least favourite, but
I still had a useful meeting, which is my real measure.  (Once again,
I note, people complained rather little and the weather was good.  I'm
pretty convinced about the weather-to-complaints correlation.  Maybe
we need a COLDNWET WG.)

> So maybe the survey should be more open and let each list his 3-5 favorable places based on the experience from earlier meetings?

No, that will produce totally useless data.  There will be no way to
reconcile it.  And anyway, because of the transitivity problem, we
could end up picking everyone's last choice out of that list to go to.

Now I really will shut up,

A

-- 
Andrew Sullivan
[e-mail]