Re: Meeting Venue Preference Survey

[Dave CROCKER <[e-mail]>]

Folks,

We really need to get these surveys produced by someone with training in survey 
design.  The intent of the survey is quite reasonable, but that the construction 
of it is not.  Survey's are quite sensitive to wording and context. This suffers 
from serious problems with both.


 From the survey:

> 2. Meeting Preferences
>
> 1. It is important to me to have the meeting venue be easy to get to:
> It is important to me to have the meeting venue be easy to get to:
 > Very unimportant
> Slides

What does "slides" mean?  I'm guessing it's an extraneous entry, since it throws 
off the apparent model of a balanced 5-choice set of responses.


> 2. Do you prefer a meeting in a gateway city,

I believe the underlying problem with this question, as demonstrated by the 
postings about it so far, is the lack of consistent criteria for defining 
"gateway" and "secondary".

I'll offer the view that a "gateway" city is a principle hub of international 
air travel, while a "secondary" city should have at least some international air 
access.  I think that's a useful distinction, but it means that more than one of 
the examples of secondary, in the survey, really would be classed as tertiary or 
worse, and there's a reasonable chance that Vancouver would count as primary.

At the least, please clarify the criteria for secondary.

I should note that it's probably still possible to get useful data from that 
survey question, simply based on respondents' subjective reactions to the terms 
gateway and secondary.  Over the years, including recently, there's been enough 
chatter about the basic distinction to make the specific lists of cities 
secondary.  Just knowing folks' preferences between gateway and 'other' might be 
helpful.  That said, "primary hub" might be a better choice than "gateway"; I 
would not be surprised to find some inconsistency in the meaning different 
people impart to the word.

(There's also some question about sampling for this survey.  The main ietf list 
is widely subscribed to, of course, but not as widely as this survey ought to 
target.  I suggest sending the notice also out to ietf-announce, at the least. 
Perhaps some other lists, such as for nanog, apnic, and ripe...)


> 3. Do you prefer going back to venues or trying new venues?

As phrased, this question probably biases responses toward 'new', since they 
sound more interesting, and possibly biases it strongly.

Presenting a statement of implications about the tradeoffs -- e.g., risks of 
new, reliability of returns -- would have set the stage for the response much 
better.


> 5. Would you be willing to pay a higher registration fee to have the meeting in a gateway city?

This is a fundamentally biased (distorted) question.  It is predicated on a 
factual assertion that is unsubstantiated and very probably false.

Gateway cities have many more travel choices and many more lodging choices. 
This very probably means that total travel costs can be /lower/ than for 
secondary cities.  At the least, this means that the relationship between cost 
and city 'class' is an open question.

Further, the registration fee is only one of a set of costs.  What is important 
is the total cost, not just the narrow, localized registration fee.

The set of responses provided also is rather oddly constrained.



> 8. Would you attend if we held the IETF in Africa?
> 9. Would you attend if we held the IETF in South or Central America?

This is yet another example of a question lacking foundation.  What is the basis 
for having a meeting in a region that produces few IETF active participants? 
Perhaps the reason is compelling.  But a question like this, lacking any 
premise, can only get a random sampling of spontaneous reactions. And given the 
way humans provide such reactions, the odds are high that repeating the survey 
in a month would produce different answers to this question.

d/















-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net