Re: NomCom 2020 Announcement of Selections

[Keith Moore <[e-mail]>]

On 1/28/21 4:46 AM, Eliot Lear wrote:

>> On 28 Jan 2021, at 10:18, Keith Moore <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> If you give people a set of demographics questions, the questions you 
>> pick bias the responses you get.  I suspect it's more useful to ask 
>> everyone what they think are the barriers to their contributing or 
>> seeking leadership roles.  The answers probably won't cluster to fit 
>> into tidy graphs and might not confirm people's preconceptions, but 
>> they'll be more informative.
>
> Let’s take some examples:
>
> One person says “there are no barriers.  I am comfortable 
> participating as is.”  Another person says, “The meetings are never 
> convenient for me to attend.”  This tells us nothing useful, even in 
> aggregate.  But if we ask an additional question, “Where do you 
> reside?”  And the people answering the former are primarily in the US 
> and Europe, and the people answering the latter are in the Southern 
> Hemisphere, then we know we have a problem with not serving the needs 
> of people in the Southern Hemisphere.
>
> If we ask the question, “Do you feel welcomed at the IETF?”  and some 
> people answer “yes” and some people answer “no”, again, this tells us 
> nothing.  But if the people who say “yes” are primarily white men from 
> Europe and North America and the people who say “no” are everyone 
> else, we clearly have a broad problem.  But perhaps it’s more nuanced. 
>  Maybe it’s a language problem.  Maybe the mix in demographics 
> *doesn’t* indicate clean groupings.
>
> All of this is possible, but all require establishing some 
> demographics.  By the way, establishing a set of groupings now doesn’t 
> mean we can’t learn about other groupings either in the process or later.

Of course it's necessary to carefully construct the questions to be 
asked if you want useful results.    So don't ask, for example, "Is it 
convenient for you to attend meetings?"  without also asking why.   Try 
to avoid inferring the reason why.   After all these questions are as 
much about perception as reality, because a perceived barrier to 
participation discourages participation.

The problem I consistently see with surveys of any kind is that they are 
constructed in such a way as to confirm the preconceptions of the party 
asking the questions, or to make that party look better than it might 
deserve.   If the questions asked are primarily or even significantly 
about "demographics" like geography, race, and gender, the survey will 
be sensitive to those variables.  But it will be insensitive to things 
which may be even more significant effects on diversity of IETF 
participation.

Keith