Re: [Rfcplusplus] Sunk cost + not about us

[John C Klensin <[e-mail]>]

--On Tuesday, July 10, 2018 10:34 -0400 Richard Barnes
<[e-mail]> wrote:

>...
> But in the real world, when you're trying to make a decision,
> you either work with dirty data or you collect better data.  I
> don't see anyone here doing any better survey work, so
> throwing out what data we have seems counterproductive.
>...

Richard, 

As someone who used to teach survey design, survey data
analysis, and threats to validity of such analyses, I beg to
disagree.  

To summarize a much longer tutorial... If one has data that are
known to reflect sample biases relative to the population of
interest (and some of this discussion leads me to believe we
don't even have consensus about what that population is) or
whose collection of questions cannot be clearly and
unambiguously interpreted, show biases toward certain types of
answers or are otherwise prone to confirmation bias, etc., then
the results are likely to be of little or no value at all.  

That is very different from "dirty data".  "Dirty data", while
not a precise technical term, usually refers to data whose
biases (statistical or substantive) we understand well enough to
at least partially de-bias or adjust for its inadequacies.

Bad data in which the biases are impossible to evaluate other
than knowing that they are present and potentially serious (and
as distinct from "dirty data" as above) may not only be worse
than "no data" but may be quite a bit worse than someone
standing up and saying "this is true because I said so".  At
least in the latter case, one has some reasonably chance of
guessing at the expertise, prejudices, preconceptions, and
general reliability of the source and calibrating the assertion
accordingly.

     best,
       john