Re: [mpowr] Experiment design

If a tirade is completely correct and justified, is it still a tirade?

Spencer, who does have a masters degree in a quantitative social
science and very little urge to exercise that side of my brain in the
IETF...

----- Original Message ----- 
From: "John C Klensin" <john@jck.com>
To: "Harald Tveit Alvestrand" <harald@alvestrand.no>; "Pekka Savola"
<pekkas@netcore.fi>
Cc: <mpowr@ietf.org>
Sent: Tuesday, January 13, 2004 7:34 AM
Subject: Re: [mpowr] Experiment design

> <tirade>
> p.s. To anyone inclined to raise the "it isn't really an
> experiment, since there is no design control group and no
> placebo" argument, please give it a rest.  I am _not_ singling
> anyone out here -- that comment is made by someone, usually
> someone different, every time someone uses the word "experiment"
> around the IETF.  Virtually any experiment (e.g., a change in
> the rules with a monitored and evaluated outcome) on a
> social/behavioral system requires working with the subjective
> evaluations of participants and/or observers as to what
> happened.  If you exclude anything that is not subject to a
> carefully managed cross-plot or split-plot design, or even
> everything on which no exogeneous variates can unexpectedly
> occur in the system, then there are no experiments except in
> agriculture, and fewer of those than is generally believed.  In
> more careful designs than I am suggesting above (because I don't
> think the design and implementation time is justified), there
> are sometimes-complex techniques to lay a basis for comparing
> pre- and post-test opinions but they still don't involve design
> control groups or placebo controls.  The latter, in their purest
> form, are applicable only if _all_ variation not measured by the
> experiment can be excluded and that is, in practice, nearly
> impossible (double-blind placebo-based tests are just intended
> to minimize the most obvious causes).
>
> If you need some really good examples from the "experimental"
> literature, read some of the papers evaluating treatments the
> claim to reduce perceived pain levels.  Then, for a change in
> pace, read Wittgenstein's discussions on how someone accurately
> communicates levels of pain to someone else.
>
> I try to discourage my statistician colleagues who are not
> trained in network design from doing their own networks, and
> believe that this community should return the favor.
> </tirade>

_______________________________________________
mpowr mailing list
mpowr@ietf.org
https://www1.ietf.org/mailman/listinfo/mpowr