Re: [Rfcplusplus] Sunk cost + not about us

[Richard Barnes <[e-mail]>]

On Mon, Jul 9, 2018 at 8:30 PM Brian E Carpenter <
[e-mail]> wrote:

> On 10/07/2018 10:32, Joel M. Halpern wrote:
> ...
> > PS: I do not see how we can draw any conclusion from the very informal
> > survey.  We have been lectured, with good reason, in the past about the
> > dangers of drawing conclusions from even carefully formulated and
> > carefully distributed surveys.
>
> It's worse than that. The survey was hastily designed and included
> at least one ad hominem entry, and I think anyone with experience
> of surveys and their analysis would simply junk the results. See
> below for something more concrete on that.
>

I'll admit it was hastily conceived; the idea came to me as I was composing
my message to the list on Friday.  I won't admit it's ad-hominem -- it's
simply fact that the process entitles Adrian can approve RFCs as he likes.

But in the real world, when you're trying to make a decision, you either
work with dirty data or you collect better data.  I don't see anyone here
doing any better survey work, so throwing out what data we have seems
counterproductive.



> On 7/9/18 6:17 PM, Richard Barnes wrote:
> ...
> > Based on those observations, I hope it's clear to folks that there is a
> > problem to be solved here..  The survey data, sketchy as they are, also
> > point toward the solution, which is to refine the RFC label to have a
> > much more limited semantic, probably only IETF and possibly only
> standards.
>
> That's a pretty perfect example of confirmation bias.
>

If you have another interpretation for the responses to questions 1 and 2,
I would love to hear it.



> I hope that as scientists and engineers, we can do better than this.
>

I have no doubt that we could do better, but so far, nobody's done the work.


By the way, the full results are at
>
> https://docs.google.com/forms/d/e/1FAIpQLSeMoeR0TBWkZNpBKXJN3Am6nUL04Vr4-12T2VgEbiRdBwzngQ/viewanalytics
>
> The last question is interesting.:
> "Does an RFC published by the IETF require IETF consensus?"
> got 58.8% "no", which is the correct answer. However, this
> seems very unlikely as the preferred response from outsiders.
> Only insiders know that only some IETF stream RFCs require
> IETF consensus. So I don't think the data can reasonably be
> assumed to represent the opinion of outsiders.
>

I think this is more likely read as people just not understanding the
question, and the role consensus plays in our process. If you think the
respondents were insiders, how do you explain the abundantly "incorrect"
responses to question 2?

I actually asked people let me know when they filled out the survey, and
got around 20 notifications, including some IETF newcomers, several senior
developers for major projects, and a few newbie developers.  Taking that
together with the Q2 observation above, I doubt the sample was dominated by
insiders.

--Richard