Re: [Rfcplusplus] Sunk cost + not about us

[Brian E Carpenter <[e-mail]>]

On 11/07/2018 02:34, Richard Barnes wrote:
> On Mon, Jul 9, 2018 at 8:30 PM Brian E Carpenter <
> [e-mail]> wrote:
> 
>> On 10/07/2018 10:32, Joel M. Halpern wrote:
>> ...
>>> PS: I do not see how we can draw any conclusion from the very informal
>>> survey.  We have been lectured, with good reason, in the past about the
>>> dangers of drawing conclusions from even carefully formulated and
>>> carefully distributed surveys.
>>
>> It's worse than that. The survey was hastily designed and included
>> at least one ad hominem entry, and I think anyone with experience
>> of surveys and their analysis would simply junk the results. See
>> below for something more concrete on that.
>>
> 
> I'll admit it was hastily conceived; the idea came to me as I was composing
> my message to the list on Friday.  I won't admit it's ad-hominem -- it's
> simply fact that the process entitles Adrian can approve RFCs as he likes.
> 
> But in the real world, when you're trying to make a decision, you either
> work with dirty data or you collect better data.  I don't see anyone here
> doing any better survey work, so throwing out what data we have seems
> counterproductive.
> 
> 
> 
>> On 7/9/18 6:17 PM, Richard Barnes wrote:
>> ...
>>> Based on those observations, I hope it's clear to folks that there is a
>>> problem to be solved here..  The survey data, sketchy as they are, also
>>> point toward the solution, which is to refine the RFC label to have a
>>> much more limited semantic, probably only IETF and possibly only
>> standards.
>>
>> That's a pretty perfect example of confirmation bias.
>>
> 
> If you have another interpretation for the responses to questions 1 and 2,
> I would love to hear it.

That people are confused, or simply don't read the boilerplate, isn't news.

What has not been demonstrated is whether this matters enough to be worth fixing
in a disruptive way.

     Brian

> 
> 
> 
>> I hope that as scientists and engineers, we can do better than this.
>>
> 
> I have no doubt that we could do better, but so far, nobody's done the work.
> 
> 
> By the way, the full results are at
>>
>> https://docs.google.com/forms/d/e/1FAIpQLSeMoeR0TBWkZNpBKXJN3Am6nUL04Vr4-12T2VgEbiRdBwzngQ/viewanalytics
>>
>> The last question is interesting.:
>> "Does an RFC published by the IETF require IETF consensus?"
>> got 58.8% "no", which is the correct answer. However, this
>> seems very unlikely as the preferred response from outsiders.
>> Only insiders know that only some IETF stream RFCs require
>> IETF consensus. So I don't think the data can reasonably be
>> assumed to represent the opinion of outsiders.
>>
> 
> I think this is more likely read as people just not understanding the
> question, and the role consensus plays in our process. If you think the
> respondents were insiders, how do you explain the abundantly "incorrect"
> responses to question 2?
> 
> I actually asked people let me know when they filled out the survey, and
> got around 20 notifications, including some IETF newcomers, several senior
> developers for major projects, and a few newbie developers.  Taking that
> together with the Q2 observation above, I doubt the sample was dominated by
> insiders.
> 
> --Richard
>