Re: Old directions in social media.

[Keith Moore <[e-mail]>]

On 1/7/21 5:30 AM, Hannes Tschofenig wrote:
> Hi Keith,
>
>> I certainly agree that we can't just wait for random outsiders to review things, and I appreciate the hard work it takes to get wide review. But I've also seen high-quality and useful feedback from the extended IETF community, and I don't think we should be adopting practices that discourage that.
>  From your experience, what are these reviews typically based on? Are they based on drafts or something else? If I recall great feedback I received "recently" then the review from Bob Briscoe on the SUIT architecture comes to mind (which happened late in the process based on the draft) and feedback from implementers (which was, of course, also based indirectly on a draft version).

I think the feedback is as likely to be based on f2f conversations and 
email conversations and also WG presentations, in addition to drafts.   
More realistically, as work progresses, early feedback is more likely to 
be based on such conversations, and later feedback more likely to be 
based on drafts.  But conversations play a large role even fairly late 
in the process.

(I often find the WG presentation materials to be MUCH better at 
facilitating useful conversations than the drafts, even if I find time 
spent "presenting" those materials to be largely wasted. Being able to 
read those materials without sitting through the presentations is 
great.  Being able to have real time conversations about those materials 
is also great if people have read those materials in advance.)

I think it's worth pointing out that reviewing documents is itself 
difficult and tedious work, and often it's easier to build consensus 
around design decisions before picking apart and wordsmithing the 
documents.  But it's also the case that people usually won't take a 
proposal seriously until there's a reasonably complete draft of it.   So 
you write a draft for people to attack (which should be complete enough 
to look credible but still leave some decisions for people to haggle 
over), then you have conversations, then you rewrite the draft - often 
substantially and multiple times, until finally you converge enough to 
do wordsmithing.   And it's in this last stage that I see github as 
being a useful though still really awkward tool.  But the conversations 
continue throughout.

Looking at this now, I wonder if one of the problems with our typical 
process is that we insist on having even the earliest proposals look 
like an RFC, as opposed to say, a set of bullet points.

>> Some of the early feedback I would not call "review" but something closer "feed forward".    WGs and document authors need to understand the constraints of the space they are designing for, and input from outside the WG core can be very valuable for this and should be encouraged.
> This is also an interesting case because it is input (for feed forward, as you call it) that is substantially different from the feedback later in the life of a specification (or set of specifications). In my experience this type of input is given during BOFs, in informal discussions, etc. and is often extremely time consuming for the persons providing the input. What is your experience there?

In my experience the feedback occurs not so much in BOFs (though this 
does happen) as in informal discussions and WG discussions.

IMO most constructive feedback, regardless of when it's given or the 
form it takes, is extremely time consuming for the persons providing the 
input.    But the hardest part of the work isn't in writing the words; 
it's in understanding what's really important to different people with 
different experiences, different points of view, different ways of 
expressing themselves.

> I am wondering whether the discussion about feedback /  input isn't largely orthogonal from the question what tools (email, Github, Slack, etc.) are being used. Providing input early in the life of a specification has always been a challenge - even before Github was around.

I think the tools have an effect at every stage.    Maybe no tool will 
make the process of understanding easier, but poor tools can certainly 
make the process more difficult.

But I also think that the work habits we have, many of them well 
entrenched in IETF culture, may also be making the process much more 
difficult than it needs to be.

Keith