Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)

[Keith Moore <[e-mail]>]

On 7/19/19 6:55 AM, Ted Lemon wrote:

> On Jul 19, 2019, at 6:17 AM, Keith Moore <[e-mail] 
> <mailto:[e-mail]>> wrote:
>> I also see no reason to have a presumption, for example, that a WG is 
>> "right" in a matter that is outside of its subject area.
>
> I agree.   However, we’ve been trying to improve our cross-area review 
> diversity since I was on the IESG, and probably since you were, and 
> it’s not clear to me that there’s a lot more win to be had here. 
>  Where we do succeed, I think it’s because we’re able to rope experts 
> in to the working group.

I've noticed the effort to improve cross-area review.  I can't tell yet 
how effective it is, but I'm pleased to see the effort being made.

> The IETF last call process doesn’t work well.

I agree.   Last Call is mostly useful for problems that are easy to fix, 
and uncontroversial.   That seems to be a tiny fraction of the problems 
that still exist by that time.

It's long seemed to me that the points at which IETF manages to get some 
broad review on an idea or proposal are at the BOFs that precede WG 
formation, and Last Call.   These also seem like the times at which 
broad review is least likely to be effective (unless, as is sometimes 
the case, the WG starts with a document that it believes is already a 
mature proposal).

The best time period for broad review seems to be after a fairly 
complete proposal exists, but before there's a lot of investment in it.  
The WG is more receptive (less exhausted) at that point, and changes 
made as a result of such feedback are less likely to be disruptive.

> That’s why I think this is a bit of a non-sequitur to the “living 
> documents” discussion.  Yes, this is a problem, and we should be 
> careful not to make it worse.  But I really don’t think it’s the case 
> that the people who have organized this meeting are unaware of the issue.

One reason I'm interested is that I'm intrigued by the potential for 
"living documents" to facilitate earlier broad review of WG proposals.   
I think use of LDs has the potential to get feedback to the WG sooner, 
and also track how the WG responds to such feedback.   I can imagine a 
day when the most external reviews come in response to "First Call" or 
"Second Call", rather than Last Call, and the IESG final review consists 
largely of looking at how the WG and authors responded to the earlier 
calls for feedback, and whether the document has changed in such a way 
as to present new issues not previously identified.    Ideally the 
responsible AD will have been tracking such feedback and responses all 
along.

Another reason I'm interested in LDs is that some people seem to want to 
use them as a way to bypass community-wide review.   There may be cases 
for which bypassing such review is Mostly Harmless, but offhand I don't 
see a way to draw a clear line between Mostly Harmless and Potentially 
Harmful.

So I see both a potential for positive change, and a danger of negative 
change.   I don't know how aware the people who have organized the 
meeting are of these potentials, though I expect that they're at least 
somewhat aware.   Mostly I've been responding to opinions expressed on 
the list.

Keith