Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)

[Keith Moore <[e-mail]>]

On 7/18/19 7:19 PM, john heasley wrote:

> Tue, Jul 16, 2019 at 09:04:17PM -0400, Keith Moore:
>> I could see some utility in having some documents being able to be
>> updated in place.  But I would have serious concerns with document
>> content like RFC7525 (i.e. technical recommendations for implementation
>> and/or operation of protocols) approved without IETF consensus.  It is
>> essentially part of a protocol specification.  So a WG should not be
>> able to "publish" such a document, nor approve it based entirely on its
>> own consensus. That would not only bypass IETF consensus (and cross-area
>> review), it would effectively bypass appeals and other safeguards we
>> have in place.
> Since no one is excluded from participating in any WG, anyone can comment or
> object to any such document.

WG-only review has been demonstrated over and over again, for literally 
decades, to be woefully insufficient.   There's far too little wide 
review with the current process, and you're proposing to make it worse.

> Appeals also can be submitted by anyone, though
> the process for such a document would likely need to be different.

The appeals process is far too cumbersome and taxing to IESG et al, to 
use for the number of issues that are likely to result from lack of 
cross-area review.

>> I also think trying to define IETF consensus for a moving target could
>> be challenging.  Not impossible, perhaps, just challenging.
>>
>> But I'd be supportive of trying to streamline the IETF consensus process
>> for such documents, on the theory that review of such documents
>> (including analysis of likely effects) should be easier than review of
>> full protocol specifications.
> How?  Coca leaves?
I was thinking more in terms of shotgun duels at ten paces, but coca 
leaves would be an interesting idea.  :)

I hadn't thought in depth about how to streamline such decisions, I just 
said I'd be supportive of it.   For instance, if one can somehow analyze 
the delta between version 1 and version 2, and version 1 has already 
been analyzed, it may not be necessary to analyze version 2 in its 
entirety.

One benefit of living documents is that it could be easier for IESG and 
its directorates to look at just the changes and avoid re-reviewing the 
entire document each time.   And the writeup for such a document could 
be expected to do exactly that.   Offhand I don't see a good way to 
bypass steps in the current process, but perhaps delta revisions could 
be given priority (by default) since they should take up much less 
review time than new documents. But it's still possible to miss things, 
of course, and occasionally (say every N revisions) a full review should 
be done.

(Thinking about how time consuming it has been to clean up some old 
standards, I'm intrigued by the possibility of a process that would 
encourage making small deltas over major rewrites.)

> I do not find any protocol changes in 7525 and it does not have an "Updates"
> header.  That appears to have taken a year to be published.  Could that be
> reduced to 1 month or less, esp. for updates, as you noted?

I'd think 6-8 weeks is doable, depending on when the clock is started.  
Something like:

(presumably this is after WG last call.   if WG can argue that a 
per-feature last call is sufficient and no per-document last call is 
needed, IESG might accept that.)
0. WG sends request to IESG with pre-written analysis of changes and 
arguments as to why they are beneficial or benign.
1. responsible AD reviews WG's request and analysis (or has a 
directorate member do it) and makes a recommendation to IESG
2. if recommendation is positive, announce 2 week last call
3. last call comments are evaluated and (if found valid) triaged: (a) 
problems with changes requested -> document goes back to WG; (b) 
problems other than with the changes requested -> comments forwarded to 
WG for evaluation and possible incorporation to a future revision but 
the WG doesn't necessarily have to fix it immediately - WG gets to decide
4. Assuming no category (a) bugs and neither WG nor IESG thinks any 
category (b) bugs have to be fixed immediately, IESG approves and issues 
public notice.
5. Somebody (RFC Editor?) updates the current version pointer in 
response to IESG's announcement.

IESG generally does things on 2-week cycles so a lot is paced by the 
timing of IESG meetings.   But things can happen faster if, for example, 
directorates return reviews more quickly.

> Admittedly, I do not see the value in IAB/IESG review, in the sense that
> their construction has no particular value to me over other's review.  A
> larger pool of potential (and equally competent) reviewers has greater
> value.

IESG review can also consider process issues, and sometimes it's the 
only external review a document gets, and sometimes IESG are the only 
ones watching out for cross-area conflicts.   So I don't think it can be 
dispensed with from a process perspective, even though it's more useful 
for some kinds of WGs and documents than others.

> Again, this is also about operations; 8212 and 8327 could have been part
> of a larger LD about BGP operations.

I'm always going to be thinking in terms of how much sense a process 
change makes for IETF as a whole.   If there's a specific corner case 
for which an expedited review process makes sense, that can be approved 
by writing up a BCP describing it and getting IETF Consensus behind that 
process.   But I fear it's a slippery slope - if we make an exception 
for one kind of group, other groups will want their own exceptions.   So 
it makes sense to think in terms of more broadly applicable mechanisms.

Keith