Re: [apps-discuss] Objection to processing draft-ietf-appsawg-text-markdown-* documents as WG drafts (was: Re: Benoit Claise's Discuss on draft-ietf-appsawg-text-markdown-use-cases-02: (with DISCUSS and COMMENT))

[John C Klensin <[e-mail]>]

--On Sunday, July 12, 2015 21:43 -0700 Dave Crocker
<[e-mail]> wrote:

> On 7/12/2015 4:20 PM, John C Klensin wrote:
>> The kinds of problematic text (and a few actual
>> errors) that are being found during and subsequent to IETF
>> Last Call suggests strongly that the WG process failed to do
>> an adequate review from several perspectives. 
> 
> 
> John,
> 
> I'm confused.  You appear to be saying that it is exceptional
> for a working group process to turn out documents for IESG
> processing that are seriously flawed.  In contrast I believe
> it is a periodic occurrence across all areas and all working
> groups, and arguably always has been.

Dave,

While I agree that there are periodic occurrences of seriously
flawed documents that are independent of any particular
procedural framework, I think there is a difference that is
worth noting.  I also wouldn't describe the text-markdown
documents as "seriously flawed" or even "crappy", just poorly
written and apparently unaware of some issues and established
terminology that overlaps them.    To be a little more specific,
I'm unconvinced that there is anything wrong with the two of
them that could not be straightened out by a pass or two from a
good technical editor who understood the area of application.
However, we've been told that we cannot expect that level of
editing from the RFC Editor, which means that, if the author
doesn't do it, someone else should notice the problem and do
something about it, with relying on editing by the IESG
post-Last-Call being a particularly unfortunate approach.  I
don't think that issue is unique to any particular type of WG or
other structure either.

> While crappy drafts really are problematic, I do not
> understand what was supposedly exceptional in the AppsArea wg
> process that distinguishes it from various other working
> groups around the IETF over the years, including these days.

With the understanding that there are "in between" cases,
consider two scenarios:

(1) We have a relatively classic-style IETF WG, chartered for a
very specific topic and limited number of documents and
typically expected to terminate when those are done.

(2) We have a WG with a charter for "catch-all" work that may
extend over a rather wide range of topics, with topics and
documents being added to the workload by consensus (or, more
often, especially if the WG leadership likes the proposals,
absence of consensus objections).  

Now, as you point out, neither of those scenarios provides
perfect protection about poor review, "crappy" drafts, or even
technically-defective work (I believe the current work _not_
technically defective).  However, under the first scenario,
there are strong assumptions that everyone who is participating
in the work is, indeed, participating.  Those who get involved
typically are relatively expert in the WG's topic area or at
least care enough to become informed (or should).  Mailing lists
tend to be less noisy than the second case if only because the
range of topics is narrower.  Documents either get reviewed or
engaged and alert WG Chairs engage in a good deal of exhortation
and shepherds have some obligation to report to the IESG when
there is an appearance of inadequate or indifferent reviews.
Certainly, it doesn't always work, especially in "victory by
attrition" cases, but the odds are in its favor.  

On the other hand, when one has a WG with a charter that is both
very broad in terms of topics and presumes it will stay alive
for a long time, adding new topics as it goes along, very few
people "in the WG" are going to be expert on, or care about,
every topic.  While some people may try to carefully review
every document that shows up, at least at WG LC, reactions
similar to "glad someone is doing that, but not my problem or
area of  expertise" are almost certain (and, IMO, common and
expected).

So I think there really is a difference, at least in risk and
expectations,... and that it does not require a catalog of
"crappy drafts" to prove it.

At the same time, I think that one of the remedies for the
"crappy draft" problem would be for the IESG to be a lot more
willing, when it becomes clear that a draft needs significantly
more review and/or work, to bounce it back to the originator or
originating WG and say "this is not of acceptable quality, take
it back to the virtual drawing board and, when you resubmit its
successor, be prepared to explain how something reached us in
this state".  I think it should be reasonable to do that as soon
as "not of acceptable quality" (perhaps the same as what you
mean by "crappy") is clear, whether that is before IETF LC,
during it, or after LC is nominally completed.  That suggestion
is not specific to any particular WG organization model or even
to the difference between WG and individual submissions but it
seems to me that the IESG's trying to fix such documents after
Last Call is not only a bad idea generally but reinforces bad
behavior.  Lest I be accused of being value, consider the
difference in incentives for someone deciding to hand a draft
off for Last Call between:

(i) "I think maybe this is good enough to get by and, if it
isn't, the IESG will fix it and it will get published".

and 

(ii) "I think maybe this is good enough but, if I'm wrong and
the IESG bounces it, it will result in loss of a lot more time
and possibly reassignment of individual submissions to a WG or
aggregate/area WG documents to a short-lived, document specific
WG", losing even more time".  

Faced with the strategy suggested above, I suggest that many
more authors, shepherds, and WG Chairs would adopt the second
line of thinking and try to put a little more energy into being
sure the document really is ready.  I think that would be A Good
Thing.   YMMD.

    john


   best,
     john