Re: The IETF, Standards process, and the impact on the RFC series document production

John C Klensin <john-ietf@jck.com> Fri, 04 October 2019 16:53 UTC

Return-Path: <john-ietf@jck.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D6F4120114 for <ietf@ietfa.amsl.com>; Fri, 4 Oct 2019 09:53:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bRV1UxXnm5IX for <ietf@ietfa.amsl.com>; Fri, 4 Oct 2019 09:53:27 -0700 (PDT)
Received: from bsa2.jck.com (ns.jck.com [70.88.254.51]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 714E81200DF for <ietf@ietf.org>; Fri, 4 Oct 2019 09:53:27 -0700 (PDT)
Received: from [198.252.137.10] (helo=PSB) by bsa2.jck.com with esmtp (Exim 4.82 (FreeBSD)) (envelope-from <john-ietf@jck.com>) id 1iGQpa-000JVT-Dh; Fri, 04 Oct 2019 12:53:26 -0400
Date: Fri, 04 Oct 2019 12:53:21 -0400
From: John C Klensin <john-ietf@jck.com>
To: Michael StJohns <mstjohns@comcast.net>
cc: ietf@ietf.org
Subject: Re: The IETF, Standards process, and the impact on the RFC series document production
Message-ID: <1FC1B07CEB134B4789B10DBA@PSB>
X-Mailer: Mulberry/4.0.8 (Win32)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
X-SA-Exim-Connect-IP: 198.252.137.10
X-SA-Exim-Mail-From: john-ietf@jck.com
X-SA-Exim-Scanned: No (on bsa2.jck.com); SAEximRunCond expanded to false
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/MD-HzACw6xKV6ZljjmNBGfkzwCo>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 04 Oct 2019 16:53:30 -0000

(sorry-- earlier copy sent from wrong address)

Mike,

Strongly +1.  We are attributing too many things to the RFC
Editor Function that are really issues with the standards
process.

Two small additional comments:

(1) You wrote "not (only?) the actual document production that
takes time, but the authoring, and then getting through WG, then
various twiddles by the IESG, and then finally IETF last call,
all before it ever gets to the RFC editor for publication.".
But it isn't "finally IETF Last Call" because there is another
round of IESG twiddling.  In my recent experience, that takes a
minimum of three weeks just to get a document scheduled for a
call.  Then IESG members make comments and vote, typically
within 24 hours of the call time.  If any of those is a DISCUSS
or contains comments of significance, there are further delays,
with a month or more after the Last Call closes not being
unusual.  As you point out, the RFC Production Center gets the
document only after that.  My experience has been that, For a
well-written and reasonably short document that doesn't have
"cluster" entanglements with others, the time they take is
usually trivial relative to everything else.

(2) While I agree that there are document marking issues with
other SDOs that are not hugely different from ours, their
specific review cycles tend to keep them out of the problems we
get into by a collection of extensions, amendment documents that
don't quite replace the originals, replacements that don't
address all outstanding issues, and so on.  While not technical
standards, the IASA2 work has illustrated many of those
problems: ANSI, IEEE, and ISO procedures would make it nearly
impossible to publish a new document that said "this revision
does not address the following problems that are believed or
known to be significant".   While not a perfect solution, the
work of the NEWTRK WG tried to address some of the issues
associated with our collections of related documents but was
never formally considered outside the WG because the IESG
refused to issue a Last Call, so we are not making much progress
in that area.

best,
   john



--On Friday, October 4, 2019 12:11 -0400 Michael StJohns
<mstjohns@comcast.net> wrote:

> Hi Christian - I thought I'd dive down into some of your
> specific comments on the standardization process.  Subject
> changed appropriately.
> 
> On 10/4/2019 3:51 AM, Christian Huitema wrote:
>> 
>> At the same time, there is a tension between the tradition
>> and the need to serve and recruit a wider community. Our
>> document production time is counted in years, and that does
>> not help convincing open source projects to work with us.
> 
> To be clear, it's not (only?) the actual document production
> that takes time, but the authoring, and then getting through
> WG, then various twiddles by the IESG, and then finally IETF
> last call, all before it ever gets to the RFC editor for
> publication.
> 
> A while back I managed to move RFC5011 from Proposed to
> Standard without a document action.  From Last Call to the
> announcement of the standards action was about 6 months.  
> Let me repeat that - 6 months.  And there were no RFC editor
> actions AFAICT.   I think it was most of a year from the
> time I requested the upgrade until it was done.
> 
> WRT to Open Source projects:  I keep running afoul of these
> in that the code tends to be where the standard is documented.
> That's not what we do here.   I do understand we need to
> work with this community, but we also need to avoid being
> captured by any one community.  We also need to have people
> willing to do the hard part of writing the documents and
> shepherding them through the system.
> 
> 
>> Our documents are cast for eternity, errors included,
>> which does not help the part the part of the readership who
>> wants to implement standards.
> 
> Generically, erratas cover non-interoperable errors (and the
> occasional error of a constant or a value).  It's pretty rare
> that not implementing an errata will break something
> AFAICT.   Or do you have a few counter examples with
> implementation experience?
> 
> 
>> Implementing our standards involves a treasure hunt
>> to find how many RFC have to be read before understanding the
>> whole picture.
> This is problematic, but inherent, not so much in the RFC
> series, but in the way we've chosen to do our standards
> process.  We extend, we rarely amend, and we run into
> problems when we amend.  DNS being the most egregious example
> of this (amend -> DNSSEC, extend -> more RFCs than I care to
> think about) that I can think of with possibly TLS a close
> second.   I think that's more of a standards discussion than
> an RFC series discussion.  Also, it is possible (and has been
> done) to write a "here's what you need to do to implement"
> RFC.   Perhaps we  (the IETF - at some cost) might think of
> paying someone to do that for a few of the more extensive
> cases?
>>   And that's before we even consider the potential of
>>   confusion between standards, experiments, independent
>> efforts, and drafts.
> 
> I rarely have a problem differentiating between the relative
> values and quality of each of these, although I'm getting a
> little concerned with the Informational RFCs coming out of the
> CFRG that are actually cryptographic standards of first
> publication.   Each and every RFC (or at least in the last
> 10-15 years or so) has the disclaimer to "go check XXX to find
> out what the status of this document is and what
> 'informational' or 'experimental' actually means".
> 
> Note that this is NOT specific to the IETF.  Every time I
> delve into the worlds of ANSI, IEEE or ISO standards I have to
> refresh my memory on their various varieties of document
> markings.
> 
>> The
>> IETF community will need to solve that over time. I assume
>> that this will imply some sharing of the work between the RFC
>> Editor and the "superstructure" of the IETF, and I wait to
>> see the discussion happening in the ad hoc working group. And
>> yes, of course the discussion process should be open and fair.
> 
> Nothing I've seen above really has an impact on the RFC editor
> as an separate institution.  Perhaps I'm misreading this, but
> perhaps what you're implying is that more attention to
> cleaning up (and speeding up) the standards process
> pre-submission-to-the RFC Editor is warranted?  What do you
> see as the RFC Series Editor's role in that?
> 
> Later, Mike