Re: Things that used to be clear (was Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)

Eric Rescorla <ekr@rtfm.com> Thu, 04 July 2019 15:32 UTC

Return-Path: <ekr@rtfm.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8647012010C for <ietf@ietfa.amsl.com>; Thu, 4 Jul 2019 08:32:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.896
X-Spam-Level:
X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rtfm-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZvL5iTQ5shct for <ietf@ietfa.amsl.com>; Thu, 4 Jul 2019 08:32:26 -0700 (PDT)
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com [IPv6:2a00:1450:4864:20::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 60EBC12006F for <ietf@ietf.org>; Thu, 4 Jul 2019 08:32:26 -0700 (PDT)
Received: by mail-lj1-x236.google.com with SMTP id m23so6509371lje.12 for <ietf@ietf.org>; Thu, 04 Jul 2019 08:32:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rtfm-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Yiewl2bmcJN3V6U9TTzWEvaz7I5zZ3rMQLiOe/XGYZw=; b=I41IkqEZoqBEQzHyg7tg9Yf8R/XcoU8tcuev3iDp2Bje4WNQf1/eIkMZ8/auQF6zaa biU6hZTJytWDE7iMVldPyglT94oelkNEk2j9LE3zDqJBgw20yA3wH3iyV/cUFfQmUZCi vLrrl9kqSaDDtdH1GYD5SYQ12X9AfMsIFLfdM8mPMsl95xqH7v5AQYsK/jBBY6MyGHvI kc1GRbTYsnXniJpZ8KjJNuyf35guDFi4uFc9QnAhGnSJdcsB0XZwboqWFdRNUABffYz1 dVSUCWo6uvoZ7p5EEeLO+IHifhbAnQ7f06S4m4ewINio9aR0VltCCy5D0EogrhG+7B9B bXFg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Yiewl2bmcJN3V6U9TTzWEvaz7I5zZ3rMQLiOe/XGYZw=; b=qThflyHgHB3/gmSxr/bfqRm5ozp+RI2PyrhPDbs9zTkUJJu8OtoncAzAmn4CU2wfo+ qlZKf2SScbyDYFWDIoG+ZeDFSzi61dKn0CtgK9a2LRv27T6kM8duNax2LAPqJddeq6MF vUHW8LogigL0TizKiV2pciGoiDW0QRXRC50fk/jmYiaccS6za6cF+dCI7/1dj4j5/rLz 59pSDjJIyQMUzqcVI48tQ4KHE9qOBMM7YG3iOCvMSh7o/Fz2DpjYRrRk7Jt3gdw+IJx2 7oMYkCXmP9QnZj3T+m/nRF0Te8qayts70CFfWio1UjO5mlMhEmRneUlQ1L30yXisHd7W 784A==
X-Gm-Message-State: APjAAAXpvSterdHVbyviic2cNV75CpTdeA1SMFW/9exXfBvXZdC1O4tq fUmGCW7/aIUpUhuYAx+0T1pDXYcFgeYIxgUxSvPXVh0fivBWmg==
X-Google-Smtp-Source: APXvYqyXDBrfzF2uL65cbWdaXFnTsLAfGpUGppEe3i6YhVkDmu8CThq0bIok6hZj1oSzWXo/U4Kvleg26hGTGsyVaaM=
X-Received: by 2002:a2e:890a:: with SMTP id d10mr25049341lji.145.1562254344476; Thu, 04 Jul 2019 08:32:24 -0700 (PDT)
MIME-Version: 1.0
References: <CAHw9_iKv7xDY-rT98F_BAEvGOGbWGL7UpXS42rSVLsHB+=SOZg@mail.gmail.com> <4567879e-aa29-aeae-72e9-33d148d30eed@network-heretics.com> <CAL02cgToQWmOrfOxS_dc4KRtT9e0PXNzmhWZHkRUyV_3V=E-mQ@mail.gmail.com> <0856af71-4d84-09d1-834d-12ac7252420c@network-heretics.com> <CAL02cgQ9qWVUTPW=Cpx=r32k3i1PLgfp5ax0pKMdH0nKObcKTg@mail.gmail.com> <e8d28a7f-128d-e8d0-17d3-146c6ff5b546@joelhalpern.com> <CAHw9_i+UBs85P+gjcF6BJd1_WD2qFrrYCnXb4rtcG9Hepqm37w@mail.gmail.com> <796c1f6c-cd67-2cd5-9a98-9059a0e516f8@network-heretics.com> <20190704013009.dlifopcbm2umnqo7@mx4.yitter.info> <b18809df-ee98-fb29-b6c4-04ed579e163a@network-heretics.com> <20190704052335.GF3508@localhost>
In-Reply-To: <20190704052335.GF3508@localhost>
From: Eric Rescorla <ekr@rtfm.com>
Date: Thu, 04 Jul 2019 08:31:47 -0700
Message-ID: <CABcZeBOw6w2tm4YYFdmLwC23ufPDupt2D1Vzwjn4Pi9bbf6R-w@mail.gmail.com>
Subject: Re: Things that used to be clear (was Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)
To: Nico Williams <nico@cryptonector.com>
Cc: Keith Moore <moore@network-heretics.com>, IETF discussion list <ietf@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000004068c6058cdcaf14"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/DPAbl-B3L3pktvFiZo7k6yZ4mRo>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Jul 2019 15:32:30 -0000

I largely agree with Nico's take here.

Ignoring labelling for a moment, in a number of WGs (HTTP, TLS, and
QUIC) we have found it necessary to have full implementations and
large-scale deployments quite early in the design process, long before
anyone thinks that the document is done.

Common practice has become something like this:

1. Once the draft has become complete enough that it's possible to do
   so, start doing test implementations based on labelled versions
   of the draft and try to interop those.

2. Once the draft has become complete enough that it's believed to be
   safe, do coordinated field trials based on identified versions of
   the draft. This may include, for instance, deploying to a large
   fraction of the user base (e.g., all Firefox Beta users or a
   fraction of the Firefox release population) and/or to a big server
   farm (e.g., Cloudflare's users). We use version identifiers that
   are associated with the draft number to avoid interop problems.

3. Once the document is published at PS, switch over to the final
   version (which is usually nearly identical to the last draft version).

This does not really fit into the PS/DS model: We absolutely need
to deploy early versions to find potential issues (this was critical
to TLS 1.3) but we all know that those documents don't meet the PS
bar as they may have known defects or at least open issues. There's
also no real use for Full Standard. The market doesn't care about it
and the people who are doing the work either are (1) fixing/extending
the protocol (using the process above) or (2) have moved onto some
other protocol.

It's not clear to me how well this Evolving Documents proposal would
fit into such a model [0], but I thought a field report on what is becoming
common practice might be useful.

-Ekr

[0] The real need I find is to be able to make minor fixes to the
documents (mostly editorial errata or clarification of points on which
there was consensus) without re-spinning the RFC, which people don't
have the energy for.


On Wed, Jul 3, 2019 at 10:24 PM Nico Williams <nico@cryptonector.com> wrote:

> On Wed, Jul 03, 2019 at 09:52:03PM -0400, Keith Moore wrote:
> > On 7/3/19 9:30 PM, Andrew Sullivan wrote:
> >
> > > > difficulties.    It used to be clear that you didn't deploy
> implementations
> > > > based on Proposed Standard, but people did anyway.
> > > When was that "clear"?
> >
> > Probably I was thinking of RFC2026 section 4.1.1, last paragraph:
> >
> >    Implementors should treat Proposed Standards as immature
> >    specifications.  It is desirable to implement them in order to gain
> >    experience and to validate, test, and clarify the specification.
> >    However, since the content of Proposed Standards may be changed if
> >    problems are found or better solutions are identified,/deploying
> > implementations of such standards into a disruption-sensitive
> environment is
> > not recommended./
> >
> > But of course that's not stating it as strongly as I remembered, and the
> > problem of deploying implementations based on Proposed Standard existed
> even
> > before that.   I remember a flap about telnet implementations circa 1992
> in
> > which implementations of a certain option didn't interoperate - one
> vendor
> > followed the PS text and all of the others implemented it in the opposite
> > way, and I heard a lot of people saying "they shouldn't have deployed at
> > Proposed".
>
> In the security area just about all major Internet protocols are at
> Proposed Standard.  PKIX?  Proposed Standard.  Kerberos?  Ditto.  TLS?
> Yup.  SSHv2?  Indeed.  IKEv2?  No, IKEv2 and CMS are among the
> exceptions, though what good IKEv2 might do anyone w/o ESP, or CMS w/o
> PKIX, I don't know.
>
> Whatever the intention originally might have been, it's certainly long
> not been the case that one should not deploy protocols that are at
> Proposed Standard.
>
> And it's very difficult to stop vendors from shipping pre-RFC protocols.
> We don't have a protocol police, and we move too slowly.  If we don't
> adapt, other SDOs will do more of our work.  A big selling point of the
> IETF is its review processes -- the adults in the room to keep authors
> from doing dreadful things.  But we need to speed up the cycle somewhat,
> and one way to do it might be to have a way to indicate expected
> stability in I-Ds, and probably only in WG work items only, and at some
> cost (e.g., early directorate reviews?).  I don't quite know -- maybe
> after reflection we might conclude we shouldn't do this, but we should
> certainly discuss it, and be able to discuss it.
>
> Nico
> --
>
>