Re: draft-farrell-ft-01.txt -- what signal are we attempting to sense?

Ted Hardie <> Wed, 05 December 2012 19:36 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 5C43A21F860F for <>; Wed, 5 Dec 2012 11:36:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.377
X-Spam-Status: No, score=-3.377 tagged_above=-999 required=5 tests=[AWL=0.221, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 0VJOtHtfNLpC for <>; Wed, 5 Dec 2012 11:36:38 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id B86FF21F842E for <>; Wed, 5 Dec 2012 11:36:37 -0800 (PST)
Received: by with SMTP id i20so2077724qad.10 for <>; Wed, 05 Dec 2012 11:36:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=g020bXBpSK+v6JxwtmSNELIyKYVfhRZQN4Lxl9d3oqk=; b=Sl82B/gC4P4xpSxTTDLFikAdgYKFqkSJ7CFwx6+0LxwemQWnU3pNYbaiGTKhwtKzFS eLLHR53b34000bIvYBOmOJQ2ReTGAfRfgklyVGN4/UXp7lQpVx/2njl4W/Ljw3pjIWaR 3dVKpxGFTvHfd8SzcqQ6/3OUmBnM5tfGBsWXB0ZfVg20ywrWwMufr1nG23LTI9ufm7hn m9/afWfwaFIcbNpV/4Lonk4RkLMRnJYTPCQ1kEX5SwAVQqOoF2yIcK0s0/YEW+AlqbXZ 0TAqkuUM0/TM0bSK4c5625q/1ViMc7pkQq3APl9tAoeqfIsmYKfBhFXdJtHDDDlr4iym 6tew==
MIME-Version: 1.0
Received: by with SMTP id k17mr3013668qen.51.1354736197269; Wed, 05 Dec 2012 11:36:37 -0800 (PST)
Received: by with HTTP; Wed, 5 Dec 2012 11:36:37 -0800 (PST)
In-Reply-To: <>
References: <> <>
Date: Wed, 05 Dec 2012 11:36:37 -0800
Message-ID: <>
Subject: Re: draft-farrell-ft-01.txt -- what signal are we attempting to sense?
From: Ted Hardie <>
To: Stephen Farrell <>
Content-Type: multipart/alternative; boundary="047d7b6da460cd7ac104d020194c"
Cc: IETF <>
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IETF-Discussion <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 05 Dec 2012 19:36:39 -0000

Hi Stephen,

Some further comments in-line.

On Wed, Dec 5, 2012 at 9:58 AM, Stephen Farrell

> Hi Ted,
> On 12/05/2012 05:22 PM, Ted Hardie wrote:
> > Reading through Stephen's draft and the discussion to date, I think there
> > is some confusion/disagreement about what it is having an implementation
> > at this stage signals.
> >
> > One way to break up the work of the IETF is:
> >
> > Engineering--making decisions about the trade-offs related to
> >              different approaches to solving a problem.
> >
> > Specification--producing text that describes how to inter-operate with
> > others.
> >
> > Standardization--describing the applicability of a specification or
> >          its suitability as the basis of other work
> >
> > (Since we reflect all of these in the same document production, it's
> > really muddier than this, but bear with me)
> >
> > My experience is that people implementing during the working group
> > discussion phase generate really useful data about the engineering;
> > they can tell you the real impact of different trade-offs, so that
> > this isn't based on general experience.  But it's not such a great
> > signal about the specification itself, since the spec is designed to
> > be usable by folks who were not part of the working group process.
> Good point.
> > Stephen's draft says:
> >
> >    Note also that this experiment just needs an implementation that
> >    makes it possible for the WG chairs and responsible AD to verify (to
> >    the extent they chose) that the implementation matches the draft.
> >
> > and later:
> >
> >    An implementation of the draft (ideally open-source) is required
> >    for fast-track last-call.  If there is no implementation or if the
> >    implementation is unavailable or does not implement the draft
> >    sufficiently closely then the document needs to be returned to the
> >    WG.  This only requires one implementation, not two and the WG
> >    chairs and responsible AD decide themselves how much validation is
> >    required for this.
> >
> > Given the "sufficiently closely" and the timing of production, I
> > assume that the signal we're looking for here is confirmation of the
> > engineering choices.  I think that's fine (though I'm not sure this
> > needs formal experiment status).
> I hadn't thought about this in terms of signals, but its an
> interesting way to look at it.
> BTW, others have also said (and I agree) that lots of the things
> suggested in the draft don't need to be a formal experiment,
> but I think doing it that way has merit in any case.
> > But I believe we need to be really
> > careful that it isn't mistaken for signal about the specification's
> > quality.  It can happen that a working group has "lore" about what to
> > do that gets folded into the implementations done by those
> > participating, but which never quite makes it into the spec "because
> > everyone knows it".  An implementation written during the working
> > group process is potentially subject to this effect.
> >
> > An interoperating implementation written to the spec by a non-working
> > group participant would be great signal about the specification quality,
> > but it  is not likely to be available at the stage of the process this
> > draft
> > targets.
> That's very true.
> I guess I'd be a bit reluctant to try to add text to the
> draft about this though, since it might be a bit of a rathole
> to try discuss generic good/bad aspects of implementations
> and specs. I could see that discussion running and running
> and getting nowhere;-)
I don't think it needs to be a rat-hole.  Let me suggest the following


   Sometimes, it can take a long time to get a Proposed Standard
   produced in the IETF.  This memo proposes an optional way to speed up
   the parts of the process that happen after a WG has done its job by
   building a "reward" for having an implementation (ideally open-
   source) available into IETF processes.


Implementations developed during the production of an Internet-draft
can indicate that a working group has had the opportunity to get
early confirmation of its engineering choices.  This memo proposes
an optional way to parallel process some final stage reviews when
the working group management and area directors believe that the
implementation can itself serve as a practical review of the engineering

Does that make sense?

> But maybe it'd be a good idea to maintain a wiki as the
> experiment runs where such things could be captured that
> could feed into later evaluation of how things went.
> If that sounds useful, I'm willing to start that (maybe
> once the draft's gotten past IETF LC) and add a bit of text
> to the draft pointing at the wiki page. I'm also happy to
> do some maintenance on that as things progress, if they do.
> If you think some other changes would help instead or
> as well, I'll gladly take text of course, or suggestions
> for how to (re-)structure the text.
I think it might generally useful to talk about this in terms of
parallel processing review, rather than issuing a reward, but
that may be simply a a style preference.


Ted Hardie

> Cheers,
> S.
> > Again, not objecting to the experiment; I just want to be clear about
> > what signal we believe we're getting from the implementation.
> >
> > Just my two cents,
> >
> > Ted
> >