Re: draft-farrell-ft-01.txt -- what signal are we attempting to sense?

Stephen Farrell <> Wed, 05 December 2012 21:55 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id BFF8F21F8C3E for <>; Wed, 5 Dec 2012 13:55:31 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -102.449
X-Spam-Status: No, score=-102.449 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id u9asH4Dn5SWq for <>; Wed, 5 Dec 2012 13:55:30 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id 8A94721F8BBA for <>; Wed, 5 Dec 2012 13:55:30 -0800 (PST)
Received: from localhost (localhost []) by (Postfix) with ESMTP id 7658FBE33; Wed, 5 Dec 2012 21:55:08 +0000 (GMT)
X-Virus-Scanned: Debian amavisd-new at
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id H5wjldnq07-S; Wed, 5 Dec 2012 21:55:07 +0000 (GMT)
Received: from [] (unknown []) by (Postfix) with ESMTPSA id 67D04BE25; Wed, 5 Dec 2012 21:55:07 +0000 (GMT)
Message-ID: <>
Date: Wed, 05 Dec 2012 21:55:07 +0000
From: Stephen Farrell <>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/17.0 Thunderbird/17.0
MIME-Version: 1.0
To: Ted Hardie <>
Subject: Re: draft-farrell-ft-01.txt -- what signal are we attempting to sense?
References: <> <> <>
In-Reply-To: <>
X-Enigmail-Version: 1.4.6
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
Cc: IETF <>
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IETF-Discussion <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 05 Dec 2012 21:55:31 -0000

Hi Ted,

Thanks that change looks good to me. I'll whack it in thanks. I do
still like the word "reward" though so I'll tag that on too:-)


On 12/05/2012 07:36 PM, Ted Hardie wrote:
> Hi Stephen,
> Some further comments in-line.
> On Wed, Dec 5, 2012 at 9:58 AM, Stephen Farrell
> <>wrote:
>> Hi Ted,
>> On 12/05/2012 05:22 PM, Ted Hardie wrote:
>>> Reading through Stephen's draft and the discussion to date, I think there
>>> is some confusion/disagreement about what it is having an implementation
>>> at this stage signals.
>>> One way to break up the work of the IETF is:
>>> Engineering--making decisions about the trade-offs related to
>>>              different approaches to solving a problem.
>>> Specification--producing text that describes how to inter-operate with
>>> others.
>>> Standardization--describing the applicability of a specification or
>>>          its suitability as the basis of other work
>>> (Since we reflect all of these in the same document production, it's
>>> really muddier than this, but bear with me)
>>> My experience is that people implementing during the working group
>>> discussion phase generate really useful data about the engineering;
>>> they can tell you the real impact of different trade-offs, so that
>>> this isn't based on general experience.  But it's not such a great
>>> signal about the specification itself, since the spec is designed to
>>> be usable by folks who were not part of the working group process.
>> Good point.
>>> Stephen's draft says:
>>>    Note also that this experiment just needs an implementation that
>>>    makes it possible for the WG chairs and responsible AD to verify (to
>>>    the extent they chose) that the implementation matches the draft.
>>> and later:
>>>    An implementation of the draft (ideally open-source) is required
>>>    for fast-track last-call.  If there is no implementation or if the
>>>    implementation is unavailable or does not implement the draft
>>>    sufficiently closely then the document needs to be returned to the
>>>    WG.  This only requires one implementation, not two and the WG
>>>    chairs and responsible AD decide themselves how much validation is
>>>    required for this.
>>> Given the "sufficiently closely" and the timing of production, I
>>> assume that the signal we're looking for here is confirmation of the
>>> engineering choices.  I think that's fine (though I'm not sure this
>>> needs formal experiment status).
>> I hadn't thought about this in terms of signals, but its an
>> interesting way to look at it.
>> BTW, others have also said (and I agree) that lots of the things
>> suggested in the draft don't need to be a formal experiment,
>> but I think doing it that way has merit in any case.
>>> But I believe we need to be really
>>> careful that it isn't mistaken for signal about the specification's
>>> quality.  It can happen that a working group has "lore" about what to
>>> do that gets folded into the implementations done by those
>>> participating, but which never quite makes it into the spec "because
>>> everyone knows it".  An implementation written during the working
>>> group process is potentially subject to this effect.
>>> An interoperating implementation written to the spec by a non-working
>>> group participant would be great signal about the specification quality,
>>> but it  is not likely to be available at the stage of the process this
>>> draft
>>> targets.
>> That's very true.
>> I guess I'd be a bit reluctant to try to add text to the
>> draft about this though, since it might be a bit of a rathole
>> to try discuss generic good/bad aspects of implementations
>> and specs. I could see that discussion running and running
>> and getting nowhere;-)
> I don't think it needs to be a rat-hole.  Let me suggest the following
> text:
> OLD:
>    Sometimes, it can take a long time to get a Proposed Standard
>    produced in the IETF.  This memo proposes an optional way to speed up
>    the parts of the process that happen after a WG has done its job by
>    building a "reward" for having an implementation (ideally open-
>    source) available into IETF processes.
> NEW:
> Implementations developed during the production of an Internet-draft
> can indicate that a working group has had the opportunity to get
> early confirmation of its engineering choices.  This memo proposes
> an optional way to parallel process some final stage reviews when
> the working group management and area directors believe that the
> implementation can itself serve as a practical review of the engineering
> choices.
> Does that make sense?
>> But maybe it'd be a good idea to maintain a wiki as the
>> experiment runs where such things could be captured that
>> could feed into later evaluation of how things went.
>> If that sounds useful, I'm willing to start that (maybe
>> once the draft's gotten past IETF LC) and add a bit of text
>> to the draft pointing at the wiki page. I'm also happy to
>> do some maintenance on that as things progress, if they do.
>> If you think some other changes would help instead or
>> as well, I'll gladly take text of course, or suggestions
>> for how to (re-)structure the text.
> I think it might generally useful to talk about this in terms of
> parallel processing review, rather than issuing a reward, but
> that may be simply a a style preference.
> regards,
> Ted Hardie
>> Cheers,
>> S.
>>> Again, not objecting to the experiment; I just want to be clear about
>>> what signal we believe we're getting from the implementation.
>>> Just my two cents,
>>> Ted