Re: Running code, take 2

[Yaron Sheffer <[e-mail]>]

Hi John,

to clarify, my proposal only applies to Internet Drafts, and clearly 
states that the implementation section should be removed from the 
document before it is published as RFC.

Formally, we don't want non-permanent stuff in RFCs. And realistically, 
even if we had an implementation wiki, it is unlikely to be kept up to 
date once the RFC is published.

Regarding the "experiment" question: I fully agree that there's nothing 
preventing us from adopting this proposal (or a variant of it, or 
Stephen's proposal) today.

The value in a 3933 experiment is in the Summary Report, otherwise I 
agree it's a waste of time. At the end of the period we will have a 
little bit of data to understand whether we have traction for this idea, 
and whether we should make it IETF-wide, allow it to quietly die or 
explicitly advise against it.

Thanks,
	Yaron

On 12/14/2012 12:47 AM, John C Klensin wrote:
>
>
> --On Thursday, December 13, 2012 23:51 +0200 Yaron Sheffer
> <[e-mail]> wrote:
>
>> Hi Randy,
>>
>> the RPKI report is very impressive and probably very useful.
>> But:
>>
>> - Other areas (e.g. the Security Area, where I'm coming form)
>> may not have this tradition.
>
> If it is important, nothing prevents establishing it.
>
>> - For "smaller" protocols the gain does not justify the effort
>> of writing a separate implementation report. A section in the
>> base document is much simpler to maintain.
>
> I'm not at all clear why it is more effort to write a separate
> report.  But, assuming it is, I am aware of nothing in any IETF
> or RFC Editor procedure that prohibits including a discussion of
> implementation experience in a standards track RFC.
>
>> - And as I noted earlier on this thread, a wiki is a good
>> alternative, provided everyone reading the base draft is
>> referred to the "implementation status" wiki. They should not
>> have to guess that one exists, or to search for it.
>
> That is a more complicated problem because the RFC Editor is
> somewhat reluctant (for good reasons, IMO) to accept references
> to web pages whose permanency is not assured.  There are,
> however, lots of other ways by which one could imagine creating
> and maintaining such pointers.  For example, if you could
> convince the community that this was important, and likely to be
> done often enough, you might lobby the IESG to negotiate an
> additional sentence in the "Status" section of standards track
> RFCs that provided a stable pointer to an index of
> implementation reports.
>
> My concern remains that we not create new formal procedures to
> do (or even experiment with) things that can be done under
> existing rules either for the whole IETF or on an area by area
> or even document by document basis,  Let me suggest, as
> co-author of the "process experiment" doc, that it was intended
> to deal to allow experiments that would otherwise be in
> violation of existing procedures.  As far as I can see, that
> situation doesn't apply with either of these proposals.  And,
> given the number of experiments carried out with reference to
> RFC 3933 that did not result in useful and formal reports back
> to the community, I'm unconvinced that formally identifying any
> particular exercise in identifying implementations or handling
> some specifications with implementations differently in the IESG
> review process than other specifications will increase the odds
> of  the results being examined carefully and critically enough
> to generate a report and, through it, permanently change
> behavior.  YMMD but, if either of these ideas are interesting,
> persuade a friendly AD to try it on whatever scale makes sense,
> and then let the rest of us know how it worked out for you.
> Since no existing rules would thereby be violated, IETF
> consensus is not required for such an exercise... or any number
> of such exercises.
>
> best,
>     john
>