Re: Running Code

[Jari Arkko <[e-mail]>]

Hannes,

> The work was done in a design team, it took a very long time (the first
> design team draft dates back to May 2006). 
>
> Jouni Malinen implemented the protocol in 8 hours!

Not a good spec time / implementation time ratio! There are also 
examples of people starting and *finishing* their PhD on the topic of an 
RFC *before* the RFC comes out. Not a good sign of the effort required 
to publish an RFC...

Picture about the process for this particular draft can be found in 
http://www.arkko.com/tools/lifecycle/draft-ietf-emu-eap-gpsk-timing.html 
-- what can we do to make the process smoother?

> There are three main problems: 
>
> * Almost random changes to the specification make early implementation work
> almost useless (if your goal is to produce code that aims having code for a
> specific RFC after all). Since it takes so long to finish the
> standardization work it is often not possible to wait till the RFC comes
> out. 
>
> * Nobody cares if you have running code. Requests to substantially change
> the specification often come at a late stage. Even IESG members have already
> re-written specifications during IETF Last Call. As a joke, I suggested to
> just submit the table of contents to the IESG and then start the work there.
>
> * Finally, because it takes so long to finish specifications the 3-level
> Standards Track process is rarely utilized anymore. That process considers
> interoperable code but what does it help? 
>   
These are issues. And my opinion is that we have to take implementation 
experience very seriously. I would like to disagree with you on the 
assertion "nobody cares if you have running code", however. I think all 
of us care. But running code does not necessarily override ALL other 
concerns. If there's a serious bug in the spec, it needs to be fixed, 
even if it is noticed late in the process. Personally, I find that we 
waste most of the time waiting (for reviews, for the author to revise a 
spec, for the AD to respond, etc). If we reduce that we can get the 
process much faster.

The entire value of the IETF specification effort is that you get 
comments and as a result the specification improves. That necessarily 
implies that there may be changes from your initial implementation. If 
the  goal is absolute minimum publication time and no changes to 
implementation, we should all just be publishing protocol specifications 
on our web pages or talking to the RFC Editor -- and this is what we do 
in many cases, but it is not the right approach for producing a standard.

Jari