Re: [MMUSIC] Coding from examples [was Re: I-D Action:draft-ietf-mmusic-ice-tcp-09.txt]

[Marc Petit-Huguenin <[e-mail]>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/06/2010 01:45 PM, Ari Keranen wrote:
> 4.9.2010 1:39, Marc Petit-Huguenin wrote:
>> On 09/03/2010 01:28 PM, Ari Keranen wrote:
>>> 3.9.2010 20:11, Marc Petit-Huguenin wrote:
>>>> On 09/03/2010 05:33 AM, Simon Perreault wrote:
>>>>> On 2010-09-02 19:32, Dan Wing wrote:
>>>>>> In any event, if we're going to do a big example, how about
>>>>>> a real example in draft-ietf-sipping-nat-scenarios, if it
>>>>>> doesn't have it already?
>>>>>
>>>>> I don't care where it is, but we need one. When you're implementing
>>>>> something, you end up looking longer at examples than at the actual
>>>>> spec. It really is extremely useful.
>>>>
>>>> And, in my opinion, terribly wrong.  I personally would ban all
>>>> examples from
>>>> RFC, because at best they are exposing only a subset of what the RFC
>>>> describe
>>>> and at worst there is bugs in it, bugs[1] that you will then find in
>>>> implementations.
>>>
>>> I agree with Simon that an example would be really useful, but I think
>>> also Marc is right that having it wrong is likely worse than not having
>>> it at all.
>>>
>>> Since we have a draft (the nat-scenarions one) that is already
>>> documenting this kind of things, I'd say that would be a better place
>>> for the example and we can add an informative reference so that
>>> implementors know where to look at.
>>>
>>> Also, the examples in the nat-scenarios draft should probably be run
>>> through multiple real implementations and still would need disclaimers
>>> saying "there are probably errors in the examples, so take a look at the
>>> errata, etc.", but that's a whole other story..
>>
>> The way I managed the examples in my recently published RFC[1] is
>> twofold:
>>
>> 1. I refused to add examples to explain something until the normative
>> text
>> itself was clear enough to in fact make the examples redundant. 
>> Examples are
>> fine for people that are just interested in vaguely understanding what
>> it is
>> about but implementers should not look at examples, or at any section
>> titled
>> "Introduction", "Overview" or "Informative References".  I wish I
>> could somehow
>> filter this sections automatically when my developers upload RFC from
>> the Internet.
> 
> I agree that implementers should not rely on examples, but they can make
> understanding the specs much easier too and can be therefore really
> useful. That said, given how easily errors creep into examples, I'm
> usually a bit reluctant on having really specific examples in something
> as immutable as an RFC.

I think that a good first step would be to move all the examples in future RFCs
in a separate section, and explicitly declare this section as non-normative.

>  > 2. I generated automatically the examples in the spec.  I first wrote a
>> reference implementation[2] of the RFC.  Then I added unit tests to
>> verify that
>> my reference implementation was correct.  Then finally I wrote the
>> code listed
>> in [3] that automatically generates the RFC2629 fragments listed in [4],
>> fragments that are automatically merged in the draft using Xinclude. 
>> Because
>> the examples were generated each time I built the text version of the
>> draft, I
>> was sure that the examples were correct - or at least as correct as my
>> reference
>> implementation.
> 
> Nice tool chain you got! Something like that would be good also for the
> examples of ICE TCP if we end up having such in the nat-scenarios draft.
> 

I will be happy to help, especially because there is an ICE TCP implementation
in my future.

- -- 
Marc Petit-Huguenin
Personal email: [e-mail]
Professional email: [e-mail]
Blog: http://blog.marc.petit-huguenin.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkyFWIgACgkQ9RoMZyVa61eqwgCePBTYNCK65mXXG6aRjaL+Hsll
eCQAnji5JyeYqR2jgRsJIlZnoE5cT2ef
=FFP1
-----END PGP SIGNATURE-----