Re: [MMUSIC] Coding from examples [was Re: I-D Action:draft-ietf-mmusic-ice-tcp-09.txt]

[Ari Keranen <[e-mail]>]

7.9.2010 0:09, Marc Petit-Huguenin wrote:
> On 09/06/2010 01:45 PM, Ari Keranen wrote:
>> 4.9.2010 1:39, Marc Petit-Huguenin wrote:
>>> On 09/03/2010 01:28 PM, Ari Keranen wrote:
>>>> 3.9.2010 20:11, Marc Petit-Huguenin wrote:
>>>>> On 09/03/2010 05:33 AM, Simon Perreault wrote:
>>>>>> On 2010-09-02 19:32, Dan Wing wrote:
>>>>>>> In any event, if we're going to do a big example, how about
>>>>>>> a real example in draft-ietf-sipping-nat-scenarios, if it
>>>>>>> doesn't have it already?
>>>>>>
>>>>>> I don't care where it is, but we need one. When you're implementing
>>>>>> something, you end up looking longer at examples than at the actual
>>>>>> spec. It really is extremely useful.
>>>>>
>>>>> And, in my opinion, terribly wrong.  I personally would ban all
>>>>> examples from
>>>>> RFC, because at best they are exposing only a subset of what the RFC
>>>>> describe
>>>>> and at worst there is bugs in it, bugs[1] that you will then find in
>>>>> implementations.
>>>>
>>>> I agree with Simon that an example would be really useful, but I think
>>>> also Marc is right that having it wrong is likely worse than not having
>>>> it at all.
>>>>
>>>> Since we have a draft (the nat-scenarions one) that is already
>>>> documenting this kind of things, I'd say that would be a better place
>>>> for the example and we can add an informative reference so that
>>>> implementors know where to look at.
>>>>
>>>> Also, the examples in the nat-scenarios draft should probably be run
>>>> through multiple real implementations and still would need disclaimers
>>>> saying "there are probably errors in the examples, so take a look at the
>>>> errata, etc.", but that's a whole other story..
>>>
>>> The way I managed the examples in my recently published RFC[1] is
>>> twofold:
>>>
>>> 1. I refused to add examples to explain something until the normative
>>> text
>>> itself was clear enough to in fact make the examples redundant.
>>> Examples are
>>> fine for people that are just interested in vaguely understanding what
>>> it is
>>> about but implementers should not look at examples, or at any section
>>> titled
>>> "Introduction", "Overview" or "Informative References".  I wish I
>>> could somehow
>>> filter this sections automatically when my developers upload RFC from
>>> the Internet.
>>
>> I agree that implementers should not rely on examples, but they can make
>> understanding the specs much easier too and can be therefore really
>> useful. That said, given how easily errors creep into examples, I'm
>> usually a bit reluctant on having really specific examples in something
>> as immutable as an RFC.
>
> I think that a good first step would be to move all the examples in future RFCs
> in a separate section, and explicitly declare this section as non-normative.

That would make sense.

>>   >  2. I generated automatically the examples in the spec.  I first wrote a
>>> reference implementation[2] of the RFC.  Then I added unit tests to
>>> verify that
>>> my reference implementation was correct.  Then finally I wrote the
>>> code listed
>>> in [3] that automatically generates the RFC2629 fragments listed in [4],
>>> fragments that are automatically merged in the draft using Xinclude.
>>> Because
>>> the examples were generated each time I built the text version of the
>>> draft, I
>>> was sure that the examples were correct - or at least as correct as my
>>> reference
>>> implementation.
>>
>> Nice tool chain you got! Something like that would be good also for the
>> examples of ICE TCP if we end up having such in the nat-scenarios draft.
>>
>
> I will be happy to help, especially because there is an ICE TCP implementation
> in my future.

That would be excellent, thanks!


Cheers,
Ari