Re: [MMUSIC] Coding from examples [was Re: I-D Action:draft-ietf-mmusic-ice-tcp-09.txt]

[Ari Keranen <[e-mail]>]

4.9.2010 1:39, Marc Petit-Huguenin wrote:
> On 09/03/2010 01:28 PM, Ari Keranen wrote:
>> 3.9.2010 20:11, Marc Petit-Huguenin wrote:
>>> On 09/03/2010 05:33 AM, Simon Perreault wrote:
>>>> On 2010-09-02 19:32, Dan Wing wrote:
>>>>> In any event, if we're going to do a big example, how about
>>>>> a real example in draft-ietf-sipping-nat-scenarios, if it
>>>>> doesn't have it already?
>>>>
>>>> I don't care where it is, but we need one. When you're implementing
>>>> something, you end up looking longer at examples than at the actual
>>>> spec. It really is extremely useful.
>>>
>>> And, in my opinion, terribly wrong.  I personally would ban all
>>> examples from
>>> RFC, because at best they are exposing only a subset of what the RFC
>>> describe
>>> and at worst there is bugs in it, bugs[1] that you will then find in
>>> implementations.
>>
>> I agree with Simon that an example would be really useful, but I think
>> also Marc is right that having it wrong is likely worse than not having
>> it at all.
>>
>> Since we have a draft (the nat-scenarions one) that is already
>> documenting this kind of things, I'd say that would be a better place
>> for the example and we can add an informative reference so that
>> implementors know where to look at.
>>
>> Also, the examples in the nat-scenarios draft should probably be run
>> through multiple real implementations and still would need disclaimers
>> saying "there are probably errors in the examples, so take a look at the
>> errata, etc.", but that's a whole other story..
>
> The way I managed the examples in my recently published RFC[1] is twofold:
>
> 1. I refused to add examples to explain something until the normative text
> itself was clear enough to in fact make the examples redundant.  Examples are
> fine for people that are just interested in vaguely understanding what it is
> about but implementers should not look at examples, or at any section titled
> "Introduction", "Overview" or "Informative References".  I wish I could somehow
> filter this sections automatically when my developers upload RFC from the Internet.

I agree that implementers should not rely on examples, but they can make 
understanding the specs much easier too and can be therefore really 
useful. That said, given how easily errors creep into examples, I'm 
usually a bit reluctant on having really specific examples in something 
as immutable as an RFC.
  > 2. I generated automatically the examples in the spec.  I first wrote a
> reference implementation[2] of the RFC.  Then I added unit tests to verify that
> my reference implementation was correct.  Then finally I wrote the code listed
> in [3] that automatically generates the RFC2629 fragments listed in [4],
> fragments that are automatically merged in the draft using Xinclude.  Because
> the examples were generated each time I built the text version of the draft, I
> was sure that the examples were correct - or at least as correct as my reference
> implementation.

Nice tool chain you got! Something like that would be good also for the 
examples of ICE TCP if we end up having such in the nat-scenarios draft.


Cheers,
Ari

> [1] https://datatracker.ietf.org/doc/rfc5928/
> [2] http://debian.implementers.org/stable/source/turnuri.tar.gz
> [3]
> example("turn:example.net", Arrays.asList(TLS, TCP, UDP), new MockDirContext(
>   domain("example.net.",
>     record(NAPTR,
>       "100 10 \"\" RELAY:turn.udp \"\" datagram.example.net.",
>       "200 10 \"\" RELAY:turn.tcp:turn.tls \"\" stream.example.net.")),
>   domain("datagram.example.net.",
>     record(NAPTR, "100 10 S RELAY:turn.udp \"\" _turn._udp.example.net.")),
>   domain("stream.example.net.",
>     record(NAPTR,
>       "100 10 S RELAY:turn.tcp \"\" _turn._tcp.example.net.",
>       "200 10 A RELAY:turn.tls \"\" a.example.net.")),
>   domain("_turn._udp.example.net.",
>     record(SRV, "0 0 3478 a.example.net.")),
>   domain("_turn._tcp.example.net.",
>     record(SRV, "0 0 5000 a.example.net.")),
>   domain("a.example.net.",
>     record(A, "192.0.2.1"))), 1, true);
> example("turn:example.com", Arrays.asList(TLS, TCP, UDP), new MockDirContext(
>   domain("example.com.",
>     record(NAPTR,
>       "100 10 \"\" RELAY:turn.udp:turn.tcp:turn.tls \"\" example.net."))), 2,
> false);
> example("turn:example.net", Arrays.asList(TLS, TCP, UDP), new MockDirContext(
>   domain("_turn._udp.example.com.",
>     record(SRV, "0 0 3478 a.example.net.")),
>   domain("_turn._tcp.example.com.",
>     record(SRV, "0 0 5000 a.example.net.")),
>   domain("_turns._tcp.example.com.",
>     record(SRV, "0 0 5349 a.example.net."))), 3, false);
>
> [4]
> <figure anchor="dns-rr1"><artwork><![CDATA[example.net.
> IN NAPTR 100 10 "" RELAY:turn.udp "" datagram.example.net.
> IN NAPTR 200 10 "" RELAY:turn.tcp:turn.tls "" stream.example.net.
>
> datagram.example.net.
> IN NAPTR 100 10 S RELAY:turn.udp "" _turn._udp.example.net.
>
> stream.example.net.
> IN NAPTR 100 10 S RELAY:turn.tcp "" _turn._tcp.example.net.
> IN NAPTR 200 10 A RELAY:turn.tls "" a.example.net.
>
> _turn._udp.example.net.
> IN SRV   0 0 3478 a.example.net.
>
> _turn._tcp.example.net.
> IN SRV   0 0 5000 a.example.net.
>
> a.example.net.
> IN A     192.0.2.1
>
> ]]></artwork></figure>
>
> <figure anchor="dns-rr2"><artwork><![CDATA[example.com.
> IN NAPTR 100 10 "" RELAY:turn.udp:turn.tcp:turn.tls "" example.net.
> ]]></artwork></figure>
>
> <figure anchor="dns-rr3"><artwork><![CDATA[_turn._udp.example.com.
> IN SRV   0 0 3478 a.example.net.
>
> _turn._tcp.example.com.
> IN SRV   0 0 5000 a.example.net.
>
> _turns._tcp.example.com.
> IN SRV   0 0 5349 a.example.net.
>
> ]]></artwork></figure>
>
> <texttable anchor="result1"><ttcol>Order</ttcol><ttcol>Protocol</ttcol><ttcol>IP
> address</ttcol><ttcol>Port</ttcol><c>1</c><c>UDP</c><c>192.0.2.1</c><c>3478</c><c>2</c><c>TLS</c><c>192.0.2.1</c><c>5349</c><c>3</c><c>TCP</c><c>192.0.2.1</c><c>5000</c></texttable>