A proposal for a scientific approach to this question [was Re: I'm struggling with 2219 language again]
Marc Petit-Huguenin <petithug@acm.org> Sat, 05 January 2013 19:13 UTC
Return-Path: <petithug@acm.org>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 69A2E21F8554 for <ietf@ietfa.amsl.com>; Sat, 5 Jan 2013 11:13:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -101.308
X-Spam-Level:
X-Spam-Status: No, score=-101.308 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, MISSING_HEADERS=1.292, NO_RELAYS=-0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BT4TAUW3Cn-l for <ietf@ietfa.amsl.com>; Sat, 5 Jan 2013 11:13:55 -0800 (PST)
Received: from implementers.org (implementers.org [IPv6:2604:3400:dc1:41:216:3eff:fe5b:8240]) by ietfa.amsl.com (Postfix) with ESMTP id 279F221F8546 for <ietf@ietf.org>; Sat, 5 Jan 2013 11:13:55 -0800 (PST)
Received: from [IPv6:2601:9:4b80:32:5de6:df9d:4864:4f45] (unknown [IPv6:2601:9:4b80:32:5de6:df9d:4864:4f45]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client CN "Marc Petit-Huguenin", Issuer "implementers.org" (verified OK)) by implementers.org (Postfix) with ESMTPS id 5A0262049D for <ietf@ietf.org>; Sat, 5 Jan 2013 19:13:53 +0000 (UTC)
Message-ID: <50E87B7B.5070605@acm.org>
Date: Sat, 05 Jan 2013 11:14:03 -0800
From: Marc Petit-Huguenin <petithug@acm.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.11) Gecko/20121122 Icedove/10.0.11
MIME-Version: 1.0
CC: ietf@ietf.org
Subject: A proposal for a scientific approach to this question [was Re: I'm struggling with 2219 language again]
References: <7ED55FF1-3E1A-4DF7-918E-07790517B848@softarmor.com>
In-Reply-To: <7ED55FF1-3E1A-4DF7-918E-07790517B848@softarmor.com>
X-Enigmail-Version: 1.4.1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 05 Jan 2013 19:13:56 -0000
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I read the responses so far, and what can be said today is that there is 2 philosophies, with supporters in both camps. The goal of the IETF is to make the Internet work better, and I do believe that RFC 2119 is one of the fundamental tool to reach this goal, but having two ways to use it does not help this goal. One way to find out would be to measure which philosophy results in the best implementations. Let's say that we can associate with each Standard Track RFC one of these two philosophy. If we had statistics on implementations then it would be a simple matter of counting which one produce the less interoperability problems, security issues and congestion problems (is there other criteria?). But as far as I know, there is no such data available - maybe we should start collecting these, but that does not help for our current problem. Another way to look at it would be to run the following experiment: 1. Someone design a new protocol, something simple but not obvious, and write in a formal language and keep it secret. 2. The same protocol is rewritten in RFC language but in two different variants according to the two philosophies. These also are kept secret. 3. The two variants are distributed randomly to a set of volunteer implementers, who all implement the spec they received the best they can and submit the result back, keeping their implementation secret. 4. A test harness is written from the formal description, and all implementations are run against each other, collecting stats related to the criteria listed above (some criterion may be tricky to automatically assess, we'll see). 5. Results are published, together with the protocol in formal form, the specs, the results and the recommendation for one or the other philosophy. That could be an interesting research project, and could even find some funding from interested parties. On 01/03/2013 09:15 PM, Dean Willis wrote: > > I've always held to the idea that RFC 2119 language is for defining levels > of compliance to requirements, and is best used very sparingly (as > recommended in RFC 2119 itself). To me, RFC 2119 language doesn't make > behavior normative -- rather, it describes the implications of doing > something different than the defined behavior, from "will break the > protocol if you change it" to "we have reason to think that there might be > a reason we don't want to describe here that might influence you not to do > this" to "here are some reasons that would cause you to do something > different" and on to "doing something different might offend the > sensibilities of the protocol author, but probably won't hurt anything > else." > > But I'm ghost-editing a document right now whose gen-art review suggested > replacing the vast majority of "is" "does" and "are" prose with MUST. The > comments seem to indicate that protocol-defining text not using RFC 2119 > language (specifically MUST) is "not normative". > > This makes me cringe. But my co-editor likes it a lot. And I see smart > people like Ole also echoing the though that RFC 2119 language is what > makes text normative. > > For example, the protocol under discussion uses TLS or DTLS for a plethora > of security reasons. So, every time the draft discusses sending a response > to a request, we would say "the node MUST send a response, and this > response MUST be constructed by (insert some concatenation procedure here) > and MUST be transmitted using TLS or DTLS. > > Or, a more specific example: > > For the text: > > In order to originate a message to a given Node-ID or Resource-ID, a node > constructs an appropriate destination list. > > > The Gen-ART comment here is: -First sentence: "a node constructs" -> "a > node MUST construct" > > > We'll literally end up with hundreds of RFC 2119 invocations (mostly MUST) > in a protocol specification. > > Is this a good or bad thing? My co-editor and I disagree -- he likes > formalization of the description language, and I like the English prose. > But it raises process questions for the IETF as a whole: > > Are we deliberately evolving our language to use RFC 2119 terms as the > principle verbs of a formal specification language? > > Either way, I'd like to see some consensus. Because my head is throbbing > and I want to know if it MUST hurt, SHOULD hurst, or just hurts. But I > MUST proceed in accordance with consensus, because to do otherwise would > undermine the clarity of our entire specification family. > - -- Marc Petit-Huguenin Email: marc@petit-huguenin.org Blog: http://blog.marc.petit-huguenin.org Profile: http://www.linkedin.com/in/petithug -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCAAGBQJQ6Ht3AAoJECnERZXWan7ETHwP/1MwWKyX4ZoTqS2AZr5VdCwx jGO/0+tbHppplfippPlJRR6cV5rfrrtkKp9j3Xbr477Jeuaaadjv3y0CfkGF+DUb fDhcB/GQLiN1oC6s3cjiib46Rnd18Ela6xUAZleiLjKKoo0TJKfQ8oAt3tYonokK onb95NAsF0FsbiqBzoUi23aEf/SFoKOg3a67DAt5XmntnNh5K6jVOmT4GFYtF3LB vW6d1x0hI0INUSXzjypD+MaqzCgHvdxAjqx44qlrFjFYz7GLcRAR3Z5ynOCCfeBi fM4xxGhytTYolrXdOQK1BgtUIewdCHMqPFZjclB2DiEITkUzxRrGKI5MizQEModB 8vwmJQJ0E98veMvBP5ce9eKPZP0gHMxEWMu2zgGulb2mLVxyfSfBIy2dIuqpHsUM dWuvLzx1HJouZ4sFfCGMarpLvoYwYu1grL+oaJiPXd1TI26BCyI703gdQE1dBRhK XrfIEgmP1VG1QhhBA6otLQlWfB0IGG2c80y0KOZ4x9rQ8Wn+Cxz4vQg6CvHPys+z ClvFH4r/v3CpavSpugcD7sJAssfp9wbe9Jjw1pCPiXgs0fN+yQtQRIVDCSCQHNJ4 iWFtwExIDQyh/lKmJNlf8P3qBSmhzqvAG1B9bAZY9JroJoqY84kkgWKHRodAQ/HM DqC0PVCig1bKYwZpT1Ov =oOlZ -----END PGP SIGNATURE-----
- I'm struggling with 2219 language again Dean Willis
- Re: I'm struggling with 2219 language again Brian E Carpenter
- Re: I'm struggling with 2219 language again Dave Cridland
- Re: I'm struggling with 2219 language again Lou Berger
- Re: I'm struggling with 2219 language again Bob Braden
- Re: I'm struggling with 2219 language again Scott Brim
- Re: I'm struggling with 2219 language again Peter Saint-Andre
- Re: I'm struggling with 2219 language again Hector Santos
- Re: I'm struggling with 2219 language again Richard Barnes
- RE: I'm struggling with 2219 language again Adrian Farrel
- Re: I'm struggling with 2219 language again Ben Campbell
- Re: I'm struggling with 2219 language again Thomas Narten
- Re: I'm struggling with 2219 language again Mark Nottingham
- Re: I'm struggling with 2219 language again ned+ietf
- Re: I'm struggling with 2219 language again Hector Santos
- A proposal for a scientific approach to this ques… Marc Petit-Huguenin
- Re: I'm struggling with 2219 language again Pete Resnick
- Re: I'm struggling with 2219 language again ned+ietf
- Re: I'm struggling with 2219 language again Dean Willis
- Re: I'm struggling with 2219 language again Riccardo Bernardini
- Re: I'm struggling with 2219 language again Scott Brim
- Re: I'm struggling with 2219 language again Marc Petit-Huguenin
- Re: I'm struggling with 2219 language again C. M. Heard
- Compliance to a protocol description? (wasRE: I'm… Robin Uyeshiro
- Re: I'm struggling with 2219 language again John Levine
- Re: I'm struggling with 2219 language again John Day
- Re: A proposal for a scientific approach to this … John Day
- Re: A proposal for a scientific approach to this … Dick Franks
- Re: A proposal for a scientific approach to this … John Day
- Re: A proposal for a scientific approach to this … Donald Eastlake
- Re: A proposal for a scientific approach to this … Martin Rex
- Re: A proposal for a scientific approach to this … Marc Petit-Huguenin
- Re: A proposal for a scientific approach to this … John Day
- Re: A proposal for a scientific approach to this … Dick Franks
- Re: A proposal for a scientific approach to this … John Day
- Re: A proposal for a scientific approach to this … Martin Rex
- Re: A proposal for a scientific approach to this … Martin Rex
- Re: A proposal for a scientific approach to this … Hector Santos