[apps-discuss] Automated specification generation tools

Phillip Hallam-Baker <phill@hallambaker.com> Tue, 22 March 2016 19:48 UTC

Return-Path: <hallam@gmail.com>
X-Original-To: apps-discuss@ietfa.amsl.com
Delivered-To: apps-discuss@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id E5DE812D899 for <apps-discuss@ietfa.amsl.com>; Tue, 22 Mar 2016 12:48:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.35
X-Spam-Status: No, score=-2.35 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id HtxXk0rTA9vn for <apps-discuss@ietfa.amsl.com>; Tue, 22 Mar 2016 12:48:17 -0700 (PDT)
Received: from mail-lf0-x22e.google.com (mail-lf0-x22e.google.com [IPv6:2a00:1450:4010:c07::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9466C12D881 for <apps-discuss@ietf.org>; Tue, 22 Mar 2016 12:48:17 -0700 (PDT)
Received: by mail-lf0-x22e.google.com with SMTP id o73so4374475lfe.0 for <apps-discuss@ietf.org>; Tue, 22 Mar 2016 12:48:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to; bh=NpkGIKoy/ybgIqU9y5KCtkwhFDxQGqkJLQiORLKAAmU=; b=mtcDalgNDpIdYaOrgbJmtdm0fWq/aasZak2YOoCP/Nht2UjZIo/FEy4K8dmlHY/s9c P7cvw/rl1kK8IbE1B4kPilLiN5MePHLVIOK6Z29QUUnut2oOUCbBihdiveWxqnR7iuT0 XUP6yBZdqeDMrphpQzFRd8dvyJiOEKblFt8rKnR2IlpJV/vKfKM7dlgaNKipvy6UfnbX hMiRKZRwgFwsArQeueDUMf2+DU3m6ttoEq8619zKCRvG6QDGbJn1Pxg0iva7HjhY3Hd+ wG42YG8d5CADm/PjIH8qMfh33ntGNCZo8uytSOoQnYTTA7J20l3Dx3xj8VzA4rGFj5eo DiGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:date:message-id:subject:from :to; bh=NpkGIKoy/ybgIqU9y5KCtkwhFDxQGqkJLQiORLKAAmU=; b=d0Ue+RV9wpOeL6jfn1mmMBt2NkQjq8g295Ljt0U2OZ1CmINwdiaWdgxaN3jyHb1U9Z MqAnnW+bWUg4M9p5P2WRMovAuflbCQgwNOmzv+oLDk5PPM7dM8kAvXPa+kzj2bgR4T8e W4+FGTTrjPJvUrqj76v8TEmMOkKNFm8vvynmULs/LlUExbtEcHVlbUuKoqctuPo51uUL DAR5ROMfIvK0knIuDUsSw8ySUUFk1xBj1e1s5k+8P74XFnulgitwga6F4NNwjW2jTIVN 0043q2wOpMg6h8bbxJU6xkR9KMqv5zEDvgILRrPkRUjIOlXnd+7BinE8j4jqXkistEwO PfKw==
X-Gm-Message-State: AD7BkJKtdIEXfl1eZC5USIaaxcaFmDqZi6AiKqPu8HE44286ccJyidzRmILdoI4fs1Afqtl/n+APLzdn/LjGWQ==
MIME-Version: 1.0
X-Received: by with SMTP id p73mr1060781lfb.67.1458676095763; Tue, 22 Mar 2016 12:48:15 -0700 (PDT)
Sender: hallam@gmail.com
Received: by with HTTP; Tue, 22 Mar 2016 12:48:15 -0700 (PDT)
Date: Tue, 22 Mar 2016 15:48:15 -0400
X-Google-Sender-Auth: GUPsyNTu2M4thcLAt-jek1UiKWk
Message-ID: <CAMm+LwjDrYrDBz5zsVMhchm195akJ0tvy3-iPEHZkDJQZHUjyg@mail.gmail.com>
From: Phillip Hallam-Baker <phill@hallambaker.com>
To: General discussion of application-layer protocols <apps-discuss@ietf.org>
Content-Type: text/plain; charset=UTF-8
Archived-At: <http://mailarchive.ietf.org/arch/msg/apps-discuss/aPAvgrqCNk_pWH_kr_9X36dOXxk>
Subject: [apps-discuss] Automated specification generation tools
X-BeenThere: apps-discuss@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: General discussion of application-layer protocols <apps-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/apps-discuss/>
List-Post: <mailto:apps-discuss@ietf.org>
List-Help: <mailto:apps-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 22 Mar 2016 19:48:20 -0000

A while back I described a tool that I have been using as a code
generator for JSON based applications. I have extended the tool since
to the point where it almost (but not quite) writes the specification
for you.

The tool bundle itself is open source (MIT license, GitHub) and
available from here:


I am now working on some documentation and an example of a spec
produced using the tool chain. This is one of the specs I have


Normally when you write a specification, there is a huge amount of
back and forth as changes to the spec cause examples to be out of
sync. Or you change the text in one place but not another. And of
course it all gets worse if someone else writes stuff that has to be
merged in.

So what tends to happen is that the editor spends a lot of time
checking the spec for consistency after each change and then reviewers
do the same and then the spec goes out with some last minute change
and you end up with errata.

Alternatively, the spec is 100% right but someone's implementation is
wrong. And because that is the implementation everyone has to interop
with you end up with two specs, the one that was written and the
defacto one you have to implement.

The Protogen toolset is a suite of code building tools. They allow you
to build specifications, examples and reference code from a single
source. So any changes to that source automatically ripple through the
spec. And anyone who does interop testing against the reference
implementation can be fairly sure that they have done it right.

Right now the tool set is (mostly) targeted at writing JSON specs but
I have developed tools that I use with RFC 821 and RFC 822 style
encodings, TLS schema and ASN.1 as well. These are only intended to
support existing IETF specs though rather than develop more of the

So the way I developed the LURK spec above in three days (i.e 24 man
hours, not pulling three all nighters in a row) was that I started off
describing the protocol in Word, then I roughed out the set of
examples I wanted to produce to describe the spec, then I developed
the schema from those. The second day, I worked on the Word
documentation again, got the examples running with only request
messages being generated. The third day I implemented the server
dispatch function to give me the responses.

If you prefer, you can use Markdown instead of Word. I am just in the
middle of converting to Carsten's particular variant of the markup.

This isn't just a quicker and less error prone way to write specs. The
same tools generate C so you can integrate with existing

The resulting specifications are a lot better than those you get
writing by hand as well. Instead of different parts of the spec being
written in a variety of styles, there is one consistent style applied

The question now is what should that style be?