[tap] Question about directives and comments

Andrew Rodland <andrew@cleverdomain.org> Mon, 20 September 2010 01:02 UTC

Return-Path: <andrew@cleverdomain.org>
X-Original-To: tap@core3.amsl.com
Delivered-To: tap@core3.amsl.com
Received: from localhost (localhost []) by core3.amsl.com (Postfix) with ESMTP id D189C3A68C4 for <tap@core3.amsl.com>; Sun, 19 Sep 2010 18:02:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.001
X-Spam-Status: No, score=0.001 tagged_above=-999 required=5 tests=[BAYES_50=0.001]
Received: from mail.ietf.org ([]) by localhost (core3.amsl.com []) (amavisd-new, port 10024) with ESMTP id EEWFLZ16yVxH for <tap@core3.amsl.com>; Sun, 19 Sep 2010 18:02:19 -0700 (PDT)
Received: from sasl.smtp.pobox.com (b-pb-sasl-quonix.pobox.com []) by core3.amsl.com (Postfix) with ESMTP id B331D3A68CD for <tap@ietf.org>; Sun, 19 Sep 2010 18:02:18 -0700 (PDT)
Received: from sasl.smtp.pobox.com (unknown []) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id F1D0AC9C4E for <tap@ietf.org>; Sun, 19 Sep 2010 21:02:41 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to :subject:date:mime-version:content-type :content-transfer-encoding:message-id; s=sasl; bh=dRusE7LLAYawx4 urW7oS4mg5cag=; b=fVSUrHs3cYPg9zdgkchj8YmapqfaUNzLUF2JHOenc1pI4f WOR7TxcZKsqvMmL8mHmTM+tzcO/F8BB1aJzxccHnA1eKYf7UqehkZo5RjK2YLRGa oUpw9qtuhKJkAfn+YhMDLPljBuohyBrA0Fi+/rzsd7liR/s9UVhHolsNG0ifg=
Received: from b-pb-sasl-quonix. (unknown []) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id EEAE8C9C4D for <tap@ietf.org>; Sun, 19 Sep 2010 21:02:41 -0400 (EDT)
Received: from goron.localnet (unknown []) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by b-sasl-quonix.pobox.com (Postfix) with ESMTPSA id C639BC9C4C for <tap@ietf.org>; Sun, 19 Sep 2010 21:02:41 -0400 (EDT)
From: Andrew Rodland <andrew@cleverdomain.org>
To: tap@ietf.org
Date: Sun, 19 Sep 2010 20:02:40 -0500
User-Agent: KMail/1.13.5 (Linux/2.6.35-20-generic; KDE/4.5.1; x86_64; ; )
MIME-Version: 1.0
Content-Type: Text/Plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Message-Id: <201009192002.40817.andrew@cleverdomain.org>
X-Pobox-Relay-ID: C5371154-C452-11DF-BED3-A3DF016DD5F0-16769786!b-pb-sasl-quonix.pobox.com
Subject: [tap] Question about directives and comments
X-BeenThere: tap@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Test Anything Protocol WG discussions <tap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tap>
List-Post: <mailto:tap@ietf.org>
List-Help: <mailto:tap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Sep 2010 01:02:25 -0000

Following some recent improvements in Regexp::Grammars and in perl 5.13, I've 
been able to unstick progress on TAP::Spec::Parser, my attempt to turn the 
draft spec into live code suitable for running a test suite.

It's gotten to the point where I can throw real, complex TAP output at it and 
have it work, which means it's time for me to start asking questions. The one 
I've got so far -- the spec shows this BNF for a test result:

> Test-Result     = Status [SP Test-Number] [SP Description]
>                   [SP "#" SP Directive [SP Reason]] EOL

and Directive is

> Directive       = "SKIP" / "TODO"

It's already been brought up in a conversation between EBo and Ovid on tap-l 
that the SP between "#" and directive and between the directive and reason are 
optional, so I'll pass over that for now.

My question regards section 8.6 "Ignored Elements", which says "Any directive 
or plan-directive which is not recognized must also be ignored", and later 
"The directive 'BANG' on test #2 should be treated as an ordinary comment, it 
is not a tap directive."

So we have three different opinions here.

1. The grammar, as written, says that if there's a "#" on a test-result line, 
the word following it must match Directive (i.e. be either "SKIP" or "TODO"). 
If it's not, the entire line will be junked. This is almost certainly wrong.

2. The first statement in the "Ignored Elements" section implies that 
Directive should match any sequence of not-SP, and that the directive should 
simply have no effect if it's not one of "SKIP" or "TODO". This seems 

3. The second statement in the "Ignored Elements" section implies that an 
unrecognized directive should instead be parsed as a Comment until the end of 
line -- but we don't match Comments within Test-Result (or any TAP-Line), only 
Comments that take up the full line. Is this an omission in the grammar (do we 
*need* to handle line-ending Comments) or is it simply a thinko on the part of 
the person who wrote that statement?

Any comments are welcome.