Re: [tap] Valids subtests (was: RFC Status?)

"Bruno P. Kinoshita" <brunodepaulak@yahoo.com.br> Wed, 17 September 2014 20:38 UTC

Return-Path: <brunodepaulak@yahoo.com.br>
X-Original-To: tap@ietfa.amsl.com
Delivered-To: tap@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 828681A6EF4 for <tap@ietfa.amsl.com>; Wed, 17 Sep 2014 13:38:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.75
X-Spam-Level:
X-Spam-Status: No, score=-0.75 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, J_CHICKENPOX_31=0.6, RCVD_IN_DNSWL_NONE=-0.0001, REPTO_QUOTE_YAHOO=0.646, URIBL_DBL_ABUSE_REDIR=0.001, URIBL_DBL_REDIR=0.001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KCvnvt9H2m0Z for <tap@ietfa.amsl.com>; Wed, 17 Sep 2014 13:38:10 -0700 (PDT)
Received: from nm8-vm2.bullet.mail.gq1.yahoo.com (nm8-vm2.bullet.mail.gq1.yahoo.com [98.136.218.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 237CB1A6EEC for <tap@ietf.org>; Wed, 17 Sep 2014 13:38:10 -0700 (PDT)
Received: from [98.137.12.188] by nm8.bullet.mail.gq1.yahoo.com with NNFMP; 17 Sep 2014 20:38:09 -0000
Received: from [98.137.12.215] by tm9.bullet.mail.gq1.yahoo.com with NNFMP; 17 Sep 2014 20:38:09 -0000
Received: from [127.0.0.1] by omp1023.mail.gq1.yahoo.com with NNFMP; 17 Sep 2014 20:38:09 -0000
X-Yahoo-Newman-Property: ymail-3
X-Yahoo-Newman-Id: 822879.25256.bm@omp1023.mail.gq1.yahoo.com
Received: (qmail 18800 invoked by uid 60001); 17 Sep 2014 20:38:09 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com.br; s=s1024; t=1410986289; bh=zJxjMxsMqQ/M+LMuIQ6nnovvJ7ivzYa3mX8cOB+HeHI=; h=References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=RH7cW+W1AHO1nxhj/YudGR3CuBn5A2zSkkjpO3ECm5M6qmdvO5KnqJbEXZyDk0W7ZhdPXEh4XQnEKefD8TziRStfc52Gi7+xpAdpD/QCFbggVNFqQhFBhJTmLtwsusOFJkTQgthHpA0ZdsupIe3vgVDxY0L0fYbXg3AcV8WiUUc=
X-YMail-OSG: QfILNFoVM1kF5PiW.5eCg_72ibZARqvZ6JoyO3aZsaDZk7o aBAhmHMYqvmvVwJqr9LdwgTEIywehMbUrYbocm49Sd7rYT4TRqsqx5zZIAxk p6MhdnXAdXmeWEkOj1sG._0rSThwNjfgJWdygIFz9gD3mScWBDqudWtmWpnY lr_BS4y2YWPzXIPgBd.5xTzwLVM7mv.sto5ZI.5ifjrXg.sjrcsAe.AYajxo frjlibgl1mVeYNyizebahCeMgvxMKzxgCImUSAqvoqc9xIbobM0c7ip0TI0v SVSZ8r5wgokPyRfA6jFOBc2uoxFvuELR9JjPOPU515BKFSBNanwSzcamrIo0 byy6iWvWerBU1Y2CB_.t2cwXlBAoQHWDSYgsXXiIGQnDDPBGJV0UtTrhC52p xWqZ4J9boHcA3YTJy.wieH2svXJBfHPOguP9YDH8yEfhe07egSZIo2v_r_oS h3jZ4bwkVkUz4Bpi4QDkiMoE2.pw9QDsyWQOdaqH7COToJm1faqI4kaa4OfV VbcqWDpp.Hlv7.y2t1fktioUoT_xNgAn0RhHUUfj6ZQFYzUdv2p6owH.EVDI RUsr1OtfujYQ0vRT9LakznfuwDnksK0UqB8SaNWcm8dC_0.bcU7.ew09dY4T krS12UQEIcTnzP2uiOUOyN3p6hWU1gROol67_g3TmwDB2J3q1xrOEpgYPAoq KkshJX7CZouomg1fcJ4o8l_immr7UKwlRTWC682r5Tl4gUpYOlXY5rYXG3iV mcwcGDurY0flAB3tNUG7AdfvwecVXWDTFkQ--
Received: from [189.50.161.77] by web163505.mail.gq1.yahoo.com via HTTP; Wed, 17 Sep 2014 13:38:09 PDT
X-Rocket-MIMEInfo: 002.001, SGFoYSwgdGhhbmtzIE92aWQhCgpXZWxsLCB0dXJucyBvdXQgdGFwNGogVEFQIHBhcnNlciBpcyBsaWtlbHkgYnJva2VuIGZvciBzdWJ0ZXN0cyB0aGVuIDopCgpJJ2xsIHByb2JhYmx5IHNwZW5kIHNvbWUgdGltZSB3cml0aW5nIGEgbmV3IHBhcnNlciAtIHdoaWNoIEknbGwgdHJ5IHRvIG1ha2UgbW9yZSBjb21wYXRpYmxlIHdpdGggUGVybCB0b29scyBhbmQgd2l0aCBleGlzdGluZyBncmFtbWFycy4gCgpCdXQgc29tZSBkZWZpbml0aXZlIGRvY3VtZW50YXRpb24gZm9yIHN1YnRlc3RzIGluIFRBUCB3ZWJzaXQBMAEBAQE-
X-Mailer: YahooMailWebService/0.8.203.696
References: <CAP4gcszybVr5Hw3mg=uTi8tqpA3wEVwo=zf2876RWhy_CmozZw@mail.gmail.com> <CAHhgV8jr6ZnsfUkpFC4OL0AwRX-aen7v-7KjcN3e0_19s7steg@mail.gmail.com> <1410980929.82809.YahooMailNeo@web126106.mail.ne1.yahoo.com> <1410981416.81953.YahooMailNeo@web163504.mail.gq1.yahoo.com> <1410982831.46047.YahooMailNeo@web126105.mail.ne1.yahoo.com> <1410984733.67023.YahooMailNeo@web163504.mail.gq1.yahoo.com> <1410985980.76329.YahooMailNeo@web126106.mail.ne1.yahoo.com>
Message-ID: <1410986289.81577.YahooMailNeo@web163505.mail.gq1.yahoo.com>
Date: Wed, 17 Sep 2014 13:38:09 -0700
From: "Bruno P. Kinoshita" <brunodepaulak@yahoo.com.br>
To: Ovid <publiustemp-tapx@yahoo.com>
In-Reply-To: <1410985980.76329.YahooMailNeo@web126106.mail.ne1.yahoo.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="-1735974723-973131396-1410986289=:81577"
Archived-At: http://mailarchive.ietf.org/arch/msg/tap/0Is8icPUhAle4OXZdRO-_uI09SY
Cc: "tap@ietf.org" <tap@ietf.org>
Subject: Re: [tap] Valids subtests (was: RFC Status?)
X-BeenThere: tap@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: "Bruno P. Kinoshita" <brunodepaulak@yahoo.com.br>
List-Id: Test Anything Protocol WG discussions <tap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tap>, <mailto:tap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tap/>
List-Post: <mailto:tap@ietf.org>
List-Help: <mailto:tap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Sep 2014 20:38:11 -0000

Haha, thanks Ovid!

Well, turns out tap4j TAP parser is likely broken for subtests then :)

I'll probably spend some time writing a new parser - which I'll try to make more compatible with Perl tools and with existing grammars. 

But some definitive documentation for subtests in TAP website would be grand too.

Cheers
Bruno


>________________________________
> From: Ovid <publiustemp-tapx@yahoo.com>
>To: Bruno P. Kinoshita <brunodepaulak@yahoo.com.br> 
>Cc: "tap@ietf.org" <tap@ietf.org> 
>Sent: Wednesday, September 17, 2014 5:33 PM
>Subject: Re: [tap] Valids subtests (was:  RFC Status?)
> 
>
>
>Bruno,
>
>
>I realized that referring you to the grammar is the same as RTFM and that's not fair when there are questions about the FM :)
>
>
>    1..2
>    ok 1 - foo
>        not ok 1 - some text
>    ok 2 - bar passed
>        not ok 1 - another text
>
>
>The above is not valid because there must always be summary ok/not ok lines *after* the subtest and the subtest must have a leading XOR trailing plan (in other words, not both) with the same indentation level as the subtest and the summary line *must* have an ok/not ok status that matches the results of the subtest. The following would be valid:
>
>
>    1..2
>        not ok 1 - some text
>        1..1
>    not ok 1 - foo (because the subtest did not pass)
>        1..2
>        ok 1 - another text
>        not ok 2 - whee! # TODO (todo tests are always considered as passing)
>    ok 2 - bar passed (because the subtest did pass)
>
>
>
>
>
>
>
>Best,
>Ovid 
>
>--
>IT consulting, training, international recruiting
>       http://www.allaroundtheworld.fr/.
>Buy my book! - http://bit.ly/beginning_perl
>Live and work overseas - http://www.overseas-exile.com/
>
>