[tap] Weird thought for the day: "skip 71 - reason"

Ovid <publiustemp-tapx@yahoo.com> Fri, 05 March 2010 15:53 UTC

Return-Path: <publiustemp-tapx@yahoo.com>
X-Original-To: tap@core3.amsl.com
Delivered-To: tap@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3D5583A8FE0 for <tap@core3.amsl.com>; Fri, 5 Mar 2010 07:53:35 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.185
X-Spam-Level:
X-Spam-Status: No, score=-0.185 tagged_above=-999 required=5 tests=[BAYES_40=-0.185]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ndSsFqdaDe2A for <tap@core3.amsl.com>; Fri, 5 Mar 2010 07:53:33 -0800 (PST)
Received: from n2d.bullet.mail.ac4.yahoo.com (n2d.bullet.mail.ac4.yahoo.com [76.13.13.86]) by core3.amsl.com (Postfix) with SMTP id AEA633A8FCE for <tap@ietf.org>; Fri, 5 Mar 2010 07:53:33 -0800 (PST)
Received: from [74.6.228.95] by n2.bullet.mail.ac4.yahoo.com with NNFMP; 05 Mar 2010 15:53:33 -0000
Received: from [76.13.10.182] by t2.bullet.mail.ac4.yahoo.com with NNFMP; 05 Mar 2010 15:53:33 -0000
Received: from [127.0.0.1] by omp123.mail.ac4.yahoo.com with NNFMP; 05 Mar 2010 15:53:33 -0000
X-Yahoo-Newman-Property: ymail-3
X-Yahoo-Newman-Id: 541384.95312.bm@omp123.mail.ac4.yahoo.com
Received: (qmail 95854 invoked by uid 60001); 5 Mar 2010 15:53:33 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1267804413; bh=PCMuGoBojVoCyrgWd95p+WV14w4sqKP39kC7yzvjWXg=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=mky7KtM9eMJ250SEo0RGDpgg3r18uZqHuFdOxoZ5xS7OzKKRQmFOhrOYETNuOZoYe0pWBF8IoZ/ZsRijXjBeyDEPCHR4SRZvCtdGQqGor6v+PmW7bD8wLukton6wjypwflVIfWrRPLJDprO3fHsX64Ho/gIgmWTeNQ99KfdQvOI=
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=AE9AlNDIoSuyR5BmNFMTdyQdkpxh59XPQdSuZnGkpBMd+3TZ3mIEFKrsPo73rA1cfdFBZXXRWW6Xr+Qbng58vMk4CkCPFKbHBo/MEO5Ps+86UCGETmuA8LgihenGnrXxAz5U8LxK/VDONlVHpHqo0IW6j54SOoaLdm3ti99lh0Q=;
Message-ID: <411886.95806.qm@web65707.mail.ac4.yahoo.com>
X-YMail-OSG: NDUoYmoVM1l_XnJqOzlIcGdf2BoiEiwoWqcyaGMq.QggS9T 7AqUE2ptuEy2Gdfudi47wEh06B4MPdRpi_QEQu8tEuylyiyiACtMzzX4n00E jElHzXa1r29LyWhVZ5D3Oqnn_eZj9RQpMr8YeZMnWbJwLH_B5ae9_RfsV2JQ IrRim9_9IrLXZDBicLl0inodl8hpMAmrMQWGeI570RADaBlRIhUrjMy4GE2G G8owPSM8ukbC2_UHSPskzI8aPTnJduQCI9G7_lRznuXKDys1UFEK7f6oHafe JmaInstM7fkPy5iLFUwOZyYkbRX5nKCCgZvlr0t25H1R1rQuFq3HleIhU5Du X4g--
Received: from [132.185.144.122] by web65707.mail.ac4.yahoo.com via HTTP; Fri, 05 Mar 2010 07:53:33 PST
X-Mailer: YahooMailRC/300.3 YahooMailWebService/0.8.100.260964
Date: Fri, 05 Mar 2010 07:53:33 -0800
From: Ovid <publiustemp-tapx@yahoo.com>
To: tap@ietf.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Subject: [tap] Weird thought for the day: "skip 71 - reason"
X-BeenThere: tap@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Test Anything Protocol WG discussions <tap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tap>
List-Post: <mailto:tap@ietf.org>
List-Help: <mailto:tap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 05 Mar 2010 15:53:35 -0000

One thing which bugs me about TAP is this:

  1..5
  ok 1 - some test
  ok 2 - another test
  ok 3 # skip cuz I said so
  not ok 4 # skip, skip, baby
  ok 5 -final test
 
What does a skipped "not ok" test mean? Its behaviour is undefined. This means that different parsers might implement this in different ways.  It's an obscure edge case, but with greater adoption of TAP, it might crop up.

Here's what I would like to see.

  1..5
  ok 1 - some test
  ok 2 - another test
  skip 3 - skip cuz I said so
  skip 4 - skip, skip, baby
  ok 5 -final test


In other words, you can't introduce ambiguity here.

Though this list is for ietf, those who really care about TAP are hopefully reading this.

Thoughts?  Something corresponding for TODO (less needed due to less ambiguity)?

Cheers,
Ovid--
Buy the book         - http://www.oreilly.com/catalog/perlhks/
Tech blog            - http://use.perl.org/~Ovid/journal/
Twitter              - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6