Re: [tap] interpretation of results
Michael G Schwern <schwern@pobox.com> Wed, 04 March 2009 21:15 UTC
Return-Path: <schwern@pobox.com>
X-Original-To: tap@core3.amsl.com
Delivered-To: tap@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id EFE5A3A6D1F for <tap@core3.amsl.com>; Wed, 4 Mar 2009 13:15:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FP6iylU4S4EQ for <tap@core3.amsl.com>; Wed, 4 Mar 2009 13:15:25 -0800 (PST)
Received: from sasl.smtp.pobox.com (a-sasl-fastnet.sasl.smtp.pobox.com [207.106.133.19]) by core3.amsl.com (Postfix) with ESMTP id BFDE93A6874 for <tap@ietf.org>; Wed, 4 Mar 2009 13:15:24 -0800 (PST)
Received: from localhost.localdomain (unknown [127.0.0.1]) by a-sasl-fastnet.sasl.smtp.pobox.com (Postfix) with ESMTP id E28509FF6E; Wed, 4 Mar 2009 16:15:51 -0500 (EST)
Received: from [10.23.42.2] (unknown [69.64.236.3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by a-sasl-fastnet.sasl.smtp.pobox.com (Postfix) with ESMTPSA id D03C49FF6B; Wed, 4 Mar 2009 16:15:46 -0500 (EST)
Message-ID: <49AEEF7C.10609@pobox.com>
Date: Wed, 04 Mar 2009 13:15:40 -0800
From: Michael G Schwern <schwern@pobox.com>
User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209)
MIME-Version: 1.0
To: Andy Armstrong <andy@hexten.net>
References: <49AE9EAA.60508@plusthree.com> <562033F9-5E53-4269-9DA1-A51F013D2D63@hexten.net>
In-Reply-To: <562033F9-5E53-4269-9DA1-A51F013D2D63@hexten.net>
X-Enigmail-Version: 0.95.7
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
X-Pobox-Relay-ID: A4041DC8-0901-11DE-9A25-CFA5EBB1AA3C-02258300!a-sasl-fastnet.pobox.com
Cc: TAP <tap@ietf.org>
Subject: Re: [tap] interpretation of results
X-BeenThere: tap@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Test Anything Protocol WG discussions <tap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tap>
List-Post: <mailto:tap@ietf.org>
List-Help: <mailto:tap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 04 Mar 2009 21:15:26 -0000
Andy Armstrong wrote: > On 4 Mar 2009, at 15:30, Michael Peters wrote: > >> Given this TAP: >> >> 1..3 >> ok 1 >> not ok 2 >> >> Does it mean that 1 of 2 tests failed and we had a bad plan? Or does >> it mean the plan was right, but only 1 of 3 tests passed. Or in other >> words, is there a 3rd implicit failed test in there? > > We planned three, ran two and one of them failed. I think that treating > the missing test as a fail would be something you'd only do if you > wanted to distill the results down to a simple failure count. Agreed. It's important to distinguish between a test failing and a test never having run at all. -- Insulting our readers is part of our business model. http://somethingpositive.net/sp07122005.shtml
- [tap] interpretation of results Michael Peters
- Re: [tap] interpretation of results Andy Armstrong
- Re: [tap] interpretation of results Michael G Schwern
- Re: [tap] interpretation of results Ovid