Re: [tap] RFC Status?

Leon Timmermans <> Sat, 20 September 2014 22:10 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 763DC1A02BA for <>; Sat, 20 Sep 2014 15:10:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: 0.701
X-Spam-Status: No, score=0.701 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id xRjHzuMhNmJG for <>; Sat, 20 Sep 2014 15:10:11 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4001:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id B56E31A02D0 for <>; Sat, 20 Sep 2014 15:10:09 -0700 (PDT)
Received: by with SMTP id tp5so652329ieb.24 for <>; Sat, 20 Sep 2014 15:10:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=nUsCAzfWdDCevaXlrkL0dqh2QDWZEBDuxSjMIriypeE=; b=KmTzbRAjcmnV6lRdR+PvFxYoh4egpQLbV3lMFiC3o8Vd37wn2iHXoC8dwt9T0gRv6s bhuu2LkYYQxKN89v4qaIBrLTRMoMV3GW8DptUwCXFXqb+7Yfe8yn9KZqoVw6pl7etxD1 BAx/a0Dj9kcTIwCHhchRPlpdvRP3Yv3jQ31lhS2iRrGPvxo65pVU9IiJfsQ2o43t21Fe WLrJUyIR3bmcgxyofeuKqha99KwRhFmhDgTB38gN4zd9FLVpd5NSMRioggzyRzGMUqHj 5jhTYCzp9tbEN1CDgI/EoG3d7/Lmc361VuQVPy30YSI9RFaolM2YyiCy1zucw5dE4OvF xosg==
X-Received: by with SMTP id v18mr5079877igr.21.1411251009048; Sat, 20 Sep 2014 15:10:09 -0700 (PDT)
MIME-Version: 1.0
Received: by with HTTP; Sat, 20 Sep 2014 15:09:48 -0700 (PDT)
In-Reply-To: <>
References: <> <> <>
From: Leon Timmermans <>
Date: Sun, 21 Sep 2014 00:09:48 +0200
Message-ID: <>
To: Ovid <>
Content-Type: multipart/alternative; boundary=047d7bdc085415619b0503867b4e
Cc: "" <>
Subject: Re: [tap] RFC Status?
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Test Anything Protocol WG discussions <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 20 Sep 2014 22:10:13 -0000

On Wed, Sep 17, 2014 at 9:08 PM, Ovid <> wrote:

> A few notes on this.
> For subtests, they are indented four spaces. Always four spaces. If they
> are indented eight spaces, you know they're a subtest of a subtest
> (assuming there's an intervening subtest. That means this is valid TAP, but
> there is no subtest because it's indented too far:
> That is very useful to know, makes life for the consumer much easier.

> ok 1 - foo
>         ok 1 - bar indented too far
> ok 2 - bar passed
> The "ok 1 - bar" line would be discarded as "unknown".

I think I agree, though one could argue that's an unknown subline of a

> So if, in a stream of TAP, you find something which would be valid tap if
> it were unindented by four spaces, it's a subtest. The trailing,
> unindented summary line must mirror the status of the subtest. Thus, the
> following is invalid:
> ok 1 - foo
>     not ok 1 - bar indented too far
> ok 2 - bar passed
> Currently, Test::Harness <>
> treats the above as passing when it should not. I believe Schwern filed a
> bug for that. I know Andy Armstrong and I both took a quick swing at
> extending Test::Harness but discovered that it was more difficult than it
> appeared.

I'm currently giving a parse error in that condition :-)

As for the YAMLish structured diagnostics, those were never official, as
> far as I recall. There were several disputes, one of which was whether or
> not we should use YAML or JSON (I recall even XML being mentioned). Some
> argued that any structured diagnostic should be allowed, but I reject that
> firmly because that makes life hell for the parser. If you asked me what my
> stance is today, I would think that structured diagnostics should valid
> JSON, but that raises questions about whitespace issues, UTF-8, and so on.
> If we could agree on the format, then we'd need to agree on the keys in the
> structured diagnostic dictionary (and how to extend them).

Deciding on the keys is bikeshedding, deciding on the format decidedly
isn't. To be honest I'm not even sure what would be good or bad at this
stage, though I can agree with not accepting every format and their cousin.