Re: [tap] W3C Evaluation and Report Language (EARL)

Leon Timmermans <> Wed, 09 November 2011 20:50 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 889361F0C7F for <>; Wed, 9 Nov 2011 12:50:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.599
X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id yFSPJCNjgGIU for <>; Wed, 9 Nov 2011 12:50:02 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id E0C351F0C74 for <>; Wed, 9 Nov 2011 12:50:01 -0800 (PST)
Received: by vcbfk1 with SMTP id fk1so2045566vcb.31 for <>; Wed, 09 Nov 2011 12:50:01 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=4RZIA55Kb0QBESxeBvpCu/vEUH+L1Wy9S9mX2rFfVsk=; b=EQhRvKuAnGt1o9cesbTVcIMhtB3o6duwykb3SGh8oaQaMAcmctIUEirNulCf7ZUfcV 0QS4HqxdojJnLYLa0xQngC2ZeNKr5gkKIvHZdpezQnlGsKdVVpnATb10fjHq4/8B60CM 1dTj3jaM0jHURhUo9ZFHch1msLZK0pQsO8mNs=
Received: by with SMTP id bm19mr7107093vdc.115.1320871801123; Wed, 09 Nov 2011 12:50:01 -0800 (PST)
MIME-Version: 1.0
Received: by with HTTP; Wed, 9 Nov 2011 12:49:40 -0800 (PST)
In-Reply-To: <>
References: <> <>
From: Leon Timmermans <>
Date: Wed, 9 Nov 2011 21:49:40 +0100
Message-ID: <>
To: Steffen Schwigon <>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc:, Shadi Abou-Zahra <>
Subject: Re: [tap] W3C Evaluation and Report Language (EARL)
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Test Anything Protocol WG discussions <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 09 Nov 2011 20:50:02 -0000

On Wed, Nov 9, 2011 at 9:25 PM, Steffen Schwigon <> wrote:
> - EARL is similar to other W3C specs in respect to specifying a
>  comprehensive snapshot of known existing topics. For example, it
>  particularly covers all known HTTP methods (POST, GET, PUT, …). That
>  enables it to build tools on top of it that sematically “know” what
>  the document is about.
> - TAP in contrast is about specifying test results, really just the
>  *result* focus without hard specification of the tested topic, i.e., a
>  single test has a “description”, so someone reading it knows what it
>  is about but that part does not have a specification.
>  For instance, a test about a HTTP method could have any description
>  from “POST” to “that strange other method that I never remember but
>  always use when GET is not sufficient”.
> See [1] for some related discussion of this aspect.
> In this respect I think TAP is more like your RDF with some extensions
> from EARL to describe test success.
> That makes the use-cases of TAP and EARL a bit different:
> - TAP allows to be produced by anything simple without toolchain
>  support, like embedded devices with nothing but a “print” function,
>  but you can not *sematically* evaluate results.
> - EARL seems to require more heavy toolchain support to produce but
>  allows more semantic result evaluation.

I agree. To me, EARL makes sense in a context that heavy into semantic
web and triplestores, but the toolchain would be too heavy for a lot
of usecases.

> Converting TAP to EARL is difficult.
> Converting EARL to TAP is easy.

Making valid EARL out of TAP shouldn't be difficult. Useful EARL on
the other hand

> On the evaluation of TAP I can point to TAP::DOM and Data::DPath, which
> provide a more structured approach to evaluate test results, see my “TAP
> Juggling” slides[2], page 30ff.