Re: [tap] W3C Evaluation and Report Language (EARL)

[Leon Timmermans <[e-mail]>]

On Wed, Nov 9, 2011 at 9:25 PM, Steffen Schwigon <[e-mail]> wrote:
> - EARL is similar to other W3C specs in respect to specifying a
>  comprehensive snapshot of known existing topics. For example, it
>  particularly covers all known HTTP methods (POST, GET, PUT, …). That
>  enables it to build tools on top of it that sematically “know” what
>  the document is about.
>
> - TAP in contrast is about specifying test results, really just the
>  *result* focus without hard specification of the tested topic, i.e., a
>  single test has a “description”, so someone reading it knows what it
>  is about but that part does not have a specification.
>
>  For instance, a test about a HTTP method could have any description
>  from “POST” to “that strange other method that I never remember but
>  always use when GET is not sufficient”.
>
>
> See [1] for some related discussion of this aspect.
>
> In this respect I think TAP is more like your RDF with some extensions
> from EARL to describe test success.
>
> That makes the use-cases of TAP and EARL a bit different:
>
> - TAP allows to be produced by anything simple without toolchain
>  support, like embedded devices with nothing but a “print” function,
>  but you can not *sematically* evaluate results.
>
> - EARL seems to require more heavy toolchain support to produce but
>  allows more semantic result evaluation.

I agree. To me, EARL makes sense in a context that heavy into semantic
web and triplestores, but the toolchain would be too heavy for a lot
of usecases.

> Converting TAP to EARL is difficult.
> Converting EARL to TAP is easy.

Making valid EARL out of TAP shouldn't be difficult. Useful EARL on
the other hand

> On the evaluation of TAP I can point to TAP::DOM and Data::DPath, which
> provide a more structured approach to evaluate test results, see my “TAP
> Juggling” slides[2], page 30ff.

Interesting…

Leon