Re: [tap] W3C Evaluation and Report Language (EARL)

[Steffen Schwigon <[e-mail]>]

Christophe Strobbe <[e-mail]> writes:
> Report Language (EARL) and a few related specifications. EARL's core
> use case is reporting the results of accessibility evaluations of
> websites (i.e. accessibility for persons with disabilities), but the
> language itself is generic, so it can also be used in other
> contexts. The language is based on RDF;
> […]
> During our last call for comments, one of the reviewers asked the
> working group if EARL duplicates TAP's efforts, or vice versa. The
> working group thinks that this is not the case; we think that EARL
> could be an alternative report format for TAP if a TAP consumer could
> be written that produces EARL. For this reason, we thought it would be
> interesting to contact you and to make sure we are aware of each
> other's work.

Thanks for sync'ing this back to us. I just skimmed through the specs
and it was indeed interesting. As far as I understand from my (very
short) skimming I think it's not that many duplication of effort as the
main difference is a philosophical one.

- EARL is similar to other W3C specs in respect to specifying a
  comprehensive snapshot of known existing topics. For example, it
  particularly covers all known HTTP methods (POST, GET, PUT, …). That
  enables it to build tools on top of it that sematically “know” what
  the document is about.

- TAP in contrast is about specifying test results, really just the
  *result* focus without hard specification of the tested topic, i.e., a
  single test has a “description”, so someone reading it knows what it
  is about but that part does not have a specification. 

  For instance, a test about a HTTP method could have any description
  from “POST” to “that strange other method that I never remember but
  always use when GET is not sufficient”.


See [1] for some related discussion of this aspect.

In this respect I think TAP is more like your RDF with some extensions
from EARL to describe test success.

That makes the use-cases of TAP and EARL a bit different: 

- TAP allows to be produced by anything simple without toolchain
  support, like embedded devices with nothing but a “print” function,
  but you can not *sematically* evaluate results.

- EARL seems to require more heavy toolchain support to produce but
  allows more semantic result evaluation.

Converting TAP to EARL is difficult. 
Converting EARL to TAP is easy.

On the evaluation of TAP I can point to TAP::DOM and Data::DPath, which
provide a more structured approach to evaluate test results, see my “TAP
Juggling” slides[2], page 30ff.

Kind regards,
Steffen


Footnotes: 
[1]  http://grokbase.com/p/perl.org/qa/2008/04/re-tap-l-user-supplied-yaml-diagnostic-keys-descriptive-version/11ymnpm2765ztojoinznq2lz5674

[2]  http://www.amd64.org/fileadmin/user_upload/pub/yapc_eu_2011_tapjuggling.pdf

-- 
Steffen Schwigon <[e-mail]>
Dresden Perl Mongers <http://dresden-pm.org/>