Re: [tap] RFC Status?

[Andrew de Andrade <[e-mail]>]

Yes, this all helps a bunch. I've been using substack's tape (
https://github.com/substack/tape) and isaacs' tap (
https://github.com/isaacs/node-tap) modules thus far.

tape currently supports v13, which as you pointed out Bruno, does not
formally specify how nesting should work, and it does support defining
tests in a nested way, but does not report them as nested. Instead, it
simply updates the plan count and continues with everything flat/unindented
and reports the plan count at the end. For example this code...

```
var test = require('tape');

test('foobar', function(t) {
  t.plan(3);
  //var test = tape.createHarness();

  t.ok(true, 'foo');
  t.ok(true, 'bar');

  t.test('fibble', function(tt) {
    tt.plan(2);
    tt.ok(true, 'wobble');
    tt.ok(true, 'wizzle');
  });
});
```
...produces...

```
TAP version 13
# foobar
ok 1 foo
ok 2 bar
# fibble
ok 3 wobble
ok 4 wizzle

1..4
# tests 4
# pass  4

# ok
```

What I did just find is an implementation from Ovid from earlier this year
in March:

http://blogs.perl.org/users/ovid/2014/03/perlqa-hackathon-in-lyon---day-1.html
https://github.com/Ovid/tap-stream

This stream approach is really nice, but the only thing that leaves me
scratching my head is that is implies that asynchrony is permitted
(parallelized tests, aggregation, out-of-order reporting), but I'm not
exactly sure what the implications for this are for the aggregate tap
producer.

Should the aggregate reporter report subtests as they are received or
should it keep the subtest results that finish out of order in a buffer and
always report the tests in a deterministic way? I did see this in the
readme, https://github.com/Ovid/tap-stream#caveats , implying AFAICT that
it is left up to the TAP aggregator consuming the stream to either report
things in order with or without numbers (if desired) or out of order, but
un-numbered.

Given my intended use case, buffering results so that results can always be
reported numbered in order would be desirable, since I want to be able to
diff TAP results across test runs under different conditions (e.g.
different dependency versions). Has anyone done any work with diffing TAP
results?

Bruno, is tap4j purely synchronous or at least in order or can subtests be
asynchronous and reported out of order?

Seeing that Ovid is still actively working on this stuff (comments on tap
on perlqa3 as recent as April this year), is it save to assume that there
are ideas here that will make it into version 14? The test blocks you
pointed to does reference version 14 in the example -->
https://web.archive.org/web/20110611083916/http://testanything.org/wiki/index.php/Test_Blocks