Re: [tap] RFC Status?

[Ovid <[e-mail]>]

Hi all,

Yes, I'm still on this list. I was pleasantly surprised at the burst of activity I saw today.

I'd love to see a TAP v14. More importantly, I'd love to see a standard created out of this so that we could actually say "look at the RFC".
 
Best,
Ovid
--
IT consulting, training, international recruiting
       http://www.allaroundtheworld.fr/.
Buy my book! - http://bit.ly/beginning_perl
Live and work overseas - http://www.overseas-exile.com/


On Tuesday, 12 August 2014, 0:41, Bruno P. Kinoshita <[e-mail]> wrote:
 

>
>
>> Yes, this all helps a bunch. I've been using substack's tape (https://github.com/substack/tape) and isaacs' tap (https://github.com/isaacs/node-tap) modules thus far. 
>
>
>I've seen some users reporting issues in the tap-plugin for Jenkins where they were using node-tap. Hadn't heard about tape before. 
>
>
>> What I did just find is an implementation from Ovid from earlier this year in March:
>>
>> http://blogs.perl.org/users/ovid/2014/03/perlqa-hackathon-in-lyon---day-1.html
>https://github.com/Ovid/tap-stream
>
>
>I liked Ovid's implementation. One thing that i have added to my TODO list is to look at his examples and use some of them as regression tests in tap4j. I think it goes along with your previous suggestion of a set of tests for TAP (as in JSON?).
>
>
>> Bruno, is tap4j purely synchronous or at least in order or can subtests be asynchronous and reported out of order?
>
>
>Purely synchronous at the moment. There is a factory, that creates a TapConsumer, which receives a TAP stream or java.util.File, and then it calls the parser.
>
>
>The parser itself uses a state machine (based on SnakeYAML parser). So my guess is that it is not too hard to add a new asynchronous consumer, that passes each line received to the parser, in order. This assync consumer would deal with java threading, and raise any exception received from the parser (such as receiving tokens in the order Test-Result -> Tap-Plan -> Test-Result).
>
>
>> Seeing that Ovid is still actively working on this stuff (comments on tap on perlqa3 as recent as April this year), is it save to assume that there are ideas here that will make it into version 14? The test blocks you pointed to does reference version 14 in the example -->  https://web.archive.org/web/20110611083916/http://testanything.org/wiki/index.php/Test_Blocks
>
>
>I'm not sure. Do you know if Ovid is in this list? Maybe we should check with him if he has interest in joining the thread and help crafting TAP 14 too? 
>
>
>Bruno
>
>
>
>>________________________________
>> From: Andrew de Andrade <[e-mail]>
>>To: Bruno P. Kinoshita <[e-mail]> 
>>Cc: Gaurav Vaidya <[e-mail]>; "[e-mail]" <[e-mail]> 
>>Sent: Monday, August 11, 2014 6:30 PM
>>Subject: Re: [tap] RFC Status?
>> 
>>
>>
>>Yes, this all helps a bunch. I've been using substack's tape (https://github.com/substack/tape) and isaacs' tap (https://github.com/isaacs/node-tap) modules thus far. 
>>
>>
>>tape currently supports v13, which as you pointed out Bruno, does not formally specify how nesting should work, and it does support defining tests in a nested way, but does not report them as nested. Instead, it simply updates the plan count and continues with everything flat/unindented and reports the plan count at the end. For example this code...
>>
>>
>>```
>>var test = require('tape');
>>
>>
>>test('foobar', function(t) {
>>  t.plan(3);
>>  //var test = tape.createHarness();
>>
>>
>>  t.ok(true, 'foo');
>>  t.ok(true, 'bar');
>>
>>
>>  t.test('fibble', function(tt) {
>>    tt.plan(2);
>>    tt.ok(true, 'wobble');
>>    tt.ok(true, 'wizzle');
>>  });
>>});
>>```
>>...produces...
>>
>>
>>```
>>TAP version 13
>># foobar
>>ok 1 foo
>>ok 2 bar
>># fibble
>>ok 3 wobble
>>ok 4 wizzle
>>
>>
>>1..4
>># tests 4
>># pass  4
>>
>>
>># ok
>>```
>>
>>
>>What I did just find is an implementation from Ovid from earlier this year in March:
>>
>>
>>http://blogs.perl.org/users/ovid/2014/03/perlqa-hackathon-in-lyon---day-1.html
>>
>>https://github.com/Ovid/tap-stream
>>
>>
>>
>>This stream approach is really nice, but the only thing that leaves me scratching my head is that is implies that asynchrony is permitted (parallelized tests, aggregation, out-of-order reporting), but I'm not exactly sure what the implications for this are for the aggregate tap producer. 
>>
>>
>>Should the aggregate reporter report subtests as they are received or should it keep the subtest results that finish out of order in a buffer and always report the tests in a deterministic way? I did see this in the readme, https://github.com/Ovid/tap-stream#caveats , implying AFAICT that it is left up to the TAP aggregator consuming the stream to either report things in order with or without numbers (if desired) or out of order, but un-numbered. 
>>
>>
>>Given my intended use case, buffering results so that results can always be reported numbered in order would be desirable, since I want to be able to diff TAP results across test runs under different conditions (e.g. different dependency versions). Has anyone done any work with diffing TAP results?
>>
>>Bruno, is tap4j purely synchronous or at least in order or can subtests be asynchronous and reported out of order?
>>
>>
>>Seeing that Ovid is still actively working on this stuff (comments on tap on perlqa3 as recent as April this year), is it save to assume that there are ideas here that will make it into version 14? The test blocks you pointed to does reference version 14 in the example -->  https://web.archive.org/web/20110611083916/http://testanything.org/wiki/index.php/Test_Blocks
>>
>>
>>
>>
>
>_______________________________________________
>tap mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/tap
>
>
>