Re: [tap] Weird thought for the day: "skip 71 - reason"

[Steffen Schwigon <[e-mail]>]

Ovid <[e-mail]> writes:
> One thing which bugs me about TAP is this:
>
>   1..5
>   ok 1 - some test
>   ok 2 - another test
>   ok 3 # skip cuz I said so
>   not ok 4 # skip, skip, baby
>   ok 5 -final test
>  
> What does a skipped "not ok" test mean? Its behaviour is
> undefined. This means that different parsers might implement this in
> different ways.  It's an obscure edge case, but with greater
> adoption of TAP, it might crop up.

I do not have a problem with this semtantic gap, I even already used
it, similar to what #TODO does.

That case for instance happens when Test::Builder toolchains are not
available and it is easier to canonically independently generate the
OK/NOTOK part and the directives.


> Here's what I would like to see.
>
>   1..5
>   ok 1 - some test
>   ok 2 - another test
>   skip 3 - skip cuz I said so
>   skip 4 - skip, skip, baby
>   ok 5 -final test
>
>
> In other words, you can't introduce ambiguity here.

“NOT OK # SKIP” is no ambiguity. It does not conflict with anything
else. It is skipped and semantically being “OK”, same as TODO.

The interpretation as SKIP by the TAP::Parser/Aggregator should just
be handled the same emotionless, statistic way as TODO: just count how
many SKIP, and if you like then also how many of them are unexpectedly
OK/NOTOK and left the “semantic strangeness” as exercise to the user.

I *strongly* vote *against* introducing a new value additionally to
OK/NOT_OK. Especially as it doesn't take away the ambiguity of the
“NOT OK # SKIP” case and would be an artificial change introduced
without real urge from users (others than the “Wizards Of TAP”).

I regularly teach TAP to users and the whole point of it's easyness is
the fact I can tell:

  Basically you either report OK or NOT OK (and a plan). 
  That's all. The rest is just sugar.

And I would like to keep the basics and the sugar divided as it
currently is: basics on the left side, and the more sugar the more on
the right.

TAP is KISS: Keep It Simple, Stupid; with emphasis on “keep”.

For non-Perl, non-Test::Builder toolchains that easiness is vital!


> Though this list is for ietf, those who really care about TAP are
> hopefully reading this.

I do care for TAP. 
I use it. 
I teach it.
I build infrastructures around it. 
Perl and non-Perl. In harmony. Quite.

I hope that qualifies my comments a bit. :-)

Kind regards,
Steffen 
-- 
Steffen Schwigon <[e-mail]>