[tap] Parse error vs failure

In Oslo, when we came across an edge case that was considered nonsense, the
tendency was to say that its a parse error.  For example, "not ok 4 # SKIP",
because why would you fail a test that didn't run?  Also "1..0" without a
skip, because why would you not run any tests and yet not skip the test?

I think I brought this up earlier, but I want to make sure I have consensus.
I've been calling them failures (with a warning) rather than parse errors.
Why?  Because a parse error means the parser can not extract any information
and can't really tell the user anything but "syntax error at line X".  If your
parser starts guessing at what sort of syntax error it is, well, then it might
as well be grammar at that point.

Worse, a TAP document parser, parsing the full document at once rather than a
traditional line-by-line parser, might invalidate the whole stream and thus
can not extract any information.

If it's just a failure, the parser can get information out of it and has
flexibility about what to do and what action it can suggest the user take.

It complicates the grammar to allow "ok 4 # SKIP" but make "not ok 4 # SKIP" a
syntax error and adds in another special case for the parser.  With "not ok #
SKIP" as normal syntax the skip directive has no effect on the truthiness of
the test.

Consensus?  Generally prefer failure-with-warning over parse error for valid
but sort of nonsense output?

-- 
184. When operating a military vehicle I may *not* attempt something
     "I saw in a cartoon".
    -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
           http://skippyslist.com/list/

_______________________________________________
tap mailing list
tap@ietf.org
https://www.ietf.org/mailman/listinfo/tap