Re: [tap] "ok 1 description" vs "ok 1 - description"

[Michael G Schwern <[e-mail]>]

Steffen Schwigon wrote:
>>>  TH2early (not stripped),
>>>  TH2late  (stripped),
>>>  TH3/TP   (not stripped),
>>>
>>> ?
>> Yep, that's about right.  Nothing before TH2 did anything at all
>> with the description.
> 
> (I'm dreaming about TAP... my thoughts drip out incrementally...)
> 
> I'm getting unsure again because: It could also mean that not
> stripping was ok and the stripping was an experiment that was reverted
> with some reason with TH3. I don't know. Who doeas? $Ovid? @Andy? 
> $You?

An intersection of the three.

The REALLY IMPORTANT thing to realize was that so few people do anything with
descriptions except maybe print them out that very little thought has been put
into their exact parsing.  So don't read too much into what the
implementations did/do.

TH2early didn't strip the description because it was the first TAP parser that
allowed you to do anything with the parsed results so it just wasn't considered.

TH2late did strip the description because it was conforming to what Test::More
did.

TH3 removed the stripping because it wasn't considered part of the fledging
protocol which we were just then starting to write down.

But in all that we never considered the simple possibility that the dash could
be an optional element.


>>> I agree then that stripping the dash from description might not be
>>> so end-of-world as I initially thought.
>>>
>>> I'm maybe one of the very few who currently depend on the exact
>>> descriptions but I can probably even change my evaluation tools.
>> Fortunately, it will be self consistent within one tool.
> 
> Hm, not neccessarily as I provide an API to evaluate the archived TAP
> by descriptions. It would just work as long as I control the
> evaluation environment, which I still do at the moment.

Are you going to be passing interpretations of the descriptions back and forth
between different implementations?  That is, when does the interpretation of
the description matter to anything but a single implementation?


>> What are you doing with them?
> 
> Archiving to track history of testruns and recognize result
> changes. Probably similar to Smolder, I think, maybe more
> finegrained. Therefore I care for single tests and their descriptions.

Looking at it another way, the WORST thing that happens with mixing a legacy
producer which isn't aware of the dash with a post-spec, dash stripping parser
is that the description might get interpreted wrong and you can't associate a
description with it's historical results (though that would indicate that the
test started using a different producer, which stretching this scenario even
further).  Since using the description as a unique test ID is already a
heuristic (something that was discussed in some depth at Oslo) your system
must already be resistant to this problem.

Finally, if the parser always strips the optional dash it should always come
out the same whether the producer uses a leading dash or not EXCEPT for one
contrived case.  When a description deliberately is supposed to have a leading
" -".  I can only think of a fairly contrived example, but if you wanted to
output, say, Morse code "- ... -.-", and your producer doesn't know about the
optional dash, it would say:

ok 1 - ... -.-

Whereas a compliant producer would say:

ok 1 - - ... -.-

A compliant parser would parse the former as "... -.-" and the latter as "-
... -.-".  Same intended meaning, different interpreted descriptions.

Is this edge case important?  It seems one simple way to allay your fears is
to simply go through your historical TAP archives and look for instances where
a description starts with "- ".  If all the descriptions in a stream start
with "- " assume it's Test::Builder output and ignore it.  If only some of
them do, then we've found our exception.

I'm interested to hear what the results are.


-- 
94. Crucifixes do not ward off officers, and I should not test that.
    -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
           http://skippyslist.com/list/