[tap] Fwd: concatenated streams discussion from tap-l

Gaurav Vaidya <gaurav@ggvaidya.com> Sun, 29 August 2010 14:40 UTC

Return-Path: <ggvaidya@gmail.com>
X-Original-To: tap@core3.amsl.com
Delivered-To: tap@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 55E8C3A6826 for <tap@core3.amsl.com>; Sun, 29 Aug 2010 07:40:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 1.22
X-Spam-Level: *
X-Spam-Status: No, score=1.22 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, RCVD_IN_BL_SPAMCOP_NET=1.96]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6mvSFO9pVS0n for <tap@core3.amsl.com>; Sun, 29 Aug 2010 07:40:34 -0700 (PDT)
Received: from mail-pz0-f44.google.com (mail-pz0-f44.google.com [209.85.210.44]) by core3.amsl.com (Postfix) with ESMTP id D52783A67D9 for <tap@ietf.org>; Sun, 29 Aug 2010 07:40:34 -0700 (PDT)
Received: by pzk6 with SMTP id 6so2132563pzk.31 for <tap@ietf.org>; Sun, 29 Aug 2010 07:41:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:cc:message-id:from:to :content-type:content-transfer-encoding:mime-version:subject:date :references:x-mailer; bh=YipnJZAdHdetS/DPqiv48vxRe/RxhrEnTgFV3spWM5c=; b=rFfofyXDzNjcKXMgB9Kej1h/YBIdEnTMT01DM2RVbbouQbyJE3HOG55vdEeN4ZJSsA 8iwtGIu4m1Wdv/UJYVUFKEzfp1s0nN2vj0xGc369g3E/UxnxGxRf00nWHguRruTSnJwN qG/tCzWHiPjxEgziUVll/WesdvlGyU3/abA+w=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:cc:message-id:from:to:content-type:content-transfer-encoding :mime-version:subject:date:references:x-mailer; b=PlW7Kyr7L8JNH1Ko2686ZY2lc4L56pDCj4Thv3ZAoKwKH6Cclqe+/4HIzr6b2CFGFF fRIPXzcNcY/ExZhT7k7bjMcMXrpqzuUFQ5Of+Q5lNG9XB3WHockQowTsy/VMnPUJu0qV rRUK+pO5aFH4GN+wtDkC/YjI0hKdZ5yB2HG/g=
Received: by 10.114.120.7 with SMTP id s7mr3647105wac.131.1283092865906; Sun, 29 Aug 2010 07:41:05 -0700 (PDT)
Received: from [192.168.1.7] (bb121-6-245-19.singnet.com.sg [121.6.245.19]) by mx.google.com with ESMTPS id c24sm11862498wam.19.2010.08.29.07.41.03 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 29 Aug 2010 07:41:04 -0700 (PDT)
Sender: Gaurav Vaidya <ggvaidya@gmail.com>
Message-Id: <2A753CCE-19F5-4494-B514-3010764CCEA4@ggvaidya.com>
From: Gaurav Vaidya <gaurav@ggvaidya.com>
To: TAP <tap@ietf.org>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v936)
Date: Sun, 29 Aug 2010 22:41:00 +0800
References: <edf9df58df8adc98cdf8cc6c99fd7e8f@swcp.com>
X-Mailer: Apple Mail (2.936)
Cc: EBo <ebo@sandien.com>
Subject: [tap] Fwd: concatenated streams discussion from tap-l
X-BeenThere: tap@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Test Anything Protocol WG discussions <tap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tap>
List-Post: <mailto:tap@ietf.org>
List-Help: <mailto:tap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 29 Aug 2010 14:40:36 -0000

Hey everybody,

As some of you know, EBo (who's working on a TAP producer on the Plan  
9 operating system) has some ideas about adding test names into the  
TAP header and has been sharing these on the tap-l@testanything.org  
list. He's been having problems signing up for the IETF mailing list,  
and so has asked me to forward his proposal onto this list.

cheers,
Gaurav

---
From: ebo@sandien.com
Subject: 	concatenated streams discussion from tap-l
Date: August 29, 2010 1:31:19 PM SST

While on the tap-l forum I started a thread on parsing concatenated
streams
<http://testanything.org/pipermail/tap-l/2010-August/000344.html>

The upshot of the thread is that by simply adding naming to TAP plans
allows a concatenated stream to be fully reparsed by a consumer.  I
have since modified the proposed grammar slightly to allow different
versions of TAP plans to be embedded in the stream:

=========================

; allow parsing of concatenated streams from different versions of TAP
; to be interspersed. It also allows for an empty file to be
; considered a valid TAP stream.

Stream      = *Testset
Testset     = Header [Comments] [Test-Name] (Plan Body / Body Plan)  
Footer
Test-Name   = "NAME" SP Name-String [SP Description] EOL

; test names are printable UTF8 without SPACE, DEL or "#"

Name-String = 1*(%x21-22 %x24-7E %x80-FF)

=========================

With this addition I was able to reparse Test::Harness's verbose
output and regenerate the complete plan-by-plan statistics after the
fact.  So, the results of runtests can be dumped to a file or pipe and
later parsed by consumers.  There are a number of subtle variants of
the proposed grammar which might be more suitable, but Test::Harness's
prove command already dumps the file name to the output as the first
item output, and just prior to the plan. Unfortunately it does
not explicitly tag it, so there is no way to know if this is a line to
be ignored or to be taken as the name.  Also, while the test names end
in the .t extension it is not a requirement, and a potential problem
with assuming that the name is always there could potentially break
consumers.  If we take Test::Harness as the canonical source, then the
current behavior is actually more closely interpreted as:

Testset     = Header [Comments] Test-Name (Plan Body / Body Plan) Footer
Test-Name   = Name-String [SP Description] EOL

As a note, I later discovered that naming and concatenated threads
addresses issues in the things missing from TAP document
<http://testanything.org/wiki/index.php/Things_missing_from_TAP>, but
does not address any hierarchical nesting of plans, just concatenated
ones.

Anyway, I am soliciting comments on the above proposal to add name
tags to TAPv14.

  Best regards,

  EBo --