[tap] RFC Status?

Andrew de Andrade <andrew@deandrade.com.br> Mon, 11 August 2014 18:42 UTC

Return-Path: <andrew@deandrade.com.br>
X-Original-To: tap@ietfa.amsl.com
Delivered-To: tap@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F22021A077F for <tap@ietfa.amsl.com>; Mon, 11 Aug 2014 11:42:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 1.502
X-Spam-Level: *
X-Spam-Status: No, score=1.502 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_NEUTRAL=0.779] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gTdvKj8bytFO for <tap@ietfa.amsl.com>; Mon, 11 Aug 2014 11:42:40 -0700 (PDT)
Received: from mail-qc0-f171.google.com (mail-qc0-f171.google.com [209.85.216.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 759911A077C for <tap@ietf.org>; Mon, 11 Aug 2014 11:42:40 -0700 (PDT)
Received: by mail-qc0-f171.google.com with SMTP id r5so1912934qcx.2 for <tap@ietf.org>; Mon, 11 Aug 2014 11:42:39 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type; bh=rImrQZmQZ1vbSZWLw4UgT4XoWCZPVKmlaRNjSDEqUQE=; b=CNUvcAv24qo7y4sLb7q+7Z0doRmfhdZiOSt2WWKSq8qbLjbwACHPRVCX0WptsDcZSI gJ8q1r/LNzViy+80GKnOny2vmE593WcsLmORBcD6jNRkI5vckNcu8EReb1iQsTG7ZMjX xRmrcIHTwyDpU4XMbzVZyUyGIJ17ekk6/jQVnSKKra6XMHvNCowlHlPpNdfuTFpB0ftR 6H9WIC3csl5X6A9V0XURq5UjcJ+9UD/6c83WrzSjTdtCBEP4WQEq1tNX4XRgTYw1gdbF nH4Y3+Hb1RPIf5Wabz3KSwcdlSrWjDM9KOzjIGQGwvjXLC3RiIGCJCJKJwUXgJlYH6mn 9M4g==
X-Gm-Message-State: ALoCoQl1EWy+rru3yEjtMYKmc05R/LUC4/eJpT7DwPxYx6NOzmCjnAS3j2i+LaPzdIlpLfex+5JF
X-Received: by 10.229.202.199 with SMTP id ff7mr65773203qcb.9.1407782559653; Mon, 11 Aug 2014 11:42:39 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.140.51.243 with HTTP; Mon, 11 Aug 2014 11:42:19 -0700 (PDT)
X-Originating-IP: [208.66.27.18]
From: Andrew de Andrade <andrew@deandrade.com.br>
Date: Mon, 11 Aug 2014 11:42:19 -0700
Message-ID: <CAP4gcszybVr5Hw3mg=uTi8tqpA3wEVwo=zf2876RWhy_CmozZw@mail.gmail.com>
To: tap@ietf.org
Content-Type: multipart/alternative; boundary=001a11c2fe3a63ba9505005eebab
Archived-At: http://mailarchive.ietf.org/arch/msg/tap/wbjsiKveZMja1rjg_4UPdcLsJhw
Subject: [tap] RFC Status?
X-BeenThere: tap@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Test Anything Protocol WG discussions <tap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tap>, <mailto:tap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tap/>
List-Post: <mailto:tap@ietf.org>
List-Help: <mailto:tap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Aug 2014 18:48:28 -0000

I'm working on some testing libraries for JavaScript based on TAP.
Specifically, I'm trying to explore making it possible for library authors
to not only check the results of their TAP tests, but also map-reduce the
TAP results of the dependents of their library to see how changes to their
library impact the correctness of their dependents.

e.g. I am the author of library A. I publish version 0.1.0. Another author
includes my library as a dependency in library B and in library C and uses
TAP to test those libraries. Given this scenario, it would be nice if I
could, when modifying my library, run not only my tests, but also run the
tests of all my dependents built with my previous version and my current
version and compare how my changes impacted the "correctness" of their
programs.

Beyond the benefits to a community, helping authors know when they can
likely safely upgrade their deps, this type of runner would also help in an
corporate environment with lots of code re-use between teams.

While thinking about this, I decided to go out and figure out if there is
such a thing as nested TAP in the protocol specification. While searching
for this, I came across this group, which is a great, but it appears there
has been no activity here in a while and it looks like all the wiki pages
with prior information about the state of the TAP protocol becoming an IETF
RFC have disappeared.

What's the current state of TAP? Is there still interest in taking this to
RFC status? I found some previous discussions on the list about nested TAP.
Did those make it into the specification in any way? If so, where can I
find examples of correct nested tap results?

Furthermore, is there a standardized set of test fixtures for the current
version that any implementation can be tested against for correctness and
performance? Having worked a bit with JSON schema, I found that one of the
most useful tools was a standardized set of tests to check if a particular
implementation conforms to the protocol standard. (see:
https://github.com/json-schema/JSON-Schema-Test-Suite )