Re: [tap] RFC Status?

Steffen Schwigon <ss5@renormalist.net> Mon, 22 September 2014 08:19 UTC

Return-Path: <ss5@renormalist.net>
X-Original-To: tap@ietfa.amsl.com
Delivered-To: tap@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 761971A1A62 for <tap@ietfa.amsl.com>; Mon, 22 Sep 2014 01:19:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.014
X-Spam-Level:
X-Spam-Status: No, score=0.014 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RP_MATCHES_RCVD=-0.786] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pEIN_zW1hFtS for <tap@ietfa.amsl.com>; Mon, 22 Sep 2014 01:19:45 -0700 (PDT)
Received: from h132974.serverkompetenz.net (renormalist.net [81.169.141.46]) by ietfa.amsl.com (Postfix) with ESMTP id 15D3D1A1A5C for <tap@ietf.org>; Mon, 22 Sep 2014 01:19:43 -0700 (PDT)
Received: by h132974.serverkompetenz.net (Postfix, from userid 1000) id 833A03F2787; Mon, 22 Sep 2014 10:19:33 +0200 (CEST)
From: Steffen Schwigon <ss5@renormalist.net>
To: Leon Timmermans <fawaka@gmail.com>
References: <CAP4gcszybVr5Hw3mg=uTi8tqpA3wEVwo=zf2876RWhy_CmozZw@mail.gmail.com> <CAHhgV8jr6ZnsfUkpFC4OL0AwRX-aen7v-7KjcN3e0_19s7steg@mail.gmail.com> <1410980929.82809.YahooMailNeo@web126106.mail.ne1.yahoo.com> <87bnqbwqhe.fsf@renormalist.net> <CAHhgV8gny-6ZJuec_83h0C+WbjAMXUA+JqLC6xgZj6FUAO-JTg@mail.gmail.com>
Date: Mon, 22 Sep 2014 10:19:33 +0200
In-Reply-To: <CAHhgV8gny-6ZJuec_83h0C+WbjAMXUA+JqLC6xgZj6FUAO-JTg@mail.gmail.com> (Leon Timmermans's message of "Sun, 21 Sep 2014 01:22:03 +0200")
Message-ID: <87ppeof4oa.fsf@renormalist.net>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Archived-At: http://mailarchive.ietf.org/arch/msg/tap/XkaPcTzduPmtXrZOPZBiHZYcwMM
Cc: "tap@ietf.org" <tap@ietf.org>
Subject: Re: [tap] RFC Status?
X-BeenThere: tap@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Test Anything Protocol WG discussions <tap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tap>, <mailto:tap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/tap/>
List-Post: <mailto:tap@ietf.org>
List-Help: <mailto:tap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tap>, <mailto:tap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 22 Sep 2014 08:19:51 -0000

Leon Timmermans <fawaka@gmail.com> writes:
> On Fri, Sep 19, 2014 at 5:59 PM, Steffen Schwigon <ss5@renormalist.net> wrote:
>     And here I seriously start to panic if that opinion should ever lead to
>     deprecating YAMLish, as 1) it much better fits the philosophy of TAP's
>     line-based'ness, 2) is easier to generate with line based tools, like in
>     a shell-only world, and 3) zillions of test results are already based on
>     it for the last 6 years. I'm not exaggerating here.
>
> I wasn't aware of anyone actually outputting YAML. Can you tell us more about this?

With great pleasure. :-) Here we go:

The Tapper test infrastructure (http://tapper-testing.org/) consumes it
with the help of TAP::DOM, Data::DPath (and a handful marines), and it's
generated for embedded benchmark data in projects like Perl::Formance
(http://perlformance.net/), general supporting data in libs like
"bash-test-utils"
(https://github.com/tapper/Tapper-autoreport/blob/master/bash-test-utils.md)
and similar, but less public projects. In general we use it in a
restricted world like OS testing with only shell/printf/printk
available.

These projects might not look very active but that's only lack of github
pushes due to lack of tuits (usually just once a year during the Perl QA
hackathons). They are in very heavy use.

Slide decks (bit old but still valid):
 - http://tapper-testing.org/docs/yapc_eu_2011_tapjuggling.pdf - pg. 23, 37, 41
 - http://tapper-testing.org/docs/linuxcon_eu_2011_linux_and_virtualization_testing_with_tapper.pdf  pg. 35, 43

Two details make it harder to generate JSON than YAMLish:
 - on generating: JSON is oversensitive to trailing comma before closing brackets/braces.
   If you generate it from templates, like Template-Toolkit, you go mental.
 - on consuming: JSON defines Javascript-specific thingies, in particular true/false.
   Sounds minor, but makes it hard in restricted worlds like OS testing.

Kind regards,
Steffen
-- 
Steffen Schwigon <ss5@renormalist.net>