[CFRG] Re: Where should test vectors live?

Carsten Bormann <cabo@tzi.org> Fri, 20 September 2024 01:24 UTC

Return-Path: <cabo@tzi.org>
X-Original-To: cfrg@ietfa.amsl.com
Delivered-To: cfrg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4AAC9C1519AC for <cfrg@ietfa.amsl.com>; Thu, 19 Sep 2024 18:24:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.906
X-Spam-Level:
X-Spam-Status: No, score=-1.906 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V5YeMleCeaBv for <cfrg@ietfa.amsl.com>; Thu, 19 Sep 2024 18:24:39 -0700 (PDT)
Received: from smtp.zfn.uni-bremen.de (smtp.zfn.uni-bremen.de [134.102.50.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 58448C151980 for <cfrg@irtf.org>; Thu, 19 Sep 2024 18:24:38 -0700 (PDT)
Received: from smtpclient.apple (p5dc5d074.dip0.t-ipconnect.de [93.197.208.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.zfn.uni-bremen.de (Postfix) with ESMTPSA id 4X8vnJ0S3dzDCbH; Fri, 20 Sep 2024 03:24:36 +0200 (CEST)
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\))
From: Carsten Bormann <cabo@tzi.org>
In-Reply-To: <CAG2Zi20N98cxpgjfRe6gWw1SQEoux+5P3NhLBFUfUHk_udYeFg@mail.gmail.com>
Date: Fri, 20 Sep 2024 03:24:25 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <282D3E19-B25D-4E1A-B13E-778A9B5A9A4E@tzi.org>
References: <CAG2Zi20N98cxpgjfRe6gWw1SQEoux+5P3NhLBFUfUHk_udYeFg@mail.gmail.com>
To: Christopher Patton <cpatton=40cloudflare.com@dmarc.ietf.org>
X-Mailer: Apple Mail (2.3776.700.51)
Message-ID-Hash: BG3C7UWDO55VPDUHA4IEEZWUL6O2UTLE
X-Message-ID-Hash: BG3C7UWDO55VPDUHA4IEEZWUL6O2UTLE
X-MailFrom: cabo@tzi.org
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-cfrg.irtf.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
CC: CFRG <cfrg@irtf.org>
X-Mailman-Version: 3.3.9rc4
Precedence: list
Subject: [CFRG] Re: Where should test vectors live?
List-Id: Crypto Forum Research Group <cfrg.irtf.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/cfrg/E8Q_gBN4rzx39BoHJNZOzu0Tupk>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cfrg>
List-Help: <mailto:cfrg-request@irtf.org?subject=help>
List-Owner: <mailto:cfrg-owner@irtf.org>
List-Post: <mailto:cfrg@irtf.org>
List-Subscribe: <mailto:cfrg-join@irtf.org>
List-Unsubscribe: <mailto:cfrg-leave@irtf.org>

On 20. Sep 2024, at 02:17, Christopher Patton <cpatton=40cloudflare.com@dmarc.ietf.org> wrote:
> 
> In theory you could write a script that downloads https://www.rfc-editor.org/rfc/rfc9180.txt and write a parser to pull out the test vectors, but does anyone really do this?

It is actually much better to download https://www.rfc-editor.org/rfc/rfc9180.xml and use a tool such as kramdown-rfc-extract-sourcecode [1] to extract the <sourcecode snippets in the document.
If the document authors have used the type= and name= attributes, the extraction result can go right into CI scripts and other processing.
(In RFC 9180, the type attribute is used wisely, but the name attributes are all empty, so kramdown-rfc-extract-sourcecode will guess file names from the context.)

Maintaining your own test vector blob of course gives you more ways to evolve the test vectors (and to handle those pesky errata), but I wanted to point out that it is easy to get at the actual test vectors in the RFCXML document without writing fragile heuristic text file parsers.

Grüße, Carsten

[1]: https://github.com/cabo/kramdown-rfc/wiki/Extracting-sourcecode