[Rfc-markdown] 1.3.36: new: kramdown-rfc-cache-i-d-bibxml

Carsten Bormann <cabo@tzi.org> Tue, 23 February 2021 21:08 UTC

Return-Path: <cabo@tzi.org>
X-Original-To: rfc-markdown@ietfa.amsl.com
Delivered-To: rfc-markdown@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 4506C3A0BAB for <rfc-markdown@ietfa.amsl.com>; Tue, 23 Feb 2021 13:08:12 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.797
X-Spam-Status: No, score=-1.797 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, TRACKER_ID=0.1, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id kZrL80BoimYf for <rfc-markdown@ietfa.amsl.com>; Tue, 23 Feb 2021 13:08:09 -0800 (PST)
Received: from gabriel-vm-2.zfn.uni-bremen.de (gabriel-vm-2.zfn.uni-bremen.de []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0497D3A0BA0 for <rfc-markdown@ietf.org>; Tue, 23 Feb 2021 13:08:08 -0800 (PST)
Received: from [] (p5089a828.dip0.t-ipconnect.de []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gabriel-vm-2.zfn.uni-bremen.de (Postfix) with ESMTPSA id 4DlWqC192tzyZ7; Tue, 23 Feb 2021 22:08:07 +0100 (CET)
From: Carsten Bormann <cabo@tzi.org>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Mao-Original-Outgoing-Id: 635807286.34206-9d42202613e0aa5366eae4b72b838743
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.\))
Date: Tue, 23 Feb 2021 22:08:06 +0100
Message-Id: <B05F1C0A-9248-4142-83F0-AFAB2A3E896F@tzi.org>
To: rfc-markdown@ietf.org
X-Mailer: Apple Mail (2.3608.
Archived-At: <https://mailarchive.ietf.org/arch/msg/rfc-markdown/FJkHJJrj0ci_uemB5F5sSEP7Wyg>
Subject: [Rfc-markdown] 1.3.36: new: kramdown-rfc-cache-i-d-bibxml
X-BeenThere: rfc-markdown@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "rfc-markdown is a discussion list for people writing I-Ds and RFCs in Markdown and the authors of the tools used for that." <rfc-markdown.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rfc-markdown>, <mailto:rfc-markdown-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rfc-markdown/>
List-Post: <mailto:rfc-markdown@ietf.org>
List-Help: <mailto:rfc-markdown-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rfc-markdown>, <mailto:rfc-markdown-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 23 Feb 2021 21:08:12 -0000

TL;DR: You don’t need the below.
But it can make you faster and work better while offline.

Grüße, Carsten

Many of us have been using rsync to get copies of the bibxml files on xml2rfc.tools.ietf.org so we did not have to wait for kramdown-rfc to fetch updates.  However, datatracker currently does not offer rsyncing, so you couldn’t get the bibxml3 directory right from the horse’s mouth.

Now you can.  1.3.36 comes with a new command:


This does a cache preload: it sends copies of the bibxml for all active(*) I-Ds (both numbered and up-to-date references) to your KRAMDOWN_REFCACHEDIR.

(The environment variable KRAMDOWN_REFCACHEDIR should point to ~/.cache/xml2rfc to share a cache between xml2rfc and kramdown-rfc.  If it is not set at all, you will be warned, but the command will happily proceed with that default.  Kramdown-rfc still defaults to ./.refcache, though, so it won’t find the cache...)

The command is rather expensive(**) on the first call (initial pull), but quite efficient after that (update pull).

You could put the invocation into your crontab.  Even though it’s quite fast, it is probably not worth calling it more than once an hour, because the all_id2.txt file on which it builds currently isn’t refreshed more often.

kramdown-rfc-cache-i-d-bibxml is not for everyone: it needs at least 2016’s Ruby 2.4 (***) and an install of the gem "net-http-persistent" (not installed as a requirement by default, as it may be problematic on some older platforms).

(*) So kramdown-rfc’s existing caching engine will take care of your very few references to outdated I-Ds referenced by version number and expired I-Ds.

(**) including for datatracker itself — it makes some 3600 HTTP requests in a row on an initial pull.  So don’t do that 20 minutes before the I-D deadline — be prepared, and only do update pulls in times of I-D frenzy.
[And now you know why I'm pushing this update a day after the I-D deadline :-)]

(***) If you have a Mac, and are using the system Ruby, this means it works on Catalina and higher.  We do prefer using a homebrew-installed Ruby in any case.