IETF Logo Mail Archive

[Ietf-and-github] Tools for GitHub backups

Martin Thomson <mt@lowentropy.net> Fri, 20 March 2020 04:51 UTC

Return-Path: <mt@lowentropy.net>
X-Original-To: ietf-and-github@ietfa.amsl.com
Delivered-To: ietf-and-github@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E9433A1582; Thu, 19 Mar 2020 21:51:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Level:
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b=ZbtD5K6D; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=1kdTblaP
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tIQPV7W8h_vb; Thu, 19 Mar 2020 21:51:37 -0700 (PDT)
Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 441F03A157E; Thu, 19 Mar 2020 21:51:37 -0700 (PDT)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id 5A4B92EA; Fri, 20 Mar 2020 00:51:36 -0400 (EDT)
Received: from imap2 ([10.202.2.52]) by compute2.internal (MEProxy); Fri, 20 Mar 2020 00:51:36 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=mime-version:message-id:date:from:to:subject:content-type; s= fm1; bh=CSAXBnWNqeH363Ry7i+QyMONocQF+yMqAVv39C/iwKs=; b=ZbtD5K6D R/p6sDV1KVGum6K4wyhzg41LKGZNVGbnPvC6qfdtOEzf2q7GZfyEsPldkB5GfvOO JedC+taq+VYixGAhByBARvymWPZj36Y8NobAH+j9fx3BVW+MqVXJBam4utV1vzZD XSYjfEe4Ff4WRO0O+hoxr8ZrNah9P3He27O2R85rf/0rpf0p1C8DQxvOqj4ZZ3o4 jbkTgg+dpm3rU5aTQ6lsZ/l0HcAPXGdxterx3NUWI7o7vZCGvXsQ38SWaD0zswA9 tL81uoL7DeOjQHbpZiAKDWuUQsXPZX/HdNVz8ZFU9OLjfObeMx46X10B2Zvs1WKo KQLDBAYwucNzpg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=CSAXBnWNqeH363Ry7i+QyMONocQF+ yMqAVv39C/iwKs=; b=1kdTblaPkyMccWAkyZM+FQotPMmCH8vNoCaD4Jd8zoKOj 0uWoopUTQsGVd6JpbOuUIdCGvzDED3QwChLsctW8rvq6SeAD8AIqpuf+M14bXk8m Pi+CSM1BucpWRwsaY6KMGLemJ3OK2yN2ViapDku7Ly1xSSiRrIl1FJdrBwaijknm CmXQ5SI8qpZ4x3/aiq21rswvwKvRZxvkXpQmH8LyuygZjYzrv3jFfQjjBN+2DJWG ZuO3sM2Svc1hZoOWx9UMpn72GuSHNldDHirnmRQ8+RPWSht0XlGCxzweLCJtTLxN M2RhvZxyegQBcUrTlLW6XU8plh62qzbdMfOLRydIg==
X-ME-Sender: <xms:10t0XjrI53-H9-5qJLaEPR1MlBVT2FFxpx0tJQd_s96R6iLz2wNyVQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrudegtddgjeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfffhffvufgtsehttdertd erredtnecuhfhrohhmpedfofgrrhhtihhnucfvhhhomhhsohhnfdcuoehmtheslhhofigv nhhtrhhophihrdhnvghtqeenucffohhmrghinhepghhithhhuhgsrdgtohhmpdhquhhitg ifghdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhr ohhmpehmtheslhhofigvnhhtrhhophihrdhnvght
X-ME-Proxy: <xmx:10t0XjC6HjDsel2L6dPLi8CEL1xq16DMpspKXs3hPzcvAQp5Tuv5ug> <xmx:10t0XspowC-4ZVnqqtl_EnQj7Go4yHb-G8te4Bz7djPl4TehNbVQmg> <xmx:10t0XsCrih2D0AhDQTfYU0E-pdrgDIhInwpxxk8AQvEKuTlofzi6Vw> <xmx:2Et0XtLRlNWE8fC3VG2hku1h-hAt_r30iuWfE0omBH0zwDW_LQJ7vw>
Received: by mailuser.nyi.internal (Postfix, from userid 501) id 9222AE00F6; Fri, 20 Mar 2020 00:51:35 -0400 (EDT)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.1.7-1021-g152deaf-fmstable-20200319v1
Mime-Version: 1.0
Message-Id: <6c820473-d7ab-47c5-885b-56ea519ebefc@www.fastmail.com>
Date: Fri, 20 Mar 2020 15:51:14 +1100
From: Martin Thomson <mt@lowentropy.net>
To: ietf-and-github@ietf.org, wgchairs@ietf.org
Content-Type: text/plain
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf-and-github/DhuoTrmo1TCsZ7cMXsDQoc_zpfg>
Subject: [Ietf-and-github] Tools for GitHub backups
X-BeenThere: ietf-and-github@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of using GitHub in IETF activities, particularly for Working Groups" <ietf-and-github.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf-and-github>, <mailto:ietf-and-github-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf-and-github/>
List-Post: <mailto:ietf-and-github@ietf.org>
List-Help: <mailto:ietf-and-github-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf-and-github>, <mailto:ietf-and-github-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 20 Mar 2020 04:51:40 -0000

In case folks here are interested.

The toolchain I maintain (https://github.com/martinthomson/i-d-template) now has a better backup for issues and pull requests.

If you have CI enabled, this creates a snapshot of the issues and pull requests (thanks to Mike Bishop for navigating the API here) and saves a copy alongside the HTML and text versions of the draft.  This archive is a JSON file that is pretty easy to read, even without tools.  Saving to the repo means that anyone who has a copy of the repo will have a copy of this archive.

The archive is only created when changes are made to the repo, though that has been OK in the past, there are plans to run the update on a timer instead.

You can see an example of a *large* database with the QUIC repo.  This takes a fair bit to download - it's a 19M database uncompressed - and it can be a little slow to sort and do other things.  But it should give you an idea of what is possible.  It might not be as fully featured as the real GitHub interface, but I find that it can be useful for some things.

https://quicwg.org/base-drafts/issues.html

Cheers,
Martin

v2.23.0 | Report a Bug | By Email