IETF Logo Mail Archive

Re: [Tools-discuss] Discontinuing large text dumps

worley@ariadne.com Thu, 08 June 2023 18:31 UTC

Return-Path: <worley@alum.mit.edu>
X-Original-To: tools-discuss@ietfa.amsl.com
Delivered-To: tools-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB412C151063 for <tools-discuss@ietfa.amsl.com>; Thu, 8 Jun 2023 11:31:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.981
X-Spam-Level:
X-Spam-Status: No, score=-0.981 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_SOFTFAIL=0.665, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=comcastmailservice.net
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mjUqgYlJm0ep for <tools-discuss@ietfa.amsl.com>; Thu, 8 Jun 2023 11:31:05 -0700 (PDT)
Received: from resqmta-a1p-077436.sys.comcast.net (resqmta-a1p-077436.sys.comcast.net [96.103.146.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DDF48C14F738 for <tools-discuss@ietf.org>; Thu, 8 Jun 2023 11:31:04 -0700 (PDT)
Received: from resomta-a1p-076784.sys.comcast.net ([96.103.145.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 256/256 bits) (Client did not present a certificate) by resqmta-a1p-077436.sys.comcast.net with ESMTP id 7ESZqPMpi7g0F7KNZqvPjS; Thu, 08 Jun 2023 18:29:01 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcastmailservice.net; s=20211018a; t=1686248941; bh=pmpXRmwy3Cs61p0UIqWRKMxWerNpPiTzRstyxceX2z8=; h=Received:Received:Received:Received:From:To:Subject:Date: Message-ID:Xfinity-Spam-Result; b=p+igue/KrYfOb8Mg0ms4NCOfM36OaZ4lj9BfK+FplmJmx4tUSXFnDRLQhqBl38ciB ug0/85GfZdMV1hg0SVrITaCgn7r1/G3zEIjNHK3b8/nHJuhcAb6ZKBJOhrIJRBjLP9 /K/Pd8s2zroGVZRihh1p/7U13uT1tMmpJsj6FFc08YYk5n5xt2cvw84pRIkJ1wgT94 xxyrK/ScteKXmBwNksLx+6jvm3lMaXHuerZI93HkUiRC87Ixl+4vDwz3TVDq7C/a/F L0VTHRnEKLs8h4Eo36ySZayBFEIVp6qBF0wY9IibXgEKBgRBvXEBxOpYVYf7xOdKb9 sUsZPuZeLNrMQ==
Received: from hobgoblin.ariadne.com ([IPv6:2601:192:4a00:430::3fdd]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 256/256 bits) (Client did not present a certificate) by resomta-a1p-076784.sys.comcast.net with ESMTPA id 7KNDqu70n8JBE7KNEq14Fk; Thu, 08 Jun 2023 18:28:40 +0000
X-Xfinity-VMeta: sc=-100.00;st=legit
Received: from hobgoblin.ariadne.com (localhost [127.0.0.1]) by hobgoblin.ariadne.com (8.16.1/8.16.1) with ESMTPS id 358IScjM365363 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Thu, 8 Jun 2023 14:28:38 -0400
Received: (from worley@localhost) by hobgoblin.ariadne.com (8.16.1/8.16.1/Submit) id 358IScZk365360; Thu, 8 Jun 2023 14:28:38 -0400
X-Authentication-Warning: hobgoblin.ariadne.com: worley set sender to worley@alum.mit.edu using -f
From: worley@ariadne.com
To: Robert Sparks <rjsparks@nostrum.com>
Cc: tools-discuss@ietf.org
In-Reply-To: <c6cf8db7-dd8d-6fb7-3bb0-e1fce8e3f958@nostrum.com> (rjsparks@nostrum.com)
Sender: worley@ariadne.com
Date: Thu, 08 Jun 2023 14:28:38 -0400
Message-ID: <87bkhpu7u1.fsf@hobgoblin.ariadne.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/tools-discuss/5bJM1Jmjm1qsjP5ZlKKBdvwvJKU>
Subject: Re: [Tools-discuss] Discontinuing large text dumps
X-BeenThere: tools-discuss@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: IETF Tools Discussion <tools-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tools-discuss>, <mailto:tools-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tools-discuss/>
List-Post: <mailto:tools-discuss@ietf.org>
List-Help: <mailto:tools-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tools-discuss>, <mailto:tools-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2023 18:31:09 -0000

Robert Sparks <rjsparks@nostrum.com> writes:
> We have several pages that have existed for some time that are _very_ 
> expensive to serve, and are not frequently accessed.

> This has been discussed repeatedly with leadership during tools team 
> meetings (see 
> https://notes.ietf.org/tools-team-20230314#Stop-serving-text-dumps---Robert) 
> for the most recent.

*If* I read those notes correctly, it seems like these files are
generated dynamically.  It's not clear to me exactly how dynamically; if
you fetch one by HTTP, it might be generated immediately, but I fetch
1id-{index,abstracts}.txt via rsync and it's not clear to me that that
can trigger dynamic generation.

If that's true, I'm quite surprised.  I would expect that they would be
generated once a day, or once a week, or something like that.

OTOH, I don't think I ever have used any of those files.

Dale

v2.23.0 | Report a Bug | By Email