[Tools-implementation] Fwd: Organization of the mail archives

Robert Sparks <rjsparks@nostrum.com> Tue, 15 September 2020 16:58 UTC

Return-Path: <rjsparks@nostrum.com>
X-Original-To: tools-implementation@ietfa.amsl.com
Delivered-To: tools-implementation@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 37E753A138F for <tools-implementation@ietfa.amsl.com>; Tue, 15 Sep 2020 09:58:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.678
X-Spam-Level:
X-Spam-Status: No, score=-1.678 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, HTML_MESSAGE=0.001, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=fail (1024-bit key) reason="fail (message has been altered)" header.d=nostrum.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RULyjLMROeAG for <tools-implementation@ietfa.amsl.com>; Tue, 15 Sep 2020 09:58:51 -0700 (PDT)
Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 90D4A3A1384 for <tools-implementation@ietf.org>; Tue, 15 Sep 2020 09:58:51 -0700 (PDT)
Received: from unescapeable.local ([47.186.30.41]) (authenticated bits=0) by nostrum.com (8.16.1/8.16.1) with ESMTPSA id 08FGwiTF089504 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 15 Sep 2020 11:58:47 -0500 (CDT) (envelope-from rjsparks@nostrum.com)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nostrum.com; s=default; t=1600189127; bh=o5X6xbyc3+JO3Z5cWechjqheG0b1wmH0HK8ARLyOIW4=; h=Subject:References:To:From:Date:In-Reply-To; b=sGtEJXPK9bCbRHd/o3VgxrzGTS9DBzt617JY5KZFCoQGfPD3x5n2xQrepRyBDKlkP Etn7/IdlMj4eEQGd7xUbjtT+OjcfTO7lJvRLCX9txMKebrjnoJADlJGVcEBgYEFf7w eqB+tuJcuO5cq56C7cm4gGsz985qTHbi1RfrPZvI=
X-Authentication-Warning: raven.nostrum.com: Host [47.186.30.41] claimed to be unescapeable.local
References: <8ec50ff6-471b-f698-5e61-b6034509e930@nostrum.com>
To: "tools-implementation@ietf.org" <tools-implementation@ietf.org>, ryan Cross <rcross@amsl.com>
From: Robert Sparks <rjsparks@nostrum.com>
X-Forwarded-Message-Id: <8ec50ff6-471b-f698-5e61-b6034509e930@nostrum.com>
Message-ID: <7e213352-f0ba-6a84-0d26-1d2cb669f5f5@nostrum.com>
Date: Tue, 15 Sep 2020 11:58:41 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.12.0
MIME-Version: 1.0
In-Reply-To: <8ec50ff6-471b-f698-5e61-b6034509e930@nostrum.com>
Content-Type: multipart/alternative; boundary="------------896B2D2FA8259A3DD3C4288E"
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/tools-implementation/VOS-FgW_QjqSe-uXUiIRU9Jfw7o>
Subject: [Tools-implementation] Fwd: Organization of the mail archives
X-BeenThere: tools-implementation@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Tools Implementation <tools-implementation.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tools-implementation>, <mailto:tools-implementation-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tools-implementation/>
List-Post: <mailto:tools-implementation@ietf.org>
List-Help: <mailto:tools-implementation-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tools-implementation>, <mailto:tools-implementation-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 15 Sep 2020 16:58:53 -0000

Continuing this in the context of today's conversation.

After the cleanup described below, we are/will-be left with:


/a/mailarch/data/archive as the _authoritative_ store.

/a/mailarch/data/mbox_archive/public as an automatically generated set 
of mbox files generated from the authoritative store.

Should we plan to move the first to be under /doc and the second to be 
under /derived?

RjS



-------- Forwarded Message --------
Subject: 	Organization of the mail archives
Date: 	Tue, 1 Sep 2020 12:30:56 -0500
From: 	Robert Sparks <rjsparks@nostrum.com>
To: 	Ryan Cross <rcross@amsl.com>, tools-implementation@ietf.org 
<tools-implementation@ietf.org>



After consulting with Ryan, I propose this disposition for the following 
email archive related directories:


>> Legacy Mailarchive /a/ietf A few very old mail archive scripts
This can simply be removed. The scripts are no longer used.
>> Email Mailarchive /a/isode ISODE IMAP server data store
>> Email Mailarchive /a/mailarch Mail Archive operations

These should stay where they are.

/a/mailarch/data/archive is the _authoritative_ store.

/a/mailarch/data/mbox_archive/public is an automatically generated set 
of mbox files generated from the authoritative store. That should be 
used to replace several things below.

>> Email Mailman /a/mailman Mailman operations
That has to remain (though we should consider what to do with it when we 
bring in Mailman3
>> Email Mailman /a/mailman/archives Mailman Legacy Archives
This has to remain - it is where mailman's internal archiving takes 
place - any lists using pipermail are kept here, so Legacy isn't really 
accurate above
>>
>> Email Postfix /a/postfix Postfix operations
This remains
>> FTP FTP/Website combined 
>> /a/www/ietf-ftp/concluded-wg-ietf-mail-archive ftp, rsync none NOT 
>> exposed by http
>>
These have all been imported, and are served through the mailarchive. 
These should be stashed. I personally don't think we should serve them 
in any way since they are available through the mailarchive. If they are 
served, they should clearly be in something I'd like to start calling 
the "museum".
>
>> FTP FTP/Website combined /a/www/ietf-ftp/ietf-mail-archive Contains 
>> text versions of mail archive ftp, rsync night-runner All web access 
>> is to the /a/www/ietf-mail-archive tree, which this is a copy of
This should go away as soon as ftp stops depending on it (and we had 
another conversation about removing it already).
>> Email Mailarchive /a/www/ietf-mail-archive Legacy mail archive
>> Email Mailarchive /a/www/ietf-mail-archive/text
>> Email Mailarchive /a/www/ietf-mail-archive/text-secure
>> Email Mailarchive /a/www/ietf-mail-archive/web
>> Email Mailarchive /a/www/ietf-mail-archive/web-old
>> Email Mailarchive /a/www/ietf-mail-archive/web-secure
>>
>> Again, has all of that been imported, and can all of this simply be 
>> removed (or stashed offline)?
All of this should be stashed. Everything here, with the possible 
exception of things in web-old have been imported into mailarchive. Ryan 
is looking at making sure those get imported. Once that's done, all of 
these should either be taken offline or put in the museum.

Are there any other puddles of email-archive related things that aren't 
in those directories?