Re: [Sidrops] notes on rsync --delete (rrdp withdraw), and garbage collection

Rob Austein <sra@hactrn.net> Mon, 16 November 2020 04:18 UTC

Return-Path: <sra@hactrn.net>
X-Original-To: sidrops@ietfa.amsl.com
Delivered-To: sidrops@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B43483A1060 for <sidrops@ietfa.amsl.com>; Sun, 15 Nov 2020 20:18:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HgSq7Xn6C7JW for <sidrops@ietfa.amsl.com>; Sun, 15 Nov 2020 20:18:07 -0800 (PST)
Received: from khatovar.hactrn.net (khatovar.hactrn.net [198.180.150.30]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 980C13A1056 for <sidrops@ietf.org>; Sun, 15 Nov 2020 20:18:07 -0800 (PST)
Received: from minas-ithil.hactrn.net (c-73-47-196-134.hsd1.ma.comcast.net [73.47.196.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nargothrond.hactrn.net", Issuer "Grunchweather Associates" (not verified)) by khatovar.hactrn.net (Postfix) with ESMTPS id 3A23F139A9; Mon, 16 Nov 2020 04:18:03 +0000 (UTC)
Received: from minas-ithil.hactrn.net (localhost [IPv6:::1]) by minas-ithil.hactrn.net (Postfix) with ESMTP id 359652019F8E25; Sun, 15 Nov 2020 23:18:07 -0500 (EST)
Date: Sun, 15 Nov 2020 23:18:07 -0500
From: Rob Austein <sra@hactrn.net>
To: Job Snijders <job@ntt.net>
Cc: sidrops@ietf.org
In-Reply-To: <20201030163827.GE34637@bench.sobornost.net>
References: <20201030163827.GE34637@bench.sobornost.net>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/27.1 Mule/6.0
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset="US-ASCII"
Message-Id: <20201116041807.359652019F8E25@minas-ithil.hactrn.net>
Archived-At: <https://mailarchive.ietf.org/arch/msg/sidrops/rAk9iyjD3fuPLie3DriFgldWPUk>
Subject: Re: [Sidrops] notes on rsync --delete (rrdp withdraw), and garbage collection
X-BeenThere: sidrops@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <sidrops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sidrops>, <mailto:sidrops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/sidrops/>
List-Post: <mailto:sidrops@ietf.org>
List-Help: <mailto:sidrops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sidrops>, <mailto:sidrops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Nov 2020 04:18:09 -0000

[Apologies for belated response, saw this while catching before
today's meeting (for some definition of "today")]

Job, I agree that flushing data solely because rsync or RRDP said to
do so is a bad idea, but I think that outlawing rsync --delete or RRDP
withdraw is going a bit overboard in the other direction.

The operative word here is "solely".  At least in my implementation,
rsync/RRDP deletion only remove data from the pre-validation cache;
there's a separate cache of previously-validated data, and I use both
of them when constructing the output of the current validation cycle.

End result is that we only gc objects that have both been deleted and
have failed to pass validation, which (still) seems about right to me.