Re: [Sidrops] notes on rsync --delete (rrdp withdraw), and garbage collection

George Michaelson <ggm@algebras.org> Mon, 16 November 2020 04:23 UTC

Return-Path: <ggm@algebras.org>
X-Original-To: sidrops@ietfa.amsl.com
Delivered-To: sidrops@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C92DA3A104B for <sidrops@ietfa.amsl.com>; Sun, 15 Nov 2020 20:23:15 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=algebras-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mH98CL0w0wzv for <sidrops@ietfa.amsl.com>; Sun, 15 Nov 2020 20:23:13 -0800 (PST)
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 42D883A104A for <sidrops@ietf.org>; Sun, 15 Nov 2020 20:23:12 -0800 (PST)
Received: by mail-lf1-x12e.google.com with SMTP id f11so23318805lfs.3 for <sidrops@ietf.org>; Sun, 15 Nov 2020 20:23:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=algebras-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=TytzL8Vy9bss4/HcnEg4zthqdh20yrYQP0OGpG7mQbs=; b=vhiSskgwHj63U1DTWpxql2ob1U8IGwOazRs23mJzKxA+jsIsvnMh7hIm3Db0qtSHZh bi9KxS1Ru5Z2FyoKX33heOF4eSst7E/aSKoPGdHUSLilPGq2lACOnKhUuFK59QwsH53q AjtRstSSEjjVh1hVyxzHHvpbd2q6/nN+QGhMIknxytXVtsbr9960dDYeEBYmKKaJmrVf HGl+fRoc7kxm0diX8s6LQKw4tEtGyn9gSV4lJCJIVYk24/4bcd7Qo82yOefMe2WkfWD8 45rPHJtcxkqeJvn5DS5zgdLCUk+j7alifkvCLMZBjmvH1cOoKptwjFZ5/UeYFUH3zjdN mwrg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TytzL8Vy9bss4/HcnEg4zthqdh20yrYQP0OGpG7mQbs=; b=szA25cMjIP+k72Rc6M/h7BSTy+7WpKQ7kVHPUnW9yWxVZRPzH1Xi/JqOWgIafUM7NN bnWpQ0gLNopPMYERgwfnDqJx84vU6lhjUqGj/FbicOPR79sEyKAVP8rNHoHisBz037SA aRtrqQewPL2HMgba3+cMESoYJovqRrlUD/9InYDdIDuXSX5qLk86sgTqxIRtM67xou3U PRH5R+HjWm/qdBxul1cmT47LKGJTJHAl6wK09zgLTkszp3l4zPivOBwaJ/SyEZFjqOZg Cz7YJljhew+LFgX0sc5jJKOkGpl2Vf+Nk/GbZKiq6tlwcgnHkHMlbgNA1RrS9s2/VsBP Dt1w==
X-Gm-Message-State: AOAM533foUc85+u2Aka0wUa6V+6/7ZyC+b6VmtsCHunp/gKXqF/y2JLs T7ZTuq+PY+mTbU8HUSHjgudTj8yaN1yQDjD4+6FJwv5FvLU=
X-Google-Smtp-Source: ABdhPJznaadRCnwMkG8hf2zvj1PId2mSH0enErTpL0OnYkRmsdzjEdXQiwxyupgvgu4xLl3CLDJVaw++lBg0vXWJ+Zc=
X-Received: by 2002:a19:48d4:: with SMTP id v203mr745667lfa.169.1605500590465; Sun, 15 Nov 2020 20:23:10 -0800 (PST)
MIME-Version: 1.0
References: <20201030163827.GE34637@bench.sobornost.net> <20201116041807.359652019F8E25@minas-ithil.hactrn.net>
In-Reply-To: <20201116041807.359652019F8E25@minas-ithil.hactrn.net>
From: George Michaelson <ggm@algebras.org>
Date: Mon, 16 Nov 2020 14:22:59 +1000
Message-ID: <CAKr6gn0jdV3Q+EYbtbnrXG0QS3GBfWKq6tdysNOFFiSGmHQigw@mail.gmail.com>
To: Rob Austein <sra@hactrn.net>
Cc: Job Snijders <job@ntt.net>, SIDR Operations WG <sidrops@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/sidrops/HZF5cVid2ql2pmlDOlUsyDXBBeE>
Subject: Re: [Sidrops] notes on rsync --delete (rrdp withdraw), and garbage collection
X-BeenThere: sidrops@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <sidrops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sidrops>, <mailto:sidrops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/sidrops/>
List-Post: <mailto:sidrops@ietf.org>
List-Help: <mailto:sidrops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sidrops>, <mailto:sidrops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Nov 2020 04:23:16 -0000

I like this behaviour, and I continue to use your validator for the
enhanced debugging it gives on the states of the repo.

I could quibble about colour marking yellow on some stuff, but
overall, "I like your approach"

-G

On Mon, Nov 16, 2020 at 2:18 PM Rob Austein <sra@hactrn.net> wrote:
>
> [Apologies for belated response, saw this while catching before
> today's meeting (for some definition of "today")]
>
> Job, I agree that flushing data solely because rsync or RRDP said to
> do so is a bad idea, but I think that outlawing rsync --delete or RRDP
> withdraw is going a bit overboard in the other direction.
>
> The operative word here is "solely".  At least in my implementation,
> rsync/RRDP deletion only remove data from the pre-validation cache;
> there's a separate cache of previously-validated data, and I use both
> of them when constructing the output of the current validation cycle.
>
> End result is that we only gc objects that have both been deleted and
> have failed to pass validation, which (still) seems about right to me.
>
> _______________________________________________
> Sidrops mailing list
> Sidrops@ietf.org
> https://www.ietf.org/mailman/listinfo/sidrops