Re: [Sidrops] notes on rsync --delete (rrdp withdraw), and garbage collection

George Michaelson <> Mon, 16 November 2020 04:23 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C92DA3A104B for <>; Sun, 15 Nov 2020 20:23:15 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id mH98CL0w0wzv for <>; Sun, 15 Nov 2020 20:23:13 -0800 (PST)
Received: from ( [IPv6:2a00:1450:4864:20::12e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 42D883A104A for <>; Sun, 15 Nov 2020 20:23:12 -0800 (PST)
Received: by with SMTP id f11so23318805lfs.3 for <>; Sun, 15 Nov 2020 20:23:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=TytzL8Vy9bss4/HcnEg4zthqdh20yrYQP0OGpG7mQbs=; b=vhiSskgwHj63U1DTWpxql2ob1U8IGwOazRs23mJzKxA+jsIsvnMh7hIm3Db0qtSHZh bi9KxS1Ru5Z2FyoKX33heOF4eSst7E/aSKoPGdHUSLilPGq2lACOnKhUuFK59QwsH53q AjtRstSSEjjVh1hVyxzHHvpbd2q6/nN+QGhMIknxytXVtsbr9960dDYeEBYmKKaJmrVf HGl+fRoc7kxm0diX8s6LQKw4tEtGyn9gSV4lJCJIVYk24/4bcd7Qo82yOefMe2WkfWD8 45rPHJtcxkqeJvn5DS5zgdLCUk+j7alifkvCLMZBjmvH1cOoKptwjFZ5/UeYFUH3zjdN mwrg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TytzL8Vy9bss4/HcnEg4zthqdh20yrYQP0OGpG7mQbs=; b=szA25cMjIP+k72Rc6M/h7BSTy+7WpKQ7kVHPUnW9yWxVZRPzH1Xi/JqOWgIafUM7NN bnWpQ0gLNopPMYERgwfnDqJx84vU6lhjUqGj/FbicOPR79sEyKAVP8rNHoHisBz037SA aRtrqQewPL2HMgba3+cMESoYJovqRrlUD/9InYDdIDuXSX5qLk86sgTqxIRtM67xou3U PRH5R+HjWm/qdBxul1cmT47LKGJTJHAl6wK09zgLTkszp3l4zPivOBwaJ/SyEZFjqOZg Cz7YJljhew+LFgX0sc5jJKOkGpl2Vf+Nk/GbZKiq6tlwcgnHkHMlbgNA1RrS9s2/VsBP Dt1w==
X-Gm-Message-State: AOAM533foUc85+u2Aka0wUa6V+6/7ZyC+b6VmtsCHunp/gKXqF/y2JLs T7ZTuq+PY+mTbU8HUSHjgudTj8yaN1yQDjD4+6FJwv5FvLU=
X-Google-Smtp-Source: ABdhPJznaadRCnwMkG8hf2zvj1PId2mSH0enErTpL0OnYkRmsdzjEdXQiwxyupgvgu4xLl3CLDJVaw++lBg0vXWJ+Zc=
X-Received: by 2002:a19:48d4:: with SMTP id v203mr745667lfa.169.1605500590465; Sun, 15 Nov 2020 20:23:10 -0800 (PST)
MIME-Version: 1.0
References: <> <>
In-Reply-To: <>
From: George Michaelson <>
Date: Mon, 16 Nov 2020 14:22:59 +1000
Message-ID: <>
To: Rob Austein <>
Cc: Job Snijders <>, SIDR Operations WG <>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <>
Subject: Re: [Sidrops] notes on rsync --delete (rrdp withdraw), and garbage collection
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 16 Nov 2020 04:23:16 -0000

I like this behaviour, and I continue to use your validator for the
enhanced debugging it gives on the states of the repo.

I could quibble about colour marking yellow on some stuff, but
overall, "I like your approach"


On Mon, Nov 16, 2020 at 2:18 PM Rob Austein <> wrote:
> [Apologies for belated response, saw this while catching before
> today's meeting (for some definition of "today")]
> Job, I agree that flushing data solely because rsync or RRDP said to
> do so is a bad idea, but I think that outlawing rsync --delete or RRDP
> withdraw is going a bit overboard in the other direction.
> The operative word here is "solely".  At least in my implementation,
> rsync/RRDP deletion only remove data from the pre-validation cache;
> there's a separate cache of previously-validated data, and I use both
> of them when constructing the output of the current validation cycle.
> End result is that we only gc objects that have both been deleted and
> have failed to pass validation, which (still) seems about right to me.
> _______________________________________________
> Sidrops mailing list