Re: [Sidrops] trying to limit RP processing variability

Robert Kisteleki <> Tue, 14 April 2020 08:54 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 1D21F3A0815 for <>; Tue, 14 Apr 2020 01:54:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.5
X-Spam-Status: No, score=-0.5 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id kl6Dz9rKsorN for <>; Tue, 14 Apr 2020 01:54:26 -0700 (PDT)
Received: from ( [IPv6:2001:67c:2e8:11::c100:1372]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 14D103A0814 for <>; Tue, 14 Apr 2020 01:54:26 -0700 (PDT)
Received: from ([]) by with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.92.3) (envelope-from <>) id 1jOHKp-0005TY-Nz; Tue, 14 Apr 2020 10:54:23 +0200
Received: from ([2001:67c:2e8:9::c100:14e6] helo=[IPv6:2001:67c:2e8:1200::400]) by with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92.3) (envelope-from <>) id 1jOHKp-0002tI-LP; Tue, 14 Apr 2020 10:54:23 +0200
To: Stephen Kent <>, "" <>
References: <> <> <> <>
From: Robert Kisteleki <>
Autocrypt:; prefer-encrypt=mutual; keydata= xsBNBEzFa6gBCADVASYXBbUF7v1D+Y9XR41SEEMiZUARlUWeP0NrFHZmRRGdR5nM/p6HguUd StIPRmdqMdyLDqBsV8XPVu6lvhcb4+ZFu/V1XFPVyPBH8U6iQ4PdGDeqFlBm3gxoDOGraGw8 bjojvASTz/Wk3ddLPm34Kb6oMI2MclC016UgrPgIj6A1Uu8qQeBDyWrk+OrWUPOUOKM7QhQg cpU4JwuaesthFvqdoPNQJi9QUfn94r14ZNDYmeJlchZiRHWO70Gwoy3ywfAM9Kyi1tx78Qc9 E5ZhGIw9qqlzqa6c6a0qhup2Zh/dhVBJ05jCDN7bUQT5tRiOV2icyX8Dsr4KaWYCsAOVABEB AAHNMVJvYmVydCBLaXN0ZWxla2kgKFJJUEUgTkNDIGtleSkgPHJvYmVydEByaXBlLm5ldD7C wHgEEwECACICGyMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheABQJWUwoeAAoJEC0ZXiKtTC3+ 04UH/jlvSR0esDGFSponUVawru+/QF61KdsNrdH6/Vs2buQvczW2Uh+S6Dic2vr2H0B1YrvL F2XpL2WJUHBUDLTA7dYTslvnHpyZrR8Sfb+h+wJ8OynxEC5wMKxfYNx2fMSk5EIU5mRjMaYg X/VkssDcoQAznNwVVYeqHYUJDMcrJhAYh44VHO208VwjPjHUDRlC+BoMGjHJnWDOAstlES8j 0r3adj2MqIHdDEjSdEx1+rbV0iZlgcDbYDex3qulOYlcZL+PJvGHzD6CkNBa8SbSN7cO0yqR OJ2sgobITOJ0GbRIbIvkUe1Iqw717CuQV/u822dFISDYOAhGYmfWGJWmkezOwE0ETMVrqAEI AKazZ2Agrv0nNFPWV69l6fEout/FaqWfyAG5V414l4yr+qVShUYzS+txA2vC+ouHvdORZ/JG xwKf6HE+YvvWS+Oa+b6h+GZfA3G43XGpQlxXrFK019TeMjhHqWprZALL4w2k6TatYT1ZW369 rORtwSgtn5ZC4uNcpZeDQddQvCjyYoknqlZqAFf1pssuGPTE8GvhrZGEp52dALYYoDIf7y/z 8fCAcy72rhMhQV02rPB49UxOEh2FZJhST0743tuMtFemBkp06B/Mcx54QT0muG8zj19oMDG3 AAaGjNP6B3qzR6F8VczR/qVhQzRvNMr8A6+y/ew/x4+48P+O/4n/I50AEQEAAcLAXwQYAQIA CQIbDAUCVlMKHgAKCRAtGV4irUwt/mvlB/sFID7mlsWAS66UyrI+tGs4Xfl59vvhRRZ4ZKiR 8VEbWbLKh/b9SoYcKt9SLEfVxJE5ebWPgIIvUSdLS6f4n9uAJteDZ4w/AVfp5a6jbfvMm7JP AMW4HtnZ3YbNevRgXdGVXN+bTLZzXoVijOKu+xHDBRNaUswaG3glrDJfUGkPQtCXFn6m6Pdw dW1/ShzwQgfuE/NXa83jhJ175P+NoQ2KG7934vu2MZdrtIqPibKuaGWMPG0L5YzPotK9ONmd taJMnuk92qqZ6S9JPwRZmogRW/sX54XvGg6RzNpdHS5C+iN01tCNJTRTlOJ1X73+RrGokvKc dp6fdfc4PHHhpcMd
Organization: RIPE NCC
Message-ID: <>
Date: Tue, 14 Apr 2020 10:54:23 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.7.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="utf-8"
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
X-ACL-Warn: Delaying message
X-RIPE-Signature: 72e00e6d7601fa19264e98abc238a2747488cda607317ef40b1c7532a06f3463
Archived-At: <>
Subject: Re: [Sidrops] trying to limit RP processing variability
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 14 Apr 2020 08:54:29 -0000

Hi Steve,

>> If, at the previous run, the RP fetched the relevant (now missing
>> object, then I see no reason to not use it again. Think of the previous
>> run as an object a cache if you will: if you're looking for an object
>> mentioned in the manifest, and you have it already (hash / name / etc.
>> matches) then you can reuse it.
> I probably should have said that an RP views an object as "missing" if
> the the object is not present in the RP's cache and cannot be retrieved
> from the relevant PP. Because RPs retrieve objects only if the objects
> have changed (newly added or updated), an RP would not be aware that an
> object is not present at a PP if the object is present in the RP's
> cache. I recall a famous quote about whether an object that has not been
> updated makes any noise when it is deleted from a PP, or something like
> that :-)

rsync has a flag (--delete) where it literally syncs, ie. not only
fetches new/updated objects but also removes the local ones that no
longer exist remotely. (Either this, or some other processes have to do
cleanup, otherwise the local copy accumulates objects eternally.)

I believe the NLnetLabs routinator and the NCC's RPKI validator use
--delete with rsync. Maybe this behaviour changes if/when not using
rsync, I'm not sure.

>> Of course it can be useful to check if it still exists in the PP, but it
>> seems to me the only benefit is to detect that it is missing from there
>> and perhaps warn the PP operator. Otherwise the RP has a hard time
>> arguing "no idea what this is since it's not there!".
> I think I agree about the utility of detecting an object that has gone
> missing from a PP, when the RP has the object locally cached. But, in my
> experience, RP software was not designed to detect this case.

I believe the discussion here is about specifying how RP software should
(or must) behave, so that their behaviour is consistent. IMO in order to
achieve that, the behaviour around missing objects should be part of the