Re: [Sidrops] trying to limit RP processing variability

Stephen Kent <> Tue, 14 April 2020 17:45 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id A6ED73A0825 for <>; Tue, 14 Apr 2020 10:45:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.101
X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id uM4pPz3QrScQ for <>; Tue, 14 Apr 2020 10:45:55 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 435343A0829 for <>; Tue, 14 Apr 2020 10:45:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=a2048; t=1586886354; bh=jUuCYh905UUpDOnDDOSKAYsklEWukEl/VRxHy9W1NGo=; h=Subject:To:References:From:Date:In-Reply-To:From:Subject; b=M9QMiQNqjd/auAeWHxba97DVAap3jUoMm/4wkJKY4UYbwKcOUdK2BOrYLgvnV+INnXgpcHvSCB2D3QgDrQvcnoWkfjBUe04pM6VTOM4EObg/GeZy12J05KV/4+H8UFI1YyyoEiWARRZCagDGSJnil2nArvSmQz5B3Fv6hC/tPHUBFCe6Wxhc7vrmwmsbzkAQP2G68b2F2H/47SIjb4UzGN6h84w+vmu2KTppk8rxwNlAdL+lN5ZHdUx9oFyA/MdlufAhW5sIRTIJ1otQ+UhWE4ZtFqzVJ+68ejaLgUjyxtj54wpq86Q22NVq8swFiaMbHG5Qf0GRcDCz/OrsDgeAsQ==
X-YMail-OSG: 7KV5xkQVM1k3ICRjEuizHPfFBwl5F107ZcdbSxoftHnd69ol24tA59bHJLLh325 5IbwtSuFWoc7B5F_wy.ZwNnnk_rzBwwOKCqF9mxmmMgMY2ONuE_wKsEOAAjJfMmdPi9BUrg75oip VsxdM3SnTfCUw649eoLVg4enQdO3QW24otQKzpnVi22BSsuvnp4XPAtZ7TtwI9Sozevw6j9IepM_ dnbwEZ9RDqHE.lAuC.CdtMk385xjE2_yntkKKD1mJHM8X1wuRw6hEB.J5PQf4VqDUNm0fm_caUOH 0fp__FckSIWMy0_1BCPDM56WY06OBmEA4YfwSGO1ugBk_WbD.vALLzdw0c00Jsjx37SgoUpczL4H vEKBI2gQxRZ.J6coiZJ..6RuzwNkh3boNcQZdFwsAPvS3uF8xi.rdBqmxzUmO3b0E_lqSWzsg_nl wVNxOoTTweoK0WzmtGJIegGq8Xjkz.ax07bpHB_vmU6c3Q82.sGo36CaIWpvYUnH.GP722BugrXL jFXlSEtJ1ON2ZFxM7fmJUcJm5XzbkNH.GqhkwJ2BSJC0amm8sfwOb78CcrpT.W.SBD56ZQYpM5XH Uk4MxzOygODxFg8ZL7QA8Hb5s23PX8l428cvrbTWF4gvmY3RqVGQ.Dxo17o0zbjt51kA2peCFTtl Xa_z4Jz7dCl6sSj5w3VxVhjSRaMZm_y3IBmGjENaXwwyWFPkOsLmnmOTRL1TB1o1PucYPeUSsDKy cXPQUWT2nqtHXcOgsm54CfAzosIzOsEKPo5tfyZzXyfoDQ3HPfjXOQ1UV4Yyb92JvoqLu_oqoGdB TOy55xm5vpkcrh4hHhGM4DpOUCQR0MW.Ne6BWlohBSpOVMbFJ7e_Sr0B4tCSDV50Bid_l6hMauJ_ JR84801WwW_5vzdJjr7xEUfM8DB7DQ9MEVdiwRjl0QvtgGFyhNqVGdADeEEdveLb5wEMfn7e6EwM kpTnC7bovDKWkg1pWf4CBOwhdA0wCF.Qw2maZNBej41Iui66OorOOgjsnh5oJswOYb_D8BpRlk4c L25VrJgZP2WZjykwtYZoJqR_IckR4c3RT.OgtbR0Ci395lyuEmINNe7aluVdAS8m0NSa6JZIOWYq ws9qFpTUN3aPsVqlWjyo4lKR5CU5NuXloKLb6PymzL5TwxmrcoRUlg.eGOxpjjUGgM87uoIRA0ra luEwiNP_6AonIUF5U3yY7Jri0XSoFU2SzCG7EETnFQUnqLyNpUVCEKKRU6ChF8QmxwhgmYTgpGOj w6.3eHEE8xYp3z7puDXHiywhbbUJrRZ7JE067Abn4oDOjFc3H3iziN2cpr95HNp3.vN3c2pZISST hGjkswZqYruk6ZifCMyOw5x477y_HqYPwX.vJGrFnWLEsQNO975J2uH8QEKRh0OrZc4CkMcA1P9Y .akpR19Fv2LDTVSefVajb28sJFkvB0enUMFlo_YHgHf7R4eObELOsWqvTayhrBPeun.v2
Received: from by with HTTP; Tue, 14 Apr 2020 17:45:54 +0000
Received: by (VZM Hermes SMTP Server) with ESMTPA ID f78a07c9b3778e5e270e4d1c3a671c9d; Tue, 14 Apr 2020 17:45:51 +0000 (UTC)
To: Robert Kisteleki <>, "" <>
References: <> <> <> <> <>
From: Stephen Kent <>
Message-ID: <>
Date: Tue, 14 Apr 2020 13:45:50 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.7.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Mailer: WebService/1.1.15651 hermes Apache-HttpAsyncClient/4.1.4 (Java/11.0.6)
Archived-At: <>
Subject: Re: [Sidrops] trying to limit RP processing variability
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: A list for the SIDR Operations WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 14 Apr 2020 17:45:57 -0000

> Hi Steve,
>>> If, at the previous run, the RP fetched the relevant (now missing
>>> object, then I see no reason to not use it again. Think of the previous
>>> run as an object a cache if you will: if you're looking for an object
>>> mentioned in the manifest, and you have it already (hash / name / etc.
>>> matches) then you can reuse it.
>> I probably should have said that an RP views an object as "missing" if
>> the the object is not present in the RP's cache and cannot be retrieved
>> from the relevant PP. Because RPs retrieve objects only if the objects
>> have changed (newly added or updated), an RP would not be aware that an
>> object is not present at a PP if the object is present in the RP's
>> cache. I recall a famous quote about whether an object that has not been
>> updated makes any noise when it is deleted from a PP, or something like
>> that :-)
> rsync has a flag (--delete) where it literally syncs, ie. not only
> fetches new/updated objects but also removes the local ones that no
> longer exist remotely. (Either this, or some other processes have to do
> cleanup, otherwise the local copy accumulates objects eternally.)

I forgot about the --delete flag. But, if one chooses to make use of the 
flag, it requires caution. if an attacker deleted files and an RP 
removed instances of those files from the local cache, this is an easy 
DoS attack. One would have to see if there is a current manifest and if 
it lists files that have been deleted from the PP. Only if there is a 
current manifest, and the deleted files are also not present in the 
manifest, would it be safe to delete those files.

If one relies on examination of a current manifest to trigger deletion 
of cached files this problem would be avoided, and thus those files 
would not accumulate forever. maybe that's why I was not thinking about 
the flag.

> I believe the NLnetLabs routinator and the NCC's RPKI validator use
> --delete with rsync. Maybe this behaviour changes if/when not using
> rsync, I'm not sure.
>>> Of course it can be useful to check if it still exists in the PP, but it
>>> seems to me the only benefit is to detect that it is missing from there
>>> and perhaps warn the PP operator. Otherwise the RP has a hard time
>>> arguing "no idea what this is since it's not there!".
>> I think I agree about the utility of detecting an object that has gone
>> missing from a PP, when the RP has the object locally cached. But, in my
>> experience, RP software was not designed to detect this case.
> I believe the discussion here is about specifying how RP software should
> (or must) behave, so that their behaviour is consistent. IMO in order to
> achieve that, the behaviour around missing objects should be part of the
> specification.