Re: [Sidrops] trying to limit RP processing variability

[Martin Hoffmann <[e-mail]>]

Robert Kisteleki wrote:
> 
> IMO an "RP has no obvious way to acquire missing objects" is not
> entirely true.
> 
> If, at the previous run, the RP fetched the relevant (now missing)
> object, then I see no reason to not use it again. Think of the
> previous run as an object a cache if you will: if you're looking for
> an object mentioned in the manifest, and you have it already (hash /
> name / etc. matches) then you can reuse it.

That is theoretical possible, but in practice you treat synchronising
and validation of repository content as two separate steps. I.e, before
you even start looking at a CA’s repository, you synchronize its
content. This is enshrined in the way both rsync and RRDP work: They
don’t update single files but entire directory trees all at once. This
step includes deleting objects that have been deleted on the server.

Since the complete RPKI repository has a hierarchical structure
following the rsync URIs of objects, many RP implementations keep the
objects in the file system only. This is in particular useful for
rsync: Just let rsync update the directory in place. An additional
bonus of this strategy is that you don’t need a fancy database.

You could, of course, concoct a mechanism that marks files for deletion
and only deletes them if they aren’t actually used in the next
validation run. But, considering that this thread is actually
subjected “trying to limit RP processing variability,” I am not sure
this is a good idea. There is a strong likelihood that different
strategies will behave slightly differently. If we really want to
come to a point where every RP implementation produces the same output
from given input, we need to defined simple rules that are easy to
implement in a wide range of circumstances.

Another consequence of doing this is that validation on a newly
deployed RP software differs from one that has been running for a
while. As a consequence, the datasets from two different caches
configured in routers differ. So now you even have difference between
caches running the same software.[0]

Kind regards,
Martin 

[0] Yes, with broken RRDP servers where snapshots differ from a
    sequence of deltas, this can happen too. We should perhaps also
    look into improving the robustness of RRDP.