Re: [Sidrops] trying to limit RP processing variability

[Stephen Kent <[e-mail]>]

Robert,
> Hi,
>
>> Missing: an object named in a Manifest, but not available for download
>> from a PP, is termed *missing*. An RP has no obvious way to acquire
>> missing objects, but operators SHOULD be warned about which objects are
>> missing.
> IMO an "RP has no obvious way to acquire missing objects" is not
> entirely true.
>
> If, at the previous run, the RP fetched the relevant (now missing)
> object, then I see no reason to not use it again. Think of the previous
> run as an object a cache if you will: if you're looking for an object
> mentioned in the manifest, and you have it already (hash / name / etc.
> matches) then you can reuse it.
I probably should have said that an RP views an object as "missing" if 
the the object is not present in the RP's cache and cannot be retrieved 
from the relevant PP. Because RPs retrieve objects only if the objects 
have changed (newly added or updated), an RP would not be aware that an 
object is not present at a PP if the object is present in the RP's 
cache. I recall a famous quote about whether an object that has not been 
updated makes any noise when it is deleted from a PP, or something like 
that :-)
> Of course it can be useful to check if it still exists in the PP, but it
> seems to me the only benefit is to detect that it is missing from there
> and perhaps warn the PP operator. Otherwise the RP has a hard time
> arguing "no idea what this is since it's not there!".
I think I agree about the utility of detecting an object that has gone 
missing from a PP, when the RP has the object locally cached. But, in my 
experience, RP software was not designed to detect this case.
> For bonus points: an implementation that fetched a PP's manifest,
> detected that it's exactly the same as before, and therefore reused the
> validation outcome from the previous run would not even have to fetch
> *any* other objects from the same PP.
If the manifest was not changed, at all, then it would not be fetched, 
right?
> The downside is that this process
> will not be able to point out the omission. The upside is saves a lot of
> resources (bandwidth, CPU and all). A further upside is that as a
> side-effect this protects against a malicious attacker (selectively)
> hiding objects.
If the only things that changed in the manifest were the updates and 
manifest number, then there would be no need to retrieve any additional 
objects, and with rsync I don't believe there would be any other file 
retrievals. I can't be sure, but I think thge BBN RPSTIR software 
behaved that way.  Do we get bonus points?
> Where this comes back to the current discussion is: would this behaviour
> be mandated, recommended, or considered a big no-no?

Which behavior? I would say recommended.

Steve