[Sidrops] Re: draft-li-sidrops-rpki-repository-problem-statement-00- reliability issues

[Job Snijders <[e-mail]>]

On Mon, Oct 14, 2024 at 07:05:54PM +0800, Yingying Su wrote:
> > 63 minus 9 minus 1 is 53. How can 60 be hosed in a single AS? Which AS
> > is this? Is there a calculation error? The above observation does not
> > match with my own datasets.
> 
> I would like to clarify my earlier statement regarding the distribution of
> repository instances. When I mentioned that 60 out of 63 repositories are
> hosted in a single AS, I meant that there are multiple ASes hosting these
> repositories, but there are 60 instances hosted in only one AS. For instance,
> if repository A hosted in AS1, B hosted in AS2, and C hosted in AS4 and AS3,
> then it could be stated that two repositories (A and B) are hosted in a single AS.

Thank you for the clarification. So you are saying that the repositories are already
almost maximally distributed across different Autonomous Systems - sounds good no?

> It is crucial to recognize that as an essential RPKI data resource storage system,
> the accessibility of the RPKI repository node should not depend solely on a single
> AS. The resources stored within these repository nodes play a key role in ensuring
> the reachability of ASes. 

An incredible number of ultra reliable Internet services use on a single AS. It does
not automatically follow that the use of a single ASN is bad for reliablity.

Additionally, already today various publication point operators host
their RRDP service in a different AS than their RSYNC service.

Example: RIPE NCC hosts their RSYNC server in AS 3333, APNIC hosts its
RSYNC service in AS 4608. Both RIPE NCC & APNIC use a CDN (in a different
AS) for RRDP. If the CDN goes down, Relying Parties will simply switch to
RSYNC and follow entirely different network paths to synchronize the repository.

ARIN even went a step further: the RSYNC and RRDP services are hosted in AS 394018,
AS 393225, AS 393220. This is accomplished by using multiple A & AAAA records
pointing to different IPs originated by different ASes.

In your analysis I fail to see recognizition for the existing redundancy &
diversity in the existing iteration of the RPKI.

You use words like 'truly distributed' and 'robust' - but the existing system
literally is distributed (and as robust as people choose to make it).

RPKI over BitTorrent has been tried, and it was not a success. RPKI over IPFS
has been studied and was deemed to be impractical.

> > Yes, problems occur in the real world. Then again, temporal inability to
> > synchronize with a given publication service does not immediately render
> > previously fetched & cached objects invalid. As per RFC 9286, Relying
> > Party implementation should use locally cached objects until such time
> > they become stale. In practise this means there is time budget for
> > publication point outages.
> 
> While local caching can indeed alleviate the impact of temporary repository
> node outages, it is only a short-term solution. Once the cached objects expire,
> RPs may find themselves unable to access updated RPKI data, potentially leading
> to ROV failures. Therefore, caching mechanisms alone do not adequately address
> the risks associated with PP failures, especially during extended outages of 7,
> 10 hours, or even longer. 
> 
> I think that while caching can serve as a temporary measure, the RPKI repository
> requires a more robust architecture to guarantee high availability and redundant
> storage solutions, rather than depending exclusively on caching and time allowances
> to resolve these issues.

Caching is more than a 'temporary measure'. Caching is an essential component of a
sane relying party implementation, because without cache the RP is doing network
reads all the time which would be costly.

While we might not agree on the problem statement at hand, I am curious what solution
direction you have in mind. Especially a cost/benefit analysis would be nice: who
pays of the redundant storage and potential increase in bandwidth usage?

Do you have a short summary of what you'd want to change to RRDP? Or do you have
something entirely different in mind?

Kind regards,

Job