Re: [Sidrops] I-D Action: draft-ietf-sidrops-prefer-rrdp-01.txt

[George Michaelson <[e-mail]>]

On Tue, 7 Dec 2021, 8:36 pm Claudio Jeker, <[e-mail]> wrote:

> On Tue, Dec 07, 2021 at 06:29:33PM +1000, George Michaelson wrote:
> > On Tue, 7 Dec 2021, 6:18 pm Claudio Jeker, <[e-mail]>
> wrote:
> >
> > >
> > > There are some much larger rsync repositories out there then the one
> from
> > > RIPE NCC. I think the main issue with rsync is that there is only one
> real
> > > implementation of the server and there is no official specification of
> the
> > > protocol.
> > >
> >
> > Aside from the 5 RIR who do not directly produce roa, although they do
> > publish roa for hosted services in distinct paths what are these much
> > larger repositories? Do you mean APNIC or another RIR? Large to object
> size
> > or large to count or ... ?
>
> rsync is still used by various software distributions to sync large
> repositories. For example the OpenBSD CVS repository is made available via
> multiple rsync servers and at 7.7GB and 500'000 files it is a fair amount
> larger than any RIR repo.
>

So as job pointed out this is a generality about the applicability of rsync
to massive filestore synchronisation which is what Tridge designed it for.

servers by changing file timestamps in their repositories. This breaks
> rsync fast path.
>

To be clear, you must mean when there's been no change to filestate because
reissuance of files demands that be resigned and retimestamped.

Because of partial update problems a lot of people server side are now
doing republication by the near-atomic operation on the root dir of the
repository so that its an integral collection. Because of validation
failure states for partially updated state. It's possible the mechanistic
cheap ways of doing this accidentally change file and dir timestamps, worth
noting. We could try harder to avoid this.


> > > rsync has a few benefits over RRDP so I would be very careful about
> > > removing rsync support without fixing or adding an RRDP replacement.
> >
> >
> > It pays to be explicit. What are these benefits? The only one I have seen
> > discussed is the cost shifting to compute the object tree from deltas
> > (shifted cost to client side in RRDP) vs the complete structure in rsync
> at
> > a cost of per fetch difference calculation, server side.
>
> RRDP lacks integrity checks. If a file is for whatever resason missing on
> the RP then it will not be refetched for a very long time. Rsync has the
> benefit that missing files are refetched automatically. After a rsync run
> your repository is the same as the on the remote side and any local
> modification is detected and fixed. RRDP has nothing like that. It only
> carries a hash for delete and update operations but there is nothing that
> ensures that other files are actually what their supposed to be.
> So every file served via RRDP needs to be kept on the RP system even if it
> has nothing todo with the RPKI.
>

I think you're conflating issues here. RRDP is a delta protocol but
includes pointers, URI to every object. All rpki objects being signed I
don't understand the point about files not being what they are supposed to
be. If you mean and adversarial attack, remember the crypto is there.

But it is probably fair that neither rsync nor rrdp were designed for bad
actor risks. I think people are driving hard to attack RRDP but remember
we're here because of the trivially low bar to attacking rsync.


> There are also disadvantages. Rsync is inherently serialised. Http has at

> > least implicit parallelism in its transport protocol (HTTPS). And is CDN
> > friendly. Rsync CDN support is tcp anycast and that's about it. And the
> > ones you stated (lack of implementations, lack of ietf standard)
>
> RRDP is inherently serialised. You can not parallel process it. You can
> run many rsync servers on different IPs. There is no need for tcp anycast.
>

I think we're talking at cross purposes. Per rsync connection there is no
internal threading it's a single linear sequence of fetches. If you run two
rsyncs in parallel between some A and B you stomp on each others fetches.

But in principle (if not in practice) you can thread and multifetch the
stream of an RRDP. In any case snapshot aside you fetch less and you do not
do per file checking, the delta sync is a smaller total fetch size.

I don't think anyone has coded RRDP to exploit parallelism. So this is only
in hypothesis. But, unless you move to zsync or other modern takes on rsync
this is pretty hard to do.


I could run rsync as a CDN but there is probably not that much interest in
> doing that from the big CDN players.
>

As an RIR employee I can only agree. There is no first class support for
rsync as a clouded service. We asked. You have to build your own. You can
now embed rsync into services which do closest TCP provider mapping, but
there's no meaningful edge cache. APNICs AS0 is in cloudflare and there is
a significant RTT advantage in second fetcher because of the caching, which
came along for free in http cloud services.

Cheers

G

>