Re: [Sidrops] Publication Point -> RP synchronization in bandwidth constrained environments (note for RRDP v2)

[Claudio Jeker <[e-mail]>]

On Thu, Jun 08, 2023 at 01:45:46PM +0200, Mikhail Puzanov wrote:
> Hi Job, all,
> 
> I think compression is probably the quickest way of mitigating the size problem:
> 
> Repeating some of your experiments:
> 
> tmp> ls -lh snapshot.xml*
> -rw-r--r--  1 mpuzanov  staff   202M Jun  8 12:44 snapshot.xml
> -rw-r--r--  1 mpuzanov  staff    89M Jun  8 12:58 snapshot.xml.bz2
> -rw-r--r--  1 mpuzanov  staff    90M Jun  8 13:01 snapshot.xml.gz
> -rw-r--r--  1 mpuzanov  staff   124M Jun  8 12:57 snapshot.xml.lz4
> -rw-r--r--  1 mpuzanov  staff    83M Jun  8 12:58 snapshot.xml.lzma
> 
> Even the good ol’ gzip can shrink RIPE NCC’s snapshot more than twofold.
> Also compression should take care of the repetitive parts, i.e. 
> XML tags, newlines, etc.
> 
> Extra 2 cents as an RP implementer: rpki-prover fetches every repository 
> in a separate OS process with constrained heap, constrained download size 
> and a timeout, so it pretty much ignores all CVE-2021-43174-related 
> considerations and compression was never disabled in it. If the fetching 
> process crashes or runs out of some resource the impact is limited to 
> the specific repository. So there is a way to have compression and 
> tolerate crashes.

The inherent problem of RRDP is that when a CA is forced to issue a new
RRDP session (or all RP lose their RRDP state for the CA) every RP system
will grab a snapshot. So you end up with a thundering herd and I doubt a
reduction by half is enough to make that effect go away. 

The worst case RRDP bandwith requirements are significantly bigger then
what is needed in the steady state. This is an inherently bad design.
The only fix is to overprovision by a lot.

-- 
:wq Claudio