Re: [Sidrops] Publication Point -> RP synchronization in bandwidth constrained environments (note for RRDP v2)

[Job Snijders <[e-mail]>]

Dear all,

On Thu, Jun 08, 2023 at 01:45:46PM +0200, Mikhail Puzanov wrote:
> I think compression is probably the quickest way of mitigating the
> size problem:

Yes. Support for gzip and deflate HTTP Content-Encoding has now been
added in OpenBSD rpki-client and will be available in a next release.

> Even the good ol’ gzip can shrink RIPE NCC’s snapshot more than
> twofold.  Also compression should take care of the repetitive parts,
> i.e.  XML tags, newlines, etc.

And as a nice bonus: compression (unsurprisingly) reduces the time spend
waiting for the network.

> Extra 2 cents as an RP implementer: rpki-prover fetches every
> repository in a separate OS process with constrained heap, constrained
> download size and a timeout, so it pretty much ignores all
> CVE-2021-43174-related considerations and compression was never
> disabled in it. If the fetching process crashes or runs out of some
> resource the impact is limited to the specific repository. So there is
> a way to have compression and tolerate crashes.

In the rpki-client implementation a similar approach is used:
establishing RRDP TLS connections, and HTTP decompression happens in a
dedicated subprocess [1] which runs in a restricted-service operating
mode using pledge() [2].

The fetches are timeboxed and size constrained (both in original and
inflated form). Inflation happens in a streaming fashion, so even if
HTTP Chunked transfer encoding is used, the size is checked on the fly
and to impose the constraint. Special attention needs to be taken to
stop the remote side from continuing to send data after Z_STREAM_END.

Kind regards,

Job

[1]: https://sha256.net/privsep.html
[2]: https://man.openbsd.org/pledge.2