Re: [Sidrops] Publication Point -> RP synchronization in bandwidth constrained environments (note for RRDP v2)

[Tim Bruijnzeels <[e-mail]>]

> On 9 Jun 2023, at 10:38, Ties de Kock <[e-mail]> wrote:
> 
> 
> 
>> On 8 Jun 2023, at 17:37, Job Snijders <[e-mail]> wrote:
>> 
>> On Thu, Jun 08, 2023 at 05:13:55PM +0200, Tim Bruijnzeels wrote:
>>> As for lessons learned for a future RRDP. There are a number of things
>>> that I think could be done. To just name a few:
>>> 
>>> - No new snapshot on every update
>>> 
>>> This is hard for the server. Writing a full snapshot of multiple MBs for
>>> every change adds up. I think it would be better to have snapshots every
>>> so often and use deltas to get to the current state
>>> 
>>> - Combine deltas
>>> 
>>> Let the server pre-calculate combined deltas from serial X to current.
>>> This saves the RPs downloading many files. It saves bandwidth in cases
>>> where files are updated multiple times (e.g. a CA published multiple
>>> ROAs as separate events).
>>> 
>>> - Binary format
>>> 
>>> We can have a DER encoded structures (since we do DER anyway) that can
>>> contain all the data. This saves a lot of XML and base64 overhead. I have
>>> not done the analysis but I think that the actual DER objects don't compress
>>> well - so we may not need compression if we do this.
>>> 
>>> - Pointers for stragglers?
>>> 
>>> Notification files should be short. This means that RPs that have not
>>> synced for a long time will have to do a full re-sync. What may help
>>> is that there are pointers included to longer notification files for
>>> those that need it.
>> 
>> Another thing that I'd consider to add to the list would be advisory
>> timing parameters in the notify file: A publication server in trouble,
>> could sorta 'increase' their capacity if the server could request RPs to
>> fetch less frequently.
> 

Currently RFC 8182 puts the limit at once per minute. I think it would be
good to have flexibility in rate-limiting rather than hard coding this for all.

> Do we need to do this in the RRDP protocol? Or could we maybe get close with
> HTTP headers?

I think headers would allow for quick reconfiguration on the HTTPS server
without the need to reconfigure/re-issue/update the back-end. This may also
help in cases where there is a need to change the rate limit, but there is
no updated content.

Tim


> 
> Kind regards,
> Ties