Re: [Sidrops] I-D Action: draft-ietf-sidrops-prefer-rrdp-00.txt

Hi all,

Following a discussion in the chat during the SIDROPS session, I think it is
better to take this to the list.

Currently, RRDP clients could (RRC8182, 3.4.5) fall back to alternative
repository access mechanisms (i.e. rsync) when there were any issues when
retrieving RPKI data from an RRDP repository server.

If this will change into a MUST without considering the timing of the fallback,
this will have an impact on the availability of rsync repositories. Let me build
the case of why RRDP clients falling back to rsync will act as a thundering herd
and will cause fallback to fail, and how we can mitigate that.

We observe that rsync clients update infrequently (~hourly) and RRDP clients
update frequently. The majority of clients in our logs check notification.xml
every 10 minutes. We also see that HTTP failures are temporally correlated,
clients detect a failure (in CDN or backend) at the same time. From this, it
follows that clients will observe a failure at the same time and will fall back
at the same time.

Scaling rsync for the total number of relying party instances, when spread out,
is achievable, even for a large repository. However, scaling rsync so it is
available when all clients fallback in a short period (we see ~1170 unique ips
retrieve notification.xml in a 10 minute window) is difficult.

Fallback will happen rarely, which implies that their copy of the repository is
potentially (very) outdated. A clean copy from rsync takes ~10-15 CPU-seconds
per client (and potentially a large number of IOPS hitting the backing storage)
on the server-side.

That is why I would like the draft to include text that recommends that fallback
is spread out. I would propose that when RRDP is failing (due to content or
connectivity), the clients retry RRDP until a set time at which they fall back
to rsync.

For a repository operator, it is optimal if all clients are spread out over
time. For fallback, this means that a client keeps trying rrdp for a
random delay from the uniform distribution of 0...rsync_update_interval after
the first failure before falling back to rsync.

The alternative is that, when clients act like a thundering herd, the rsync
repositories will not be available for an extended period. If clients retry at a
very similar interval, the repositories will be overloaded for an extended time.

Please let me know what you think of this mechanism,

Ties

> On 23 Feb 2021, at 08:44, Tim Bruijnzeels <tim@nlnetlabs.nl> wrote:
> 
> Dear working group,
> 
> I have asked for 15 minutes on this subject at the upcoming WG session.
> 
> Differences in this version:
> - rename 'deprecate-rsync' -> 'prefer-rrdp' - as this is the first goal (phases 0-1-2)
> - discuss deprecate rsync as a post-phase-2 goal
> - start discussion of what would be needed to have transport agnostic names (no rsync in uris)
> - moved implementation status overview to the end
> 
> 
> Regards,
> Tim
> 
> 
> 
>> On 23 Feb 2021, at 00:45, internet-drafts@ietf.org wrote:
>> 
>> 
>> A New Internet-Draft is available from the on-line Internet-Drafts directories.
>> This draft is a work item of the SIDR Operations WG of the IETF.
>> 
>>       Title           : Resource Public Key Infrastructure (RPKI) Repository Requirements
>>       Authors         : Tim Bruijnzeels
>>                         Randy Bush
>>                         George Michaelson
>> 	Filename        : draft-ietf-sidrops-prefer-rrdp-00.txt
>> 	Pages           : 13
>> 	Date            : 2021-02-22
>> 
>> Abstract:
>>  This document formulates a plan of a phased transition to a state
>>  where RPKI repositories and Relying Party software performing RPKI
>>  Validation will use the RPKI Repository Delta Protocol (RRDP)
>>  [RFC8182] as the only mandatory to implement access protocol.
>> 
>>  The first objective is to make RRDP the preferred access protocol,
>>  and require rsync as a fallback option only.  This will greatly
>>  reduce the operational burden and concerns for RPKI repository
>>  operators.
>> 
>>  In phase 0, today's deployment, RRDP is supported by most, but not
>>  all Repositories, and most but not all RP software.
>> 
>>  In the proposed phase 1 RRDP will become mandatory to implement for
>>  Repositories, in addition to rsync.  This phase can start as soon as
>>  this document is published.
>> 
>>  Once the proposed updates are implemented by all Repositories phase 2
>>  will start.  In this phase RRDP will become mandatory to implement
>>  for all RP software, and rsync will be required as a fallback option
>>  only.
>> 
>>  It should be noted that although this document currently includes
>>  descriptions and updates to RFCs for each of these phases, we may
>>  find that it will be beneficial to have one or more separate
>>  documents for these phases, so that it might be more clear to all
>>  when the updates to RFCs take effect.
>> 
>>  Furthermore, this document currently includes an early discussion of
>>  a future objective, which would be to change the RPKI standards such
>>  that names in RPKI objects are no longer tightly coupled to rsync.
>>  By using transport independent names and validation, we will obtain
>>  the agility needed to phase out rsync altogether and/or introduce
>>  other future access protocols.
>> 
>> 
>> The IETF datatracker status page for this draft is:
>> https://datatracker.ietf.org/doc/draft-ietf-sidrops-prefer-rrdp/
>> 
>> There are also htmlized versions available at:
>> https://tools.ietf.org/html/draft-ietf-sidrops-prefer-rrdp-00
>> https://datatracker.ietf.org/doc/html/draft-ietf-sidrops-prefer-rrdp-00
>> 
>> 
>> Please note that it may take a couple of minutes from the time of submission
>> until the htmlized version and diff are available at tools.ietf.org.
>> 
>> Internet-Drafts are also available by anonymous FTP at:
>> ftp://ftp.ietf.org/internet-drafts/
>> 
>> 
>> _______________________________________________
>> Sidrops mailing list
>> Sidrops@ietf.org
>> https://www.ietf.org/mailman/listinfo/sidrops
> 
> _______________________________________________
> Sidrops mailing list
> Sidrops@ietf.org
> https://www.ietf.org/mailman/listinfo/sidrops