[Sidrops] Re: WG Last Call for draft-ietf-sidrops-publication-server-bcp

[Christopher Morrow <[e-mail]>]

On Wed, May 28, 2025 at 4:26 AM Tim Bruijnzeels <[e-mail]> wrote:
>
> Hi Chris, all,
>
> Thank you for your comments! Late comments are better than no comments. This is at
> least partly why we asked for Last Call, becuase this tends to motivate people ;)
>
> In any case, a quick update:
>
> - We incorporated Job's comments and made him co-author (he contributed significant
>   text earlier in the doc lifecycle as well)
> - We incorporated minor improvements in wording by Mike Hollyman
>
> We currently have a PR for the next document revision here:
> https://github.com/timbru/draft-sidrops-publication-server-bcp/pull/33
>

oh neat! I'll go have a read there...

> Of course, that is just transparency. Feel free to comment here on the list. If you
> comment on github we will make sure that the list is informed.
>
> In any case, the current edit buffer does not yet include your comments Chris, but
> we plan to work on this over the coming days.
>
> My proposal to the chairs would be do a new last call on -03 when it's ready, and
> addresses Chris's comments.
>

sounds ok to me, thnx!

> Tim
>
>
>
>
> > On 23 May 2025, at 19:35, Christopher Morrow <[e-mail]> wrote:
> >
> > owdy sidorops and authors of the subject document... I probably should
> > have made 1-2 of these next comments sooner :(
> > (incidentally I think the document will be useful!)
> >
> > I can also attempt to make a  github PR similar to job's once done
> > typing here.. I think there should be a bit more information /
> > guidance
> > provided to the operators of Publication Services around availability,
> > I think there are scenarios today where publication servers:
> >   * live behind a single ISP
> >       - limited redundancy/reslience to routing anomalies in that
> > provider (or their peers)
> >   * are deployed in a single geographical region (same town)
> >      - same power grid
> >      - same natural-disaster zone
> >   * live behind smaller than expected links
> >      - what is expected? 1 Gbps? 10Gbps?
> >      - link size vs utilization?
> >      - capacity planning concerns exist here - "Can service N
> > concurrent request streams?"
> >   * are a single entity
> >      - pub service is a single computer/server
> >      - inability to scale efficiently as demand rises
> >      - in ability to split services across domains (geographical /
> > network / power / etc)
> >      - prone to hardware enforced outages
> >
> > I think the idea that the RPKI system of publication services can be a
> > loose group of things sprinkled across the internet without a ton
> > of care about resilience of the system, because overall the relying
> > parties can always call back later if a particular Publication Service
> > is not currently available, has helped growth, experience, and
> > experimentation in the system. I don't think the system works well
> > long
> > term ini this model, because more users and more tight coupling to the
> > business of packet forwarding will require closer observation
> > of the RPKI system's artifacts. I think this will drive more thought
> > and effort into resiliency and redundancy, this document seems like
> > a good place to start that conversation. Some has been done in
> > RFC6481[0] for sure, but a slightly more structured and comprehensive
> > discussion would help Publication Service operators align with network
> > operations on requirements/expectations.
> >
> > If we look at the guidance in RFC6481 section 3 there is one pointer to:
> > " *  The publication repository SHOULD be hosted on a highly
> >         available service and high-capacity publication platform."
> >
> > This is a nice start, but does not provide guidance and reasoning for
> > that guidance, really.
> > I think being a bit more explicit about what that means and why it's
> > necessary would be good.
> > I think it's reasonable to build a simple model and drive the advice
> > from that, something like:
> >
> > * 20k ASN participating in the RPKI system
> > * An hourly update cadence for Relying Parties
> > * Average repository size of 1MBytes
> >
> > Missing repository information because of outages between the relying
> > party and the publication service MAY not be critical
> > immediately, but will lead to delays in the system and possibly
> > imapcts on the relying parties as they force age out content
> > and suffer delays in their workload collecting from the globally
> > distributed system. One view is that the delays are expected
> > because this is the internet and over time such outages will occur, so
> > we should be resilient in the total system to this, I think
> > this is the prevailing sentiment today... Some study about lengthy
> > outages and impact to the RPKI  and routing systems would
> > be super informative, I bet.
> >
> > Using wikipedia's 'how much outage is too much' page[1]  and the below
> > information might be handy in deciding how much
> > time can elapse between a Publication Service going offline for a
> > percentage of the Internet and when flags should go up.
> > As a starting stake in the ground I'd suggest 99.9 uptime seems like a
> > great start?
> >
> > When it comes to discussing capabilities and expectations about the
> > Publication Service itself, It would be expected for a
> > Publication Service to be able to service:
> >  * 20000/3600 == 6 full request sets per second
> >     - every ASN has 1 RP talking to every Publication Service every hour
> >     - of course this will probably be at least 2x larger, so round to
> > 10x for growth and ease of math
> >     - assume spherical cow request/repository can accomplish all
> > tasks in 1 connection (and in that 1 second)
> >     - 60 requests * 1MB each second
> >     - produces about 500 mbps sustained traffic for this request rate
> >  * It's possible that RFC9286[2] expectations on nextUpdate existing
> > may cause the relying party clients behave
> >    differently, but that remains to be seen.
> >
> > I think the above would guide a discussion on the point raised in
> > RFC6481, I think adding to that some discussion
> > about resilience and redundancy (which RFC6481 attempts to talk about
> > a bit in Section 2.1 and point 5 of section 3)
> > and how that could improve the ability of the Publication Service to
> > be available would be helpful. I expect that guidance such
> > as:
> >  * maintain at least 2 transit paths to the wider Internet for each deployment
> >    - plan for outages on the local and immediate provider network systems
> >    - plan for outages to your networking systems as maintenance
> > requirements arise
> >  * plan more than one Publication Service deployment, in more than
> > one geographically distinct region(s)
> >    - remove local power/natural-disaster scenarios as much as
> > possible from outage causes
> >    - provide for disaster recovery  capabilities
> >  * move the Publication Service closer to the core of the network if possible
> >    - shorten the connections between relying parties and the service
> >
> > Along with the last set of points would be some guidance about how to
> > (again RFC6481 points a bit at this in section 3) to
> > provide a coherent view of the Publication Service data across
> > multiple relying party accessible endpoints. This should not
> > be hard to accomplish, but certainly some thought and guidance would
> > be handy to help folks move toward that world.
> >
> > I think a review of historic outages to Publication Services would
> > also show where each of the above points would have
> > proven useful to the wider community of relying parties and
> > Publication Services.
> >
> > Sorry, this set of comment(s) got longer than I expected :(
> >
> > -chris
> >
> > 0: https://www.rfc-editor.org/rfc/rfc6481.html#page-8 - 'Resource
> > Certificate Publication Repository Considerations'
> > 1: https://en.wikipedia.org/wiki/High_availability - How much downtime
> > to tolerate?
> > 2: https://www.rfc-editor.org/rfc/rfc9286.html#name-manifest -
> > next-update definition in Manifest
> >
> > On Wed, May 21, 2025 at 6:07 AM Job Snijders <[e-mail]> wrote:
> >>
> >> Dear all,
> >>
> >> I am _in favor_ of publishing this publication BCP (no pun intended ;-)
> >> but would like to see a few changes in order to help future RPKI
> >> operators.
> >>
> >> I've submitted the proposed changes through a github pull-request to
> >> save the author team a bit of time.
> >> https://github.com/timbru/draft-sidrops-publication-server-bcp/pull/31
> >>
> >> My comments so far:
> >>
> >> 1/ The Glossary can perhaps be expanded a bit: at the moment it makes
> >> mention of some entities but not all entities that play a role. Maybe
> >> add "Publication Point Operator" and "Relying Party", because
> >> "Publisher" already is there?
> >>
> >> 2/ As for the word "repository"... we are approaching the point where
> >> everything is called a repository. I suggest replacing the phrase "RRDP
> >> Repository" with "RRDP Server" and "Rsync Repository" with "Rsync
> >> Server".
> >>
> >> It is the RRDP _server_ and Rsync _server_ host one or more
> >> 'caRepository'.  Calling the RRDP server endpoint a repository (of
> >> repositories...) may increase confusion.
> >>
> >> 3/ Add to glossary: "rsyncd - Rsync daemon"
> >>
> >> 4/ "Because RPs use cached data" ... this is not a given, not all
> >> RP implementations have implemented the concept of a 'validated cache'
> >> (yet).
> >>
> >> I suggested different wording:
> >>
> >>        RPs are expected to make use of cached data from a previous,
> >>        successful fetch (Section 6 of @!RFC9286). Therefor, short
> >>        outages on the server side don't need to be cause for immediate
> >>        concern, provided the server operator restores access
> >>        availability in a timely fashion (e.g. before objects expire).
> >>
> >> 5/ Section 4.1 sorta buried the lede. I'd like it if the first sentence
> >> can strongly suggest to NOT RUN YOUR OWN. Add a sentence "In general, it
> >> is NOT RECOMMENDED to run your own Publication Server, but to make use
> >> of a publication server provided by the Issuer". Or, s/Issuer/RIR/.
> >> This is a BCP document after all, no need to beat around the bush.
> >>
> >> 6/ "... RPs rejecting CA publication points" I don't think RPs reject
> >> the publication point, RPs simply stop considering connecting to a
> >> publication point because the referal became invalid. Perhaps the
> >> paragraph can be reworded.
> >>
> >>        """
> >>        If the Publication Server is unavailable for some reason, this
> >>        will prevent Publishers from making updated RPKI objects
> >>        available. The most immediate impact of this is that the
> >>        publisher cannot distribute new issuances or revocations of
> >>        ROAs, ASPAs or BGPSec Router Certificates for the duration of
> >>        this outage. Thus, in effect, it cannot signal changes in its
> >>        routing operations. If the outage persists for an extended
> >>        period, then RPKI Manifests, CRLs, and Signed Objects might
> >>        became stale, hampering, for example, BGP Origin Validation
> >>        ([@!RFC6811].
> >>        """
> >> 7/ "content regression has occured due to data loss" maybe just needs to
> >> be "content regression has occured"
> >>
> >> 8/ Section 4.4 "The ROAs may not reflect what the publisher intended."
> >>   perhaps reword to:
> >>   "The published ROAs no longer reflect the CA's intentions." ?
> >>
> >> 9/ The paragraph with "The publisher may not renew their manifest or CRL
> >>   in time" contains a duplication (expiry == stale). Suggestion:
> >>   """
> >>   - The CA might not re-issue their Manifest or CRL in time, because
> >>     they operated under the assumption that the currently published
> >>     Manifest and CRL have not yet became stale.
> >>   """
> >>
> >> 10/ Some commas are missing here: "In addition to this the publishing CA
> >> MAY perform ..." I've submitted suggested rewording through a pull
> >> request.
> >>
> >> 11/ ref update is needed, [I-D.ietf-sidrops-rrdp-same-origin]
> >>
> >> 12/ "publishing in a repository" -> "publishing through a server" ?
> >>
> >> ---------
> >>
> >> I'll review the second half of the document soon!
> >>
> >> Thank you for your work on this document.
> >>
> >> Kind regards,
> >>
> >> Job
> >>
> >>
> >> On Wed, May 21, 2025 at 10:01:14AM +0200, Luigi Iannone wrote:
> >>> Hi All,
> >>>
> >>> the authors of draft-ietf-sidrops-publication-server-bcp requested WG Last Call as they consider that the document is ready to be published.
> >>>
> >>> Abstract:
> >>>  This document describes best current practices for operating an RFC
> >>>   8181 RPKI Publication Server and its rsync (RFC 5781) and RRDP (RFC
> >>>   8182) public repositories.
> >>>
> >>> Details can be found at: https://datatracker.ietf.org/doc/draft-ietf-sidrops-publication-server-bcp/
> >>>
> >>> This email formally opens the two weeks WG Last Call, to be ended June 4, 2025.
> >>>
> >>> Please review this WG document and let the WG know if you agree that it is ready to be handed over to the AD.
> >>>
> >>> If you have objections, please state your reasons why, and explain what it would take to address your concerns.
> >>>
> >>> Thanks in adavance for your feedback.
> >>>
> >>> Luigi & Russ
> >>
> >>> _______________________________________________
> >>> Sidrops mailing list -- [e-mail]
> >>> To unsubscribe send an email to [e-mail]
> >>
> >> _______________________________________________
> >> Sidrops mailing list -- [e-mail]
> >> To unsubscribe send an email to [e-mail]
> >
> > _______________________________________________
> > Sidrops mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
>