[bmwg] Re: [hackathon] Hackathon Project: ROVBench – Benchmarking Framework for Route Origin Validation

[Libin Liu <[e-mail]>]

Hi Christian,

Thank you for these comments. They help improve the draft. I plan to address them in the next revision. Please see my replies inline.

________________________________
From: Christian Giese <[e-mail]>
Sent: Saturday, March 14, 2026 6:58 PM
To: Libin Liu <[e-mail]>
Cc: bmwg <[e-mail]>; IETF IPPM WG ([e-mail]) <[e-mail]>
Subject: Re: [hackathon] Hackathon Project: ROVBench – Benchmarking Framework for Route Origin Validation


Hi Libin,

here a few more comments:

*  Frame size(s) used (e.g., 64, 512, 1518 bytes).


> For this benchmark, where only the time of arrival is relevant, smaller packets are clearly preferable. However, a 64-byte frame size is often insufficient to accommodate test traffic headers (e.g. when using BNG Blaster). Therefore, I propose a recommended frame size of 128 bytes at Layer 3. As an example, the BNG Blaster header is 48 bytes, which, when combined with UDP and IPv6 headers, results in a total of 96 bytes.

[Libin:] I agree that this benchmark should use small packets, since smaller packets improve time resolution for traffic-based convergence measurements. I also agree that some traffic generators may not support very small packets because headers and encapsulation consume space. I will revise the text to state that the benchmark should use a fixed packet size, and that 128-byte packets at Layer 3 can serve as one practical choice, while other sizes may be used if the test setup requires them.


  *  Traffic pattern (constant rate, burst, IMIX).


> IMIX specifies frame size, not rate. To simplify the calculation of convergence time based on traffic, a constant rate measured in packets per second (PPS) is preferable (e.g. 1PPS allows seconds resolution, 1000 PPS allows milliseconds, ...).

[Libin:] Thank you for mentioning this. I will update the draft to recommend constant-rate traffic in PPS for convergence measurements. The draft may still allow burst traffic for various stress scenarios.



  *  Source and destination IP address ranges.


> The destination IP address is implicitly defined by the BGP updates that are tested. Assuming that one stream is sent per prefix, the destination IP address would typically be the first IP address within each prefix.

[Libin:] I will revise the text to state that, when the setup sends one stream per tested prefix, the destination address should come from within that prefix. The tester may use the first usable address as one example. The draft will also require the tester to document the selection rule and apply it across runs.



BGP convergence time, VRP storage size, CPU and memory utilization, and ROV state rebuild time.


> Here would explicitly mention system CPU and memory utilisation as well as BGP process utilization.

[Libin:] I will update the text to require collection of system-level CPU and memory utilization, ROV process utilization, BGP process or routing process utilization. These metrics help show both overall device impact and process-level cost related to ROV.


[Libin:] Thank you again for the careful review. I will incorporate these changes in the next version.




Regards

Christian


On Sat, Mar 14, 2026 at 5:57 PM Libin Liu <[e-mail]<mailto:[e-mail]>> wrote:
Hi Christian,

(I am copying this reply to the WG mailing list.)

Thank you very much for your valuable feedback.

Your comments relate to the descriptions in Sections 4, 5.2, and 7 of the draft. I understand your points and agree that it would be better to explicitly distinguish the prefix quantities for IPv4 and IPv6 in the benchmarking setup. I will update the descriptions in Sections 4 and 5.2 to incorporate this recommendation in the next version. Also, I will add explicit parameter requirements for the IPv4 and IPv6 prefix quantities in Section 7.



Best,
Libin

________________________________
From: Christian Giese <[e-mail]<mailto:[e-mail]>>
Sent: Saturday, March 14, 2026 2:35 PM
To: Libin Liu <[e-mail]<mailto:[e-mail]>>
Subject: Re: [hackathon] Hackathon Project: ROVBench – Benchmarking Framework for Route Origin Validation

Hi Libin,

refering to following lines in your draft:

The BGP traffic generator establishes one or more BGP peering sessions with the DUT and is responsible for delivering a full global routing table, on the order of 800,000 to 1,000,000 prefixes...

A stable and realistic baseline BGP RIB-in (e.g., ~1M global routes)....

I recommend being explicit about the different prefix quantities when discussing performance testing for IPv4 and IPv6, rather than referring to prefixes generally, as testing with the same total number may yield different results. For example, specify tests using a mixed set of 1M IPv4 and 250K IPv6 prefixes, or test each AFI separately with those specified amounts.

Regards
Christian

________________________________
From: Libin Liu <[e-mail]<mailto:[e-mail]>>
Sent: Friday, March 13, 2026 11:20 PM
To: Christian Giese <[e-mail]<mailto:[e-mail]>>

Cc: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>; bmwg <[e-mail]<mailto:[e-mail]>>; IETF IPPM WG ([e-mail]<mailto:[e-mail]>) <[e-mail]<mailto:[e-mail]>>
Subject: Re: [hackathon] Hackathon Project: ROVBench – Benchmarking Framework for Route Origin Validation

Hi Christian,
Thank you for sharing this and for the detailed introduction.
BNG Blaster looks very relevant for this work, especially for measuring the time between BGP updates and forwarding behavior. It would be great to explore how it could be integrated into the ROV benchmarking setup.
I will be onsite tomorrow at the hackathon and look forward to discussing this further and collaborating.

Best,
Libin
________________________________
From: Christian Giese <[e-mail]<mailto:[e-mail]>>
Sent: Friday, March 13, 2026 11:01 PM
To: Libin Liu <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>; bmwg <[e-mail]<mailto:[e-mail]>>; IETF IPPM WG ([e-mail]<mailto:[e-mail]>) <[e-mail]<mailto:[e-mail]>>
Subject: Re: [hackathon] Hackathon Project: ROVBench – Benchmarking Framework for Route Origin Validation


Hi Libin,


This is a great initiative! I've developed a similar benchmark to measure the time between receiving a BGP update and the corresponding forwarding entry becoming active. The open-source (BSD licensed) tool BNG Blaster is used for this test. Although the name is misleading (it began as a BNG tester but has grown into a complete router tester), it could be ideal for this specific purpose.


The test operates by sending pre-built BGP updates and generating traffic streams for each prefix (e.g. one million streams or more) to verify successful forwarding. Running this test initially without any policies establishes a baseline, which allows you to observe how the test results change as different configurations are applied.


Happy to contribute during tomorrow's hackathon!


BNG Blaster: https://rtbrick.github.io/bngblaster/index.html

Benchmark Test: https://github.com/rtbrick/BGP-CP-DP-Testing


There is also a video from a live demo of thie benchmark, but unfortunately in german.

https://youtu.be/ttbjXxCpKvE?si=6hayD7VWcduKBN-t&t=53


Regards

Christian Giese


On Fri, Mar 13, 2026 at 10:06 PM Libin Liu <[e-mail]<mailto:[e-mail]>> wrote:
Dear All,

We would like to introduce a Hackathon project (ROVBench) related to benchmarking Route Origin Validation (ROV) and invite interested participants to join.

ROV is increasingly deployed to improve BGP routing security. As deployment grows, it becomes important to evaluate how routers process RPKI validation data and how ROV affects routing behavior and control-plane performance. In BMWG, ongoing work is defining a benchmarking methodology for routers that implement ROV, including evaluation of VRP update processing, BGP-RPKI interactions, control-plane resource usage, and scalability.

The ROVBench hackathon project aims to explore how such benchmarking procedures can be implemented in practice by building a small prototype benchmarking environment. We welcome participation from anyone interested in ROV deployment, router benchmarking, or BGP control-plane behavior. More information about the project can be found here: https://wiki.ietf.org/en/meeting/125/hackathon#rovbench-a-benchmarking-framework-for-route-origin-validation-rov.

Comments, suggestions, and collaboration are very welcome.


Best,
Libin
_______________________________________________
hackathon mailing list -- [e-mail]<mailto:[e-mail]>
To unsubscribe send an email to [e-mail]<mailto:[e-mail]>
Unsubscribe: mailto:[e-mail]<mailto:[e-mail]>?subject=unsubscribe

NOTICE TO RECIPIENT This e-mail message and any attachments are confidential and may be privileged. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of this e-mail is strictly prohibited. Please notify us immediately of the error by return e-mail and please delete this message from your system. For more information about Rtbrick, please visit us at www.rtbrick.com<http://www.rtbrick.com>

NOTICE TO RECIPIENT This e-mail message and any attachments are confidential and may be privileged. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of this e-mail is strictly prohibited. Please notify us immediately of the error by return e-mail and please delete this message from your system. For more information about Rtbrick, please visit us at www.rtbrick.com<http://www.rtbrick.com>

NOTICE TO RECIPIENT This e-mail message and any attachments are confidential and may be privileged. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of this e-mail is strictly prohibited. Please notify us immediately of the error by return e-mail and please delete this message from your system. For more information about Rtbrick, please visit us at www.rtbrick.com<http://www.rtbrick.com>