[bmwg] Re: Progress so far -- Re: Revisiting BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsearch

[Gábor LENCSE <[e-mail]>]

Hi Giuseppe,

Thank you very much for your reply. Please see my answers inline.

On 11/13/2025 12:24 PM, Giuseppe Fioccola wrote:
>
> Hi Gabor,
>
> Thank you for your initial analysis.
>
> As a contributor, I have some comments:
>
>   * I agree that RFC 4814 should update RFC 2544 since it discusses
>     hash and stuffing for testing, but I’m not sure about RFC 8219. Is
>     there something in RFC 8219 (BM for IPv6 Transition Technologies)
>     that has relevance for the general benchmarking guidelines?
>
I believe YES.

For example, RFC 2544 defined a rather simple latency measurement 
procedure.  It uses a single tagged frame sent after 60s in a 120s long 
stream for latency measurement. (Please refer to Section 26.2 of RFC 
2544.) On the other hand, RFC 8219 defined a much better one: it uses at 
least 500 tagged frames sent after 60s in a 120s long stream for latency 
measurement. Please see the details here: 
https://datatracker.ietf.org/doc/html/rfc8219#section-7.2 Thus it can 
give much better characterizations of the latency conditions.

In addition to that, RFC 8219 also uses Packet Delay Variation and Inter 
Packet Delay Variation to give further insights.

IMHO, if these improvements were deemed to be necessary to characterize 
the performance IPv6 transition technologies, then they could also be 
useful when performing simple IPv4 and IPv6 packet forwarding 
performance measurements, too.

What do you think?


>   * Another document that could also be considered is RFC 7640 on
>     Traffic Management Benchmarking.
>   * I would also add draft-ietf-bmwg-mlrsearch to the list of the
>     possible updates to RFC 2544.
>   * I suggest to include draft-lencse-bmwg-multiple-ip-addresses too.
>     Even though it is still individual, I think it is relevant.
>
Yes, I would like to consider them, too.

So far, I did only a very small portion of the planned task and I wanted 
to check, if there is interest in BMWG to tackle with this topic. If 
yes, then I am happy to continue the effort. The lion's share of the 
work is still left.

Best regards,

Gábor


> Regards,
>
> Giuseppe
>
> *From:* Gábor LENCSE <[e-mail]>
> *Sent:* Tuesday, November 11, 2025 9:59 AM
> *To:* Giuseppe Fioccola <[e-mail]>; [e-mail]
> *Cc:* [e-mail]
> *Subject:* Progress so far -- Re: [bmwg] Revisiting BMWG RFCs -- Re: 
> Re: AD Review of draft-ietf-bmwg-mlrsearch
>
> Hi BMWG,
>
> I started to systematically review  all documents produced by BMWG. I 
> collected my notes to an Excel sheet having the following column headings:
> - Document (RFC / I-D)
> - Full Title
> - Main Contributions
> - Potential Issues (omissions, contradictions, etc.)
> - It refers to
> - It is (formally) updated by
> - It (perhaps) should be updated by
>
> Do you consider theses aspects relevant?
>
> What else should be considered?
>
> I copy here the part of my Excel sheet that I have already filled in. 
> I am not sure, how it will go through the mailing list:
>
> Document (RFC / I-D)
>
> 	
>
> Full Title
>
> 	
>
> Main Contributions
>
> 	
>
> Potential Issues (omissions, contradictions, etc.)
>
> 	
>
> It refers to
>
> 	
>
> It is (formally) updated by
>
> 	
>
> It (perhaps) should be updated by
>
> RFC 1212
>
> 	
>
> Benchmarking Terminology for Network Interconnection Devices
>
> 	
>
> Defines the following terms:
> Back-to-back
> Bridge
> bridge/router
> Constant Load
> Data link frame size
> Frame Loss Rate
> Inter Frame Gap
> Latency
> Link Speed Mismatch
> MTU-mismatch behavior
> Overhead behavior
> Overloaded behavior
> Policy based filtering
> Restart behavior
> Router
> Single frame behavior
> Throughput
>
> 	
> 	
>
> -
>
> 	
> 	
>
> RFC 1944
>
> 	
>
> Benchmarking Methodology for Network Interconnect Devices
>
> 	
>
> -- Obsoleted by RFC 2544
>
> 	
> 	
>
> RFC 1242
>
> 	
> 	
>
> RFC 2285
>
> 	
>
> Benchmarking Terminology for LAN Switching Devices
>
> 	
>
> Defines further terms, theoretically regarding LAN switching, but 
> several of them is of general interest (e.g. DUT/SUT, bidirectional 
> traffic, etc.)
>
> 	
> 	
>
> RFC 1242, RFC 1944
>
> 	
> 	
>
> RFC 2432
>
> 	
>
> Terminology for IP Multicast Benchmarking
>
> 	
>
> Defines further terms regarding IP multicast benchmarking.
> Introduces the two major classes of documents: terminology and 
> methodology.
> Also addresses encapsulation and decapsulation performance.
>
> 	
>
> IPv4 specific, but it is natural.
>
> 	
>
> RFC 1242, RFC 2285
>
> 	
> 	
>
> RFC 2544
>
> 	
> 	
>
> Defines all important circumstances including test setup, conditions, 
> measurement procedures, frame formats, etc.
>
> 	
>
> uses fixed frame format (fixed IP addresses and port numbers)
> the Latency measurement procedure only uses single tagged frame.
>
> 	
>
> RFC 1242
>
> 	
>
> RFC 9004, RFC 6201, RFC 6815
>
> 	
>
> RFC 4814!, RFC 8219?
>
> RFC 2647
>
> 	
>
> Benchmarking Terminology for Firewall Performance
>
> 	
> 	
> 	
>
> RFC 1242, RFC 2285
>
> 	
> 	
>
> Do you consider this review useful?
>
> Best regards,
>
> Gábor
>
> On 4/15/2025 10:26 AM, Giuseppe Fioccola wrote:
>
> Hi Gabor,
>
> Thank you for sharing these considerations.
>
> I think it is worth discussing further in the WG. A possible approach 
> could be to collect all these possible updates to RFC 2544 in a brand 
> new draft, which would aim to explicitly update RFC 2544. It might 
> also be a short document since some potential updates have already 
> been published as RFCs. What do you think?
>
> Regards,
>
> Giuseppe
>
> *From:* Gábor LENCSE <[e-mail]> <mailto:[e-mail]>
> *Sent:* Friday, April 11, 2025 5:39 PM
> *To:* [e-mail]
> *Subject:* [bmwg] Revisiting BMWG RFCs -- Re: Re: AD Review of 
> draft-ietf-bmwg-mlrsearch
>
> Hi Med,
>
>     As a future work, the WG can revisit some of the reference RFCs it
>     produced. +20 years is sufficiently enough to learn from
>     deployments/experience and make some recommendations about the
>     various specs. That effort can then formally discuss 2544 vs MLRS
>     and sketch relevant recommendations. Hope to see volunteers to
>     help the WG explore that path.
>
> I think it would be worth doing so. Such an overview and discussing 
> potential updates, changes, etc. could be useful.
>
> For example, IMHO, RFC 4814 should have updated RFC 2544, but it did 
> not happen. When I implemented my RFC 8219 compliant SIIT tester 
> called siitperf [1], I followed the test frame format defined in RFC 
> 2544 using fixed port numbers [2]. After finishing it, Al Morton drew 
> my attention to the usage of pseudorandom port numbers recommended by 
> RFC 4814. Then, I modified siitperf to support pseudorandom port 
> numbers [3]. However, it would have been a more straightforward way to 
> design it so rather than later modifying it.
>
> Another thing is that, for example, RFC 8219 redefined the latency 
> measurement procedure of RFC 2544, but only a few people know about 
> this, IMHO, better procedure (those who are interested in IPv6 
> transition technologies).
>
> So, I think it would be a good idea to revisit BMWG RFCs and discuss 
> some similar questions. I would be willing to contribute to such an 
> effort.
>
> Best regards,
>
> Gábor
>
> [1] G. Lencse, "Siitperf: an RFC 8219 compliant SIIT and stateful 
> NAT64/NAT44 tester", https://github.com/lencsegabor/siitperf
>
> [2] G. Lencse, "Design and Implementation of a Software Tester for 
> Benchmarking Stateless NAT64 Gateways", /IEICE Transactions on 
> Communications/, vol. E104-B, no.2, pp. 128-140. February 1, 2021. 
> DOI: 10.1587/transcom.2019EBN0010, available: 
> http://doi.org/10.1587/transcom.2019EBN0010
>
> [3] G. Lencse, "Adding RFC 4814 Random Port Feature to Siitperf: 
> Design, Implementation and Performance Estimation", /International 
> Journal of Advances in Telecommunications, Electrotechnics, Signals 
> and Systems/, vol 9, no 3, pp. 18-26, 2020, DOI: 
> 10.11601/ijates.v9i3.291, available: 
> https://doi.org/10.11601/ijates.v9i3.291
>
>