Re: [bmwg] WG Last Call: draft-ietf-bmwg-benchmarking-stateful-04

[Vasilenko Eduard <[e-mail]>]

Hi all,
I have read the draft carefully. I support this RFC publishing.

A few comments:

1.       I have not found any logical problems. Everything looks right.

2.       I am not a native speaker, but it looks to me that the draft has grammar errors.
I typically accept about 80% of what Grammarly proposes.
I have put Abstract into grammarly.com – I agree to all 3 corrections.
IMHO: it makes sense to pass the document through any spell checker.

3.       A tradeoff of measurement complexity is probably right:
Any software stateful device has a dependency between cps (new sessions per second), pps, and bps. The more one performance – the fewer others, because all need CPU cycles.
Heavy stateful hardware devices may have a hardware separation between “slow path” and “fast path” and then cps becomes independent from pps and bps.
(yet all historical attempts for such a separation failed, heavy hardware our days is NP that plays both roles, i.e. “slow path” and “fast path” share the same ASIC, cores split is dynamic)

A)      I did propose before to assume some proportion between cps:bps:pps and then raise the load proportionally, It is what I did many years ago. It is the best method for a specific environment/production. Yet, it has 2 problems:
1. It is very specific for the environment, not general at all. Tests from different proportions may not be comparable. Approximation of cps, bps, and pps influence may be challenging.
2. Sometimes, It is difficult to find/assume this proportion, especially for a new deployment.

B)      The draft proposes a separation for 2 test phases: test the “slow path” and then test the “fast path” separately.
IMHO: it is better for the RFC because it is general, no need to assume cps:bps:pps proportion

Eduard
From: bmwg [mailto:[e-mail]] On Behalf Of Paolo Volpato
Sent: Wednesday, January 3, 2024 5:36 PM
To: bmwg <[e-mail]>
Cc: [e-mail]
Subject: Re: [bmwg] WG Last Call: draft-ietf-bmwg-benchmarking-stateful-04

Dear BMWG,

After reading the draft, I support that it is published as an Informational RFC.
BR
Paolo

From: bmwg <[e-mail]<mailto:[e-mail]>> On Behalf Of [e-mail]<mailto:[e-mail]>
Sent: Wednesday, December 6, 2023 9:22 PM
To: bmwg <[e-mail]<mailto:[e-mail]>>
Subject: [bmwg] WG Last Call: draft-ietf-bmwg-benchmarking-stateful-04

Hello BMWG,
              We are starting a working group last call (WGLC) for the "Benchmarking Methodology for Stateful NATxy Gateways using RFC 4814 Pseudorandom Port Numbers”. It’ll start tomorrow, December 7, and run for 4 weeks, closing on January 4. The extended WGLC allows for the holiday and new year breaks.

              Please read the draft and express your opinion on whether or not this Internet-Draft should be forwarded to the Area Directors for publication and an Informational RFC. Send your comments to this list (preferred), or to the co-chairs at [e-mail]<mailto:[e-mail]>.

For the co-chairs,
Sarah