Re: [bmwg] Review of draft-vfv-bmwg-srv6-bench-meth-02.txt

[Giuseppe Fioccola <[e-mail]>]

Dear All,
We just uploaded the new versions of the drafts on benchmarking methodology for SR-MPLS and SRv6 performance:
https://datatracker.ietf.org/doc/draft-vfv-bmwg-srmpls-bench-meth/03/
https://datatracker.ietf.org/doc/draft-vfv-bmwg-srv6-bench-meth/03/

These new revisions address the comments received by Gabor Lencse.

We always welcome your comments and inputs.

Best Regards,

Giuseppe

(on behalf of the coauthors)

From: bmwg <[e-mail]> On Behalf Of Giuseppe Fioccola
Sent: Thursday, September 29, 2022 5:44 PM
To: Gabor LENCSE <[e-mail]>; [e-mail]
Subject: Re: [bmwg] Review of draft-vfv-bmwg-srv6-bench-meth-02.txt

Dear Gabor,
Thanks a lot for your revision.
Please find my replies inline tagged as [GF].

Regards,

Giuseppe

From: bmwg <[e-mail]<mailto:[e-mail]>> On Behalf Of Gabor LENCSE
Sent: Wednesday, September 28, 2022 9:17 AM
To: [e-mail]<mailto:[e-mail]>
Subject: [bmwg] Review of draft-vfv-bmwg-srv6-bench-meth-02.txt


Dear Authors,

At IETF 114, I promised to review your draft: https://www.ietf.org/archive/id/draft-vfv-bmwg-srv6-bench-meth-02.txt

First of all, I must admit, that I am not familiar with segment routing, I read the draft only from benchmarking point of view.

I have a few comments and suggestions.

Abstract:

   This document defines a methodology for benchmarking Segment Routing

   (SR) performance for Segment Routing over IPv6 (SRv6).  It builds

   upon [RFC2544], [RFC5180], [RFC5695] and [RFC8402].

Neither RFC4814 nor RFC8219 are mentioned here.

[GF]: We may consider to mention RFC 4814 and RFC 8219 in the draft rather than in the Abstract. The transition technologies can be mentioned but they should not influence functionality or performance of a SR node, which is our DUT.

I recommend mentioning RFC 4814, because when I was not aware of it, I used fixed UDP port numbers as defined in the Appendix C.2.6.4 of RFC 2544. And thus the DUT (actually a multi-core server running Linux) was not able to distribute the interrupts of the arriving packets among the CPU cores (that is, Receive-Side Scaling did not work).

I am not aware of the architecture of segment routing devices, but I suspect that they may have several CPU cores. So using psedorandom UDP port number can be important.

[GF]: The architecture of a router is different and has less restrictions but it is worth mentioning RFC 4814 implications since SR functionalities may be tested on a server running Linux as well.

I am aware that:

   The purpose of this document is to describe a methodology specific to

   the benchmarking of Segment Routing over IPv6.  The methodology

   described is a complement for [RFC5180] and [RFC5695].

But I believe that in some cases (e.g. Latency measurement) a better methodology can be recommended, if we consider RFC8219. (Please see later, why.)

I think that the following part may be reconsidered:

   Recommended frame sizes are the following:



   o  Ethernet Minimal: 64+8+n*16



   o  DUT Minimal Wire Speed: 128-256 (it depends on the range

      recommended in the DUT specification)



   o  Ethernet Typical: 1518+8+n*16



   o  DUT Maximum Wire Speed: 9000

In contrast, RFC 2544 recommends the following frame sizes:

9.1 Frame sizes to be used on Ethernet



       64, 128, 256, 512, 1024, 1280, 1518

RFC 8219 recommends even more:

   Based on the recommendations of [RFC5180<https://datatracker.ietf.org/doc/html/rfc5180>], the following frame sizes

   SHOULD be used for benchmarking IPvX/IPvY traffic on Ethernet links:

   64, 128, 256, 512, 768, 1024, 1280, 1518, 1522, 2048, 4096, 8192, and

   9216.

The Authors may be right that some of the above frame sizes are out of interest, but I would like to be sure that they were considered and then judged as redundant.

[GF]: We will look at RFC8219 but we already said in the document that any other frame sizes may be tested if there is an interest: “Any other frame sized may be added if suspected of abnormal behavior.  For example, some architectures may allocate buffer memory in big fixed chunks that may drop performance if frame sizes are chosen just a few octet more than the fixed chunk size (the second chunk would have a very low memory utilization).”

The following text occurs 3 times (in 3 different measurement procedures):

   Procedure: Similar to [RFC5695] with potential extension to test SID

   list longer than 1 SID (2 are recommended, many are possible).



Why "recommended" and not "RECOMMENDED"?

[GF] Yes, it should be upper case

Regarding the measurement procedures, I have two observations:

1. I believe that the latency measurement procedure could be (and should be) improved.

5.2. Latency

   Procedure: Similar to [RFC5695] with potential extension to test SID

   list longer than 1 SID (2 are recommended, many are possible).

RFC5695 reuses the Latency measurement procedure defined in Section 26.2 of RFC2544. That one uses only a simple timestamp per experiment.

However, RFC8219 has an improved procedure for latency measurements: it uses at least 500 timestamps: https://datatracker.ietf.org/doc/html/rfc8219#section-7.2

I believe that it would be worth using that improved procedure.

[GF] Indeed, it is possible to improve latency tests. We will have a look to all the related documents.

2. I have noticed that back-to-back frames is missing from among the measurement procedures.

In Section 3.1 there is such text:

[RFC9004] updates the procedures of

   the test to measure the Back-to-Back Frames since their

   characterization is relevant in software-packet processing.

But the back-to-back frames measurement procedure is missing from Section 5. If it is an accident, then I recommend it to be included. If it is intentional, then is should probably be mentioned why it was left out.

[GF] Yes, we can add tests for back-to-back frames measurement procedure. RFC9004 is the reference document for this.

6.  Security Considerations

Special capabilities SHOULD NOT exist in the DUT/SUT

   specifically for benchmarking purposes.

I believe that it is a very important statement, which should stand somewhere else, and not hidden in this section. (Perhaps most people do not read it here.)

[GF] Agree, we will revise.

Nits:

In SRv6 the --> In SRv6, the



This document is benchmarking only "source routing".

I think that it should be expressed somehow better.

[GF] Ok.

All in all: they are just my observations and recommendations. Perhaps some of them are not relevant or even incorrect. Please feel free to ignore them. :-)

Best regards,

Gábor