[bmwg] Review of draft-vfv-bmwg-srv6-bench-meth-02.txt

[Gabor LENCSE <[e-mail]>]

Dear Authors,

At IETF 114, I promised to review your draft: 
https://www.ietf.org/archive/id/draft-vfv-bmwg-srv6-bench-meth-02.txt

First of all, I must admit, that I am not familiar with segment routing, 
I read the draft only from benchmarking point of view.

I have a few comments and suggestions.

Abstract:

    This document defines a methodology for benchmarking Segment Routing
    (SR) performance for Segment Routing over IPv6 (SRv6).  It builds
    upon [RFC2544], [RFC5180], [RFC5695] and [RFC8402].

Neither RFC4814 nor RFC8219 are mentioned here.

I recommend mentioning RFC 4814, because when I was not aware of it, I 
used fixed UDP port numbers as defined in the Appendix C.2.6.4 of RFC 
2544. And thus the DUT (actually a multi-core server running Linux) was 
not able to distribute the interrupts of the arriving packets among the 
CPU cores (that is, Receive-Side Scaling did not work).

I am not aware of the architecture of segment routing devices, but I 
suspect that they may have several CPU cores. So using psedorandom UDP 
port number can be important.


I am aware that:

    The purpose of this document is to describe a methodology specific to
    the benchmarking of Segment Routing over IPv6.  The methodology
    described is a complement for [RFC5180] and [RFC5695].

But I believe that in some cases (e.g. Latency measurement) a better 
methodology can be recommended, if we consider RFC8219. (Please see 
later, why.)


I think that the following part may be reconsidered:

    Recommended frame sizes are the following:

    o  Ethernet Minimal: 64+8+n*16

    o  DUT Minimal Wire Speed: 128-256 (it depends on the range
       recommended in the DUT specification)

    o  Ethernet Typical: 1518+8+n*16

    o  DUT Maximum Wire Speed: 9000

In contrast, RFC 2544 recommends the following frame sizes:

9.1 Frame sizes to be used on Ethernet

        64, 128, 256, 512, 1024, 1280, 1518

RFC 8219 recommends even more:

    Based on the recommendations of [RFC5180  <https://datatracker.ietf.org/doc/html/rfc5180>], the following frame sizes
    SHOULD be used for benchmarking IPvX/IPvY traffic on Ethernet links:
    64, 128, 256, 512, 768, 1024, 1280, 1518, 1522, 2048, 4096, 8192, and
    9216.

The Authors may be right that some of the above frame sizes are out of 
interest, but I would like to be sure that they were considered and then 
judged as redundant.


The following text occurs 3 times (in 3 different measurement procedures):

    Procedure: Similar to [RFC5695] with potential extension to test SID
    list longer than 1 SID (2 are recommended, many are possible).

Why "recommended" and not "RECOMMENDED"?


Regarding the measurement procedures, I have two observations:

1. I believe that the latency measurement procedure could be (and should 
be) improved.

5.2. Latency

    Procedure: Similar to [RFC5695] with potential extension to test SID
    list longer than 1 SID (2 are recommended, many are possible).

RFC5695 reuses the Latency measurement procedure defined in Section 26.2 
of RFC2544. That one uses only a simple timestamp per experiment.

However, RFC8219 has an improved procedure for latency measurements: it 
uses at least 500 timestamps: 
https://datatracker.ietf.org/doc/html/rfc8219#section-7.2

I believe that it would be worth using that improved procedure.


2. I have noticed that back-to-back frames is missing from among the 
measurement procedures.

In Section 3.1 there is such text:

[RFC9004] updates the procedures of
    the test to measure the Back-to-Back Frames since their
    characterization is relevant in software-packet processing.

But the back-to-back frames measurement procedure is missing from 
Section 5. If it is an accident, then I recommend it to be included. If 
it is intentional, then is should probably be mentioned why it was left out.


6.  Security Considerations

Special capabilities SHOULD NOT exist in the DUT/SUT
    specifically for benchmarking purposes.

I believe that it is a very important statement, which should stand 
somewhere else, and not hidden in this section. (Perhaps most people do 
not read it here.)


Nits:

In SRv6 the --> In SRv6, the

This document is benchmarking only "source routing".

I think that it should be expressed somehow better.


All in all: they are just my observations and recommendations. Perhaps 
some of them are not relevant or even incorrect. Please feel free to 
ignore them. :-)

Best regards,

Gábor