[bmwg] Benoit Claise's No Objection on draft-ietf-bmwg-vswitch-opnfv-03: (with COMMENT)

[Benoit Claise <[e-mail]>]

Benoit Claise has entered the following ballot position for
draft-ietf-bmwg-vswitch-opnfv-03: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/



----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

For sure, the topic of benchmarking of benchmarking virtual switches is
important in the industry today.

Technical Summary: This draft summarizes the progress of the OPNFV project on
virtual switch performance characterization, and notes that the OPNFV work
builds upon the IETF work, particularly the BMWG work within IETF, with an
emphasis on RFC2544.

I like the cross SDO aspects of this draft, as mentioned in the shepherd
writeup: The document itself was reviewed in ETSI NFV, and used as a reference
by at least one of the TST WG and IFA WF specifications (2 references in
total). The document has also been reviewed by several individuals from the
OPNFV community, and NFVRG (IRTF) is aware of the draft.

I would have preferred to see a document that specifies "benchmarking virtual
switches" instead of "benchmarking virtual switches in OPNFV". In other words,
just looking at the abstract, this is completely different to write:

   This memo describes the way to benchmark virtual switch performance.
   The specifications are based on the Open Platform for NFV (OPNFV) project on
   virtual switch performance "VSPERF".

As opposed to:

   This memo describes the progress of the Open Platform for NFV (OPNFV)
   project on virtual switch performance "VSPERF".

Regarding the "describes the progress of", sentences such as ...
    - This project intends to build ...
    - Therefore, this memo begins to describe the additional considerations ...
    - This memo summarizes the progress of the Open Platform for NFV (OPNFV)
    project on virtual switch performance characterization, "VSPERF", through
    the Brahmaputra (second) release [BrahRel].
... lead me to think that this is unfinished work, or that work will anyway
have to be updated.

Again, looking at the scope:

   The primary purpose and scope of the memo is to inform the industry
   of work-in-progress that builds on the body of extensive BMWG
   literature and experience, and describe the extensions needed for
   benchmarking virtual switches.

I was hoping that the primary purpose and scope of the memo would be the
specifications of the virtual switch benchmarking, not to inform about the
work-in-progress.

Reading further, I see text such as:

    Specifications to be included in future updates of the LTD include:

       o  [RFC3918] Methodology for IP Multicast Benchmarking

       o  [RFC4737] Packet Reordering Metrics

or

   When considering characteristics important to "telco" network
   functions, we must begin to consider additional performance metrics.

or

   Tests have been (or will be) designed to collect the metrics below:

On one side, I appreciate that you're publishing what you learned, which should
lead up to revisions in the future. On the other side, I've not been used to
that approach in BMWG. I'll trust the responsible AD regarding the publication
of this document as a kind of intermediary document.

I believe the document would be a better flow (and that some concerns would
disappear) if it would have the following structure
    - benchmarking virtual switches is key in the industry today
    - there are the virtual switches benchmarking specification today
    - we worked on this with OPNFV (VSPERF). Great collaboration.
    - this is a complex and evolving field:
      compared to RFC 2544, we learn that ...
      we realize that this is work in progress ...
    - future plan: we will need to integrate more "stuff" in the benchmark,
    such as RFC3918, RFC4737, additional perf metrics, you-name-it - we'll work
    with OPNFV (and other groups) on this future plan.

Editorial
- Section 3.4. Flow timeout.
You mean the active timeout, section 2.2.23 in RFC 6645?
Should this section refer to RFC6645, or RFC 5470 (section 5.1.1)?

-

   In addition to this, the LTD also re-uses the terminology defined by:

   o  [RFC2285] Benchmarking Terminology for LAN Switching Devices

   o  [RFC5481] Packet Delay Variation Applicability Statement

RFC5481 is already mentioned in the previous bullet list (The base performance
tests in the LTD are based on the pre-existing specifications developed by BMWG
to test the performance of physical switches. These specifications include). So
hopefully, it re-uses the terminology, right?