Re: [bmwg] Comments on draft-morton-bmwg-virtual-net-03

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Barry,

Thanks for your comments and suggestions on the draft.
I have a few responses below, but otherwise there will
be a revised draft available shortly.

You wrote:

Ø  Section 2, paragraph 2.  I think it is also essential to benchmark a "bare metal" software network function so that the performance of a physical device, software version on bare metal OS, and software version in VM can be compared
[ACM]
There may be more than one definition of "bare metal", but I tend to use the
one from Wikipedia  http://en.wikipedia.org/wiki/Hypervisor
when referring to virtualization.  For networking devices with special HW,
proprietary OS and dedicated SW, I've simply referred to them as
Physical Network Functions (PNF) and avoided using the term "bare metal"
in their context.

Section 3.3, item #4

Ø  I am not sure if this is the spirit of this bullet, but what I have observed in VM based applications is the performance degradation when a VM is rotated as others are added.  I think it will be important to define a test which benchmarks the performance of say a switch under RFC 2544 conditions as new VMs are added by the Hypervisor.  This has to do with concurrency and capacity testing for which I have comments a few points below

Yes, this is exactly what I was talking about, that there is a new genre of benchmarking
during VNF Lifecycle operations which BMWG has approached for PNFs, most recently in
the ISSU draft (and there were some related efforts/drafts about 10 years ago, too).
In a future where VNF and service chain deployment/termination is dynamic to meet
changing demands within a day or smaller time interval, performance implications
must be understood.

I've also tried something with the 3x3 matrix to address the
topic of concurrency, but you'll have to wait for the draft to see it.

regards, and thanks for your support,
Al



From: bmwg [mailto:[e-mail]] On Behalf Of Barry Constantine
Sent: Tuesday, March 31, 2015 8:54 AM
To: [e-mail]
Subject: [bmwg] Comments on draft-morton-bmwg-virtual-net-03

Hi Al,

I read the draft in detail and first off, want to state that I fully support this work in BMWG.

Here are my first round of comments:

1. Section 2, paragraph 2.  I think it is also essential to benchmark a "bare metal" software network function so that the performance of a physcial device, software version on bare metal OS, and software version in VM can be compared

2. Section 3.2.

- It would be good to expand what is meant by Infrastructure Virtual Network.  Since the VM running on Hypervisor is a VNF, this was not clear

- "number of VNF components in the service function chain"  -->  Should this be more of a defintion of the service function chain itself, from which the number of components becomes apparent?  Service function chain would then be clearer with an example as well

3. Section 3.3, item #4

I am not sure if this is the spirit of this bullet, but what I have observed in VM based applications is the performance degradation when a VM is rotated as others are added.  I think it will be important to define a test which benchmarks the performance of say a switch under RFC 2544 conditions as new VMs are added by the Hypervisor.  This has to do with concurrency and capacity testing for which I have comments a few points below

4. Section 3.4,  I think it should be recommended that physical hardware based tester should be used as the first choice for the test traffic device and that a separate tester VNF can be used but with stronger language that this then requires that the tester VNF function be independently benchmarked against it's physical counterpart

5. Section 4.1, same as comment #1, add bare metal OS test of software network function

6. Section 4.2, paragraph 1.  I think an example of a performance results from internal OS would help here.  Did you have in mind network counters of dropped packets, etc?

7. Section 4.4.  This is my first exposure to the 3 x 3 matrix and I this is interesting.  I think what is needed is a section that describes a technique to document the concurrency of VNFs within a hardware platform and the performance / breaking points.

An example would be a firewall VNF benchmarked as a single instance and then how many Firewall VNF instances can exist concurrently within the given hardware platform before performance starts to degrade.

Then we would get into VNF functions A,B,C and how many of each can exist concurrently within a platform, so there might be a reporting matrix that shows functions A, B, C as columns and rows which show the various combinations of each that can be hosted with 100% performance for each combination (I hope this makes sense as this might be complicated to explain in an email).

Again, fully support this work and hope that these comments are helpful.

Regards,
Barry