[bmwg] Benchmarking Methodology for SDN Controller Performance - Updated Draft Version

["Bhuvan \(Veryx Technologies\)" <[e-mail]>]

Dear BMWG Members,

 

We have updated the draft (draft-bhuvan-bmwg-of-controller-benchmarking-01
<https://tools.ietf.org/html/draft-bhuvan-bmwg-of-controller-benchmarking-01
> ) about SDN Controller benchmarking addressing comments received in
IETF-91 meeting and the mailing list. Thank you very much for providing your
valuable comments. The latest drafts can be found here:

 

draft-bhuvan-bmwg-sdn-controller-benchmark-term-00
<http://tools.ietf.org/html/draft-bhuvan-bmwg-sdn-controller-benchmark-term-
00> 

draft-bhuvan-bmwg-sdn-controller-benchmark-meth-00
<http://tools.ietf.org/html/draft-bhuvan-bmwg-sdn-controller-benchmark-meth-
00> 

 

Summary of Changes:

a.       Split into two different drafts (Terminology and Methodology)

b.      Defined an additional metric - Control Sessions Capacity

c.       Provided an example Benchmarking Methodology for OpenFlow
Controller

d.      Addressed review comments from last IETF meeting and on the mailing
list

 

Summary of Comments Addressed:

 

1.       The following changes have been made to address Al's comments

a.        Updated the following SDN Terms - Flow, Learning Rate, Northbound
interface, Path, and Cluster/Redundancy Mode in the terminology draft to
provide more clarity and for consistency of terms with respect to other RFCs

b.      Updated the test setup section in methodology draft to show the the
network path more explicitly between the nodes.

c.       Updated test traffic consideration section to provide references to
default recommended traffic sizes.

d.      Renamed Measurement Accuracy section title to Measurement
Specification Point and Recommendation.

e.      Recommended to capture HW specifications under Test Reporting
section

f.        Benchmarking tests - Added text to capture test results for
unsuccessful topology discovery.

g.       Added note to handle network latency while measuring Topology
Discovery Time.

h.      Updated test procedure for Synchronous message processing time to
handle re-transmissions and packet loss.

i.         Added pre-requisite to handle connection requirements between
controller and SDN nodes while performing Synchronous message processing
rate benchmarking test.

j.        Added provision to capture Loss Ratio for Synchronous message
processing rate benchmarking test.

k.       Reworded the Performance to Speed under 3*3 matrix test coverage in
terminology document

l.         Need to discuss about adding accuracy in 3*3 matrix 

2.       Added Connection Recommendation section under Test Considerations
to handle device failures and processing overheads on devices in the path to
controller following the comments received from Andrew.

3.       Added clarification to Proactive Path Provisioning procedure for
Path Provisioning Time benchmarking test following the comment received from
Jay Karthik.

4.       Clarified the document scope for handling federation of controllers
following the comment received from Ramakrishnan

5.       Added appendix to provide examples of measurement based on OpenFlow
following the comment received from Marius Georgescu

6.       Added terminologies for SDN Application and Traffic Endpoint in
terminology document following the comment received from Marius Georgescu

7.       Added note about sending invalid messages to controller while
performing Exception Handling Benchmarking test following the comment from
Scott Bradner.

 

We would love to hear any comments and queries on the same.

 

Thanks,

Authors