Re: [bmwg] Genart last call review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07

Hi Bhuvan

Looks like a few lines of editing is all that it will take to help the 
reader.

Will take a look at the new version when it is uploaded.

- Stewart

On 31/01/2018 09:31, bhuvaneswaran.vengainathan@veryxtech.com wrote:
> Hi Stewart Bryant,
>
> Thank you for reviewing the draft and sharing your comments.
> Please find below our responses inline
>
> Thanks,
> Bhuvan
> On Tue, Jan 30, 2018 at 10:54 PM, Stewart Bryant 
> <stewart.bryant@gmail.com> wrote:
>
>     Reviewer: Stewart Bryant
>     Review result: Ready with Nits
>
>     I am the assigned Gen-ART reviewer for this draft. The General Area
>     Review Team (Gen-ART) reviews all IETF documents being processed
>     by the IESG for the IETF Chair.  Please treat these comments just
>     like any other last call comments.
>
>     For more information, please see the FAQ at
>
>     <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
>     Document: draft-ietf-bmwg-sdn-controller-benchmark-meth-07
>     Reviewer: Stewart Bryant
>     Review Date: 2018-01-30
>     IETF LC End Date: 2018-02-02
>     IESG Telechat date: Not scheduled for a telechat
>
>     Summary:
>
>     This is a well written comprehensive test set for SDN controllers.
>     It could be
>     published as is, but some thought about how to address the issues
>     below might
>     be helpful to the user of this technology. Major issues: None
>
>     Minor issues:
>
>     I find the large amount of text on Openflow that appears out of
>     the blue in the
>     appendix somewhat strange. The test suit is controller protocol
>     agnostic, so I
>     wonder why so much text is devoted to this specific SDN control
>     protocol. If
>     they are there by way of illustrative example of packet exchanges,
>     it might be
>     useful to the reader to point to them from the measurement text.
>     [Bhuvan] We agree with you. But we wanted to explain this
>     methodology using some well established protocol for better
>     understanding. Since OpenFlow is widely used , we have given
>     illustration of this methodology using OF protocol
>
>
>     Something I am slightly surprised by is the lack of statistical
>     sophistication.
>     Average is a very crude metric giving no information on the
>     distribution of the
>     results.
>     [Bhuvan] We are planning to add variance to all tests besides
>     average metric. Hope this address your concern.
>
>
>     I imagine that it is now ingrained in this aspect of the industry
>     to specify
>     graphs and tables, but I would have expected that the results would be
>     specified in some machine readable format such as xml for input to
>     a database
>     rather than in the human readable format that is hard coded into this
>     specification.
>     [Bhuvan] We are planning to leave this option to individuals
>     performing the methodology.
>     Nits/editorial comments:
>
>     Abstract
>
>     This document defines the methodologies for benchmarking control
>     plane performance of SDN controllers. Terminology related to
>     benchmarking SDN controllers is described in the companion
>     terminology document.
>
>     SB> It would be convenient to the reader to provide the reference
>     to or name of
>     SB> the companion document - the twin of the comment in the other
>     review.
>
>     SB> it would also be useful to include such a reference early in
>     the main text.
>     [Bhuvan] We will specify the name of the companion document explicitly
>
>
>     =============
>
>     4. Test Considerations
>
>     4.1. Network Topology
>
>     The test cases SHOULD use Leaf-Spine topology with at least 1
>     Network Device in the topology for benchmarking.
>     SB> Leaf-Spine could use a reference. In Fig 2 I am not sure this
>     is SL rather
>     than SB> a linear sequence of nodes. There is a better SL diagram
>     later in the
>     SB> document and it would be useful to the reader to forward
>     reference it.
>
> [Bhuvan]  We will work on the figures to reflect Leaf-Spine topo
>
>
>     ========
>
>     The test traffic
>     generators TP1 and TP2 SHOULD be connected to the first and the last
>     leaf Network Device.
>
>     SB> I am sure I know what does first and last mean, but the
>     meaning should be
>     called out.
>     [Bhuvan] I agree with you. We mean TP1 should be connected to the
>     flow source endpoint and TP2 should be connected to the flow
>     destination endpoint.
>     =========
>
>     Procedure:
>
>     5. Stop the trial when the discovered topology information matches
>     the deployed network topology, or when the discovered topology
>     information return the same details for 3 consecutive queries.
>
>     SB> What do you report in the latter case?
>     [Bhuvan] Step 5 is just a test stop criteria. Both cases we report
>     topology discovery time (as in Step 6)
>     ===========
>
>     _______________________________________________
>     bmwg mailing list
>     bmwg@ietf.org <mailto:bmwg@ietf.org>
>     https://www.ietf.org/mailman/listinfo/bmwg
>
> DISCLAIMER: Privileged and/or Confidential information may be 
> contained in this message. If you are not the addressee of this 
> message, you may not copy, use or deliver this message to anyone. In 
> such event,you should destroy the message and kindly notify the sender 
> by reply e-mail. It is understood that opinions or conclusions that do 
> not relate to the official business of the company are neither given 
> nor endorsed by the company. 