Re: [Gen-art] [bmwg] Genart last call review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07
bhuvaneswaran.vengainathan@veryxtech.com Wed, 31 January 2018 09:31 UTC
Return-Path: <bhuvaneswaran.vengainathan@veryxtech.com>
X-Original-To: gen-art@ietfa.amsl.com
Delivered-To: gen-art@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 09FDA12EBE2; Wed, 31 Jan 2018 01:31:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5Ykp20hYeJ1V; Wed, 31 Jan 2018 01:31:28 -0800 (PST)
Received: from smtpout4.netcore.co.in (smof.nsmailserv.com [202.162.237.6]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2B2F712EC85; Wed, 31 Jan 2018 01:31:27 -0800 (PST)
Received: from smtpin3.netcore.in (unknown [192.168.2.198]) by cf3.netcore.co.in (Postfix) with ESMTP id 41EA01200BA; Wed, 31 Jan 2018 15:01:05 +0530 (IST)
Received: from cloudmail14.netcore.co.in (cloudmail12.netcore.co.in [202.162.231.3]) by smtpin3.netcore.in (Postfix) with ESMTP id 5AF90FF837; Wed, 31 Jan 2018 15:01:16 +0530 (IST)
Mime-Version: 1.0
Date: Wed, 31 Jan 2018 09:31:15 +0000
Content-Type: multipart/alternative; boundary="----=_Part_852_272555570.1517391075"
Message-ID: <581dae0368675be61ae9a4cb65f77fe2@cloudmail14.netcore.co.in>
X-Mailer: AfterLogic webmail client
From: bhuvaneswaran.vengainathan@veryxtech.com
To: Stewart Bryant <stewart.bryant@gmail.com>
Cc: gen-art@ietf.org, ietf@ietf.org, bmwg@ietf.org, draft-ietf-bmwg-sdn-controller-benchmark-meth.all@ietf.org
In-Reply-To: <151733282426.27425.5227273121366749553@ietfa.amsl.com>
References: <151733282426.27425.5227273121366749553@ietfa.amsl.com>
X-Priority: 3 (Normal)
X-SMTP30-MailScanner-Information: Please contact the ISP for more information
X-MailScanner-ID: 5AF90FF837.A800E
X-SMTP30-MailScanner: Found to be clean
X-MailScanner-From: bhuvaneswaran.vengainathan@veryxtech.com
X-Cloudmilter-Processed: 1
Archived-At: <https://mailarchive.ietf.org/arch/msg/gen-art/ua5PJfHnf3XIlnPgyLDIYJusH6I>
Subject: Re: [Gen-art] [bmwg] Genart last call review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07
X-BeenThere: gen-art@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "GEN-ART: General Area Review Team" <gen-art.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/gen-art>, <mailto:gen-art-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/gen-art/>
List-Post: <mailto:gen-art@ietf.org>
List-Help: <mailto:gen-art-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/gen-art>, <mailto:gen-art-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 31 Jan 2018 09:31:32 -0000
Hi Stewart Bryant, Thank you for reviewing the draft and sharing your comments. Please find below our responses inline Thanks, Bhuvan On Tue, Jan 30, 2018 at 10:54 PM, Stewart Bryant wrote: Reviewer: Stewart Bryant Review result: Ready with Nits I am the assigned Gen-ART reviewer for this draft. The General Area Review Team (Gen-ART) reviews all IETF documents being processed by the IESG for the IETF Chair. Please treat these comments just like any other last call comments. For more information, please see the FAQ at . Document: draft-ietf-bmwg-sdn-controller-benchmark-meth-07 Reviewer: Stewart Bryant Review Date: 2018-01-30 IETF LC End Date: 2018-02-02 IESG Telechat date: Not scheduled for a telechat Summary: This is a well written comprehensive test set for SDN controllers. It could be published as is, but some thought about how to address the issues below might be helpful to the user of this technology. Major issues: None Minor issues: I find the large amount of text on Openflow that appears out of the blue in the appendix somewhat strange. The test suit is controller protocol agnostic, so I wonder why so much text is devoted to this specific SDN control protocol. If they are there by way of illustrative example of packet exchanges, it might be useful to the reader to point to them from the measurement text. [Bhuvan] We agree with you. But we wanted to explain this methodology using some well established protocol for better understanding. Since OpenFlow is widely used , we have given illustration of this methodology using OF protocol Something I am slightly surprised by is the lack of statistical sophistication. Average is a very crude metric giving no information on the distribution of the results. [Bhuvan] We are planning to add variance to all tests besides average metric. Hope this address your concern. I imagine that it is now ingrained in this aspect of the industry to specify graphs and tables, but I would have expected that the results would be specified in some machine readable format such as xml for input to a database rather than in the human readable format that is hard coded into this specification. [Bhuvan] We are planning to leave this option to individuals performing the methodology. Nits/editorial comments: Abstract This document defines the methodologies for benchmarking control plane performance of SDN controllers. Terminology related to benchmarking SDN controllers is described in the companion terminology document. SB> It would be convenient to the reader to provide the reference to or name of SB> the companion document - the twin of the comment in the other review. SB> it would also be useful to include such a reference early in the main text. [Bhuvan] We will specify the name of the companion document explicitly ============= 4. Test Considerations 4.1. Network Topology The test cases SHOULD use Leaf-Spine topology with at least 1 Network Device in the topology for benchmarking. SB> Leaf-Spine could use a reference. In Fig 2 I am not sure this is SL rather than SB> a linear sequence of nodes. There is a better SL diagram later in the SB> document and it would be useful to the reader to forward reference it.[Bhuvan] We will work on the figures to reflect Leaf-Spine topo ======== The test traffic generators TP1 and TP2 SHOULD be connected to the first and the last leaf Network Device. SB> I am sure I know what does first and last mean, but the meaning should be called out. [Bhuvan] I agree with you. We mean TP1 should be connected to the flow source endpoint and TP2 should be connected to the flow destination endpoint. ========= Procedure: 5. Stop the trial when the discovered topology information matches the deployed network topology, or when the discovered topology information return the same details for 3 consecutive queries. SB> What do you report in the latter case? [Bhuvan] Step 5 is just a test stop criteria. Both cases we report topology discovery time (as in Step 6) =========== _______________________________________________ bmwg mailing list bmwg@ietf.org (mailto:bmwg@ietf.org) https://www.ietf.org/mailman/listinfo/bmwg (https://www.ietf.org/mailman/listinfo/bmwg) DISCLAIMER: Privileged and/or Confidential information may be contained in this message. If you are not the addressee of this message, you may not copy, use or deliver this message to anyone. In such event,you should destroy the message and kindly notify the sender by reply e-mail. It is understood that opinions or conclusions that do not relate to the official business of the company are neither given nor endorsed by the company.
- [Gen-art] Genart last call review of draft-ietf-b… Stewart Bryant
- Re: [Gen-art] [bmwg] Genart last call review of d… bhuvaneswaran.vengainathan