[bmwg] Genart telechat review of draft-ietf-bmwg-sdn-controller-benchmark-term-09

Stewart Bryant <stewart.bryant@gmail.com> Mon, 16 April 2018 12:22 UTC

Return-Path: <stewart.bryant@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E21D127337; Mon, 16 Apr 2018 05:22:06 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Stewart Bryant <stewart.bryant@gmail.com>
To: <gen-art@ietf.org>
Cc: ietf@ietf.org, bmwg@ietf.org, draft-ietf-bmwg-sdn-controller-benchmark-term.all@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.78.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <152388132614.20870.2549463758105874964@ietfa.amsl.com>
Date: Mon, 16 Apr 2018 05:22:06 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/4CiWbkKWYyFH2FgZgR8bBKBd7ZI>
Subject: [bmwg] Genart telechat review of draft-ietf-bmwg-sdn-controller-benchmark-term-09
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Apr 2018 12:22:06 -0000

Reviewer: Stewart Bryant
Review result: Ready with Nits

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair. Please wait for direction from your
document shepherd or AD before posting a new version of the draft.

For more information, please see the FAQ at


Document: draft-ietf-bmwg-sdn-controller-benchmark-term-09
Reviewer: Stewart Bryant
Review Date: 2018-04-16
IETF LC End Date: 2018-02-02
IESG Telechat date: 2018-04-19

Summary: Generally a well written document. The various comments I make are
mostly editorial with some on the fringe of being technical. I do not seem to
have received a response to a number of issues raised, and as far as I can see
there is no change to the text in the issues noted below. Apologies if I have
missed your email.

Major issues: None

Minor issues:

I do not seem to have received a response to this issue, and as far as I can
see there is no change to the text. Apologies if I have missed your email.

*rate* implies actions per unit time, but this does not seem to feature in your
definition. Asynchronous Message Processing Rate

   The number responses to asynchronous messages (such as new flow
SB> That should be the number of responses per second.

   As SDN assures flexible network and agile provisioning, it is
   important to measure how many network events the controller can
   handle at a time. This benchmark is obtained by sending asynchronous
   messages from every connected Network Device at the rate that the
   controller processes (without dropping them).

SB> So what you are testing here is the control network and the
SB> controller. This is perhaps the only practical way to run the
SB> test, but it seems a pity that you do not deconvolve these
SB> two aspects of the test.
SB> I suppose this is really network Async Msg Proc rate rather than
SB> controller Async proc rate.
SB> We may get to this in the companion document, but doesn't there
SB> need to be some standardization of the event message to compare
SB> apple with apples over time?

Nits/editorial comments:

2.2.4. Number of Cluster nodes

   This parameter is relevant when testing the controller performance
   in clustering/teaming mode. The number of nodes in the cluster MUST
   be greater than 1.

SB> I see what you are saying, but you may wish to clarify that this
SB> constraint does not apply all the time. For example one of two nodes
SB> may start later than another, or fail, or maybe I worry over nothing here. Control Sessions Capacity

Measurement Units:
SB> Surely this should be in units of sessions? Network Discovery Size

Measurement Units:


SB> How can this be N/A surely it is a number of network units of various type. Forwarding Table Capacity