[bmwg] Genart telechat review of draft-ietf-bmwg-sdn-controller-benchmark-term-09

Stewart Bryant <stewart.bryant@gmail.com> Mon, 16 April 2018 12:22 UTC

Return-Path: <stewart.bryant@gmail.com>
X-Original-To: bmwg@ietf.org
Delivered-To: bmwg@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E21D127337; Mon, 16 Apr 2018 05:22:06 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: Stewart Bryant <stewart.bryant@gmail.com>
To: gen-art@ietf.org
Cc: ietf@ietf.org, bmwg@ietf.org, draft-ietf-bmwg-sdn-controller-benchmark-term.all@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.78.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <152388132614.20870.2549463758105874964@ietfa.amsl.com>
Date: Mon, 16 Apr 2018 05:22:06 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/4CiWbkKWYyFH2FgZgR8bBKBd7ZI>
Subject: [bmwg] Genart telechat review of draft-ietf-bmwg-sdn-controller-benchmark-term-09
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Apr 2018 12:22:06 -0000

Reviewer: Stewart Bryant
Review result: Ready with Nits

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair. Please wait for direction from your
document shepherd or AD before posting a new version of the draft.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-sdn-controller-benchmark-term-09
Reviewer: Stewart Bryant
Review Date: 2018-04-16
IETF LC End Date: 2018-02-02
IESG Telechat date: 2018-04-19

Summary: Generally a well written document. The various comments I make are
mostly editorial with some on the fringe of being technical. I do not seem to
have received a response to a number of issues raised, and as far as I can see
there is no change to the text in the issues noted below. Apologies if I have
missed your email.

Major issues: None

Minor issues:

I do not seem to have received a response to this issue, and as far as I can
see there is no change to the text. Apologies if I have missed your email.

*rate* implies actions per unit time, but this does not seem to feature in your
definition.

2.3.1.3. Asynchronous Message Processing Rate

Definition:
   The number responses to asynchronous messages (such as new flow
SB> That should be the number of responses per second.

Discussion:
   As SDN assures flexible network and agile provisioning, it is
   important to measure how many network events the controller can
   handle at a time. This benchmark is obtained by sending asynchronous
   messages from every connected Network Device at the rate that the
   controller processes (without dropping them).

SB> So what you are testing here is the control network and the
SB> controller. This is perhaps the only practical way to run the
SB> test, but it seems a pity that you do not deconvolve these
SB> two aspects of the test.
SB>
SB> I suppose this is really network Async Msg Proc rate rather than
SB> controller Async proc rate.
SB>
SB> We may get to this in the companion document, but doesn't there
SB> need to be some standardization of the event message to compare
SB> apple with apples over time?

Nits/editorial comments:

2.2.4. Number of Cluster nodes

Discussion:
   This parameter is relevant when testing the controller performance
   in clustering/teaming mode. The number of nodes in the cluster MUST
   be greater than 1.

SB> I see what you are saying, but you may wish to clarify that this
SB> constraint does not apply all the time. For example one of two nodes
SB> may start later than another, or fail, or maybe I worry over nothing here.

2.3.2.1. Control Sessions Capacity

Measurement Units:
SB> Surely this should be in units of sessions?

2.3.2.2. Network Discovery Size

Measurement Units:

   N/A

SB> How can this be N/A surely it is a number of network units of various type.

2.3.2.3. Forwarding Table Capacity