[bmwg] Review comments of draft-rosa-bmwg-vnfbench

"Luis M. Contreras" <contreras.ietf@gmail.com> Wed, 31 July 2019 15:33 UTC

Return-Path: <contreras.ietf@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 75B121202EA for <bmwg@ietfa.amsl.com>; Wed, 31 Jul 2019 08:33:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id QlIpavLVEKh2 for <bmwg@ietfa.amsl.com>; Wed, 31 Jul 2019 08:33:14 -0700 (PDT)
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 310501202CA for <bmwg@ietf.org>; Wed, 31 Jul 2019 08:33:14 -0700 (PDT)
Received: by mail-ed1-x52b.google.com with SMTP id v15so66118002eds.9 for <bmwg@ietf.org>; Wed, 31 Jul 2019 08:33:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=/8uU5SI08uXSRKZwILrgNfz8o7UVqiV7IspO33QljLY=; b=Iv49Hgw/X7Z4H0RVqCxQhtdC4JoIPi3WHDXCKzgDfpXJElWPa2PhVynQHAUrUfNYib 5nuFPveBACLUqm5WILWicNPp3MsYJIA76XWBLYwTD3Baht9roewUFBqNrWPChJ2smNmH vZJtDhgEmmbeMxA5guEl4K5IEeraZ24gX5KwbUnmjToecAUE+di93JVLJYrQ7zDdtSm6 Wq7/JZZ/8m3R3Df584J3jyNWZXmO8HKJ1aqWLsk8W7sxdnXZ9Bcagf35GStUF9kAnwvx 5tL1wqqk3yH8l42ScV1TQFR4DYPCJ55pGaXUfKRv9vBWiQMtGDDsUMEeRywm4vducBfN mTEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=/8uU5SI08uXSRKZwILrgNfz8o7UVqiV7IspO33QljLY=; b=m4XYhZRKdn9D9rwzYZ6icA8/5wNUVaNAClMAF8ACfgJ21DQWMhxvk0y9E8KCQTQMXm dL3QqtebfzObgwZuRxNIoeKue7paAT8nkyIQY2zX6Od/JmiC7yabPF8QJRl2fTa/C4BK UIjR9CeWYtB6cB+8IA6dk+VPO2Da0xCQMHmlmmeDfQRxYD9DIj1/3mKWUG3u/xWTeg3u Wbu40+RH5tE140NE9oEVLzrq/DvoYAYHBRbzDMXJhfSmQoPwzZ4hZfKr+jQgHZX2T8LC jCvJ/UuTpRaEOsusXjkpa0dnYf4qe5CMIhNazuxlPTub2lN50T9K92LcGi3q+L6LDSXw ZhjQ==
X-Gm-Message-State: APjAAAWAbxyndXW3W4VAjCoqfKvTI8OV0zs4o51zdc6A30CHo/OKfIrA R7v8lBpkGRKOMQDWHDoM5dHaOCtsA3DAdXtYMA8CZe/d
X-Google-Smtp-Source: APXvYqzQerTb96ypfbw3nDDtbZ8uPEEnRmR1xv3hv0L+l3Yo+L56wrB9avxLwqBX9UBYXB0I4GL/eJ47s4wPzBNyXbc=
X-Received: by 2002:a50:9929:: with SMTP id k38mr105435205edb.4.1564587192572; Wed, 31 Jul 2019 08:33:12 -0700 (PDT)
MIME-Version: 1.0
From: "Luis M. Contreras" <contreras.ietf@gmail.com>
Date: Wed, 31 Jul 2019 17:33:01 +0200
Message-ID: <CAE4dcxnUdkm=zmtmy+Ve+G=CFNTAVnasfC6_Oda+yvgxSNxAMg@mail.gmail.com>
To: bmwg@ietf.org
Cc: LUIS MIGUEL CONTRERAS MURILLO <luismiguel.contrerasmurillo@telefonica.com>
Content-Type: multipart/alternative; boundary="000000000000d55d92058efbd761"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/ZeoTNutrUeynvDIWSkMppVKMVoU>
Subject: [bmwg] Review comments of draft-rosa-bmwg-vnfbench
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 31 Jul 2019 15:33:17 -0000

Hi all,

as committed last week during BMWG session, I have performed a review
of draft-rosa-bmwg-vnfbench.

These are the comments coming from my review.

/General comments/

.- it would be good to have some text in the draft indicating how this
benchmarking methodology relates with the activities in ETSI NFV TST
working group

.- an special case for benchmarking could be the redundancy. There are
different schemas of redundancy (e.g., VNFs with active / standby VNFCs,
M:1 redundant VNFCs, non-redundant VNFCs, etc). Would the redundancy be
part of the scope of the draft/methodology that you describe? And if so,
how this could be included as part of the descriptors / setup that you

.- Can we in general terms assume that Agents represent active probing
while Monitors represent passive probing? If so, it would be probably good
to make it explicit

/Specific comments/

.- Section 5, bullet on VNF (after Fig. 1): with relation to the VNFCs, can
be those components tested individually or should them be always tested as
part of the comprehensive VNF?

.- Figure 1: the Monitor box, as depicted, it is not too much clear. The
boundaries of the box overlap with the boundaries of VNF component and
Execution environment. This could be done on purpose, but the figure
becomes a bit confusing, at least to me. Additionally, I can see arrows
to/from the agents, but no arrows to/from the monitor. From/to where the
information is send to Monitor box?

.- Section 6.1: should it be included information about the kind of VIM,
MANO, etc to use for on-boarding, managing and running the VNF? Should the
NMS of the VNF part of the tests (maybe assisting on the
configuration/collection of information)? Should it be also declared the
usage of VM, containers, etc in the setup? Where?

.- Section 6.3.1: should it be included there the duration of the tests?

.- Section 6.2.2: the VNF processing / Active metrics, are those equivalent
to the kind of metrics can be obtained from a PNF? Any difference? If not,
it could be maybe convenient to reflect that fact since same metric
description could be reused by the operators for performing the tests (and
for comparison, as well)

.- Section 6.3.2: if the Manager collects all measurements, then it has to
support some kind of interface for information retrieval, hasn’t it? If so,
it could be maybe convenient to reflect it in figure 1 and in the text.

.- Section 6.4.3: failure handling can be considered as active or passive

.- Section 8: during the VNF benchmarking, it could be considered the
running of security tests? For instance DDoS, etc. If so, it could be

/Editorial comments/

.- The Agent/Prober and Monitor/Listener bullets should be better aligned,
maybe using different levels of bullets and skipping existing space lines

.- section 6.3.2, bullet 1: s/ … compose the all the permutations … / …
compose all the permutations …

.- section 6.4.1, 1st paragraph:  s/ … to mitigateside effects … / … to
mitigate side effects …

I would like to tahnks the authors for the very good document produced so

Best regards


Luis M. Contreras
Global CTIO unit / Telefonica