Re: [bmwg] Review comments of draft-rosa-bmwg-vnfbench (Luis M. Contreras)

Raphael Vicente Rosa <raphaelvrosa@gmail.com> Thu, 01 August 2019 13:23 UTC

Return-Path: <raphaelvrosa@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1123D12016D for <bmwg@ietfa.amsl.com>; Thu, 1 Aug 2019 06:23:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.997
X-Spam-Level:
X-Spam-Status: No, score=-1.997 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hqldO_YPfupJ for <bmwg@ietfa.amsl.com>; Thu, 1 Aug 2019 06:23:08 -0700 (PDT)
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3164A12015F for <bmwg@ietf.org>; Thu, 1 Aug 2019 06:23:08 -0700 (PDT)
Received: by mail-wm1-x331.google.com with SMTP id f17so63161185wme.2 for <bmwg@ietf.org>; Thu, 01 Aug 2019 06:23:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=ECy+QldJS1s7fRVfdIaDYq6gzBtFnDXuIGoJjaBBfyk=; b=Z3D4Pm9kn/ONCoa+wuIIqtQsjUC6Henm7XeIh5Yu9ukJz3Ut61ssV7g8zn9s2ogQBd Swyn8ricQY8U3tLUrlzYPx/E2fz20najSm4IhJcXjrEo3QWZeS8No5wqdL+euNTdVirx wkEERQS6dwC+o3vW1w/7YaG8v1b4TZf4KRpx6tzEhjcHCZTlUMX3H9n0D4vllaOzQ2RE +MyeKqb3GfWhA/vCpZNV4qAOinblA3Wr626WZMRyhB7xyhrwZFSGvHAl67el5Bhnhen1 Y4wxH/0kSao2K+kYK42ffzAY45za05GRgW2fuYky3IijYK1tY+Gg7LDJUURjosaX9DOm vQNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ECy+QldJS1s7fRVfdIaDYq6gzBtFnDXuIGoJjaBBfyk=; b=NPTVJkDWdO95GfUvK62ccY/qYjFGEbkL27uy5Y41bqrRSqcykG5503DYW4GneBJnOl zPth/zzxO/v+z+hjFkVnRhAaVIvp0eMGhgx6lJikN0gnrmBH2cuow7rm3hwUg/mYj8u3 ej0Y2pkqAMRbmns8g5JyD4q6VrOaOi3CCPUtoWrkB0qvpK86a67427XayxwEuBtP8gt3 fl0KRhMsNL8CgKesdDl67VDaFSrRspfOvpd2gvljCc7BIS1ltj3WzE2WEiaBsvOBx74Y Ra17UYP4EpN0BD1u9rhPCyEAZAbTf9KX4rjn9Q07eItOuXwXFdGUj2O/tk5QaCXfa+rw xxGA==
X-Gm-Message-State: APjAAAVG9go0wlLLAqa+LcPnmNNOoqvarPGqUdbHfS9NYoJDceRPtw9r H/Mw5J2Mo+gKA5uY5BqI4YZC1FA/kCRLZ0QvErz9Obgk
X-Google-Smtp-Source: APXvYqwUnROhm/xe9KpGU4LTaFKuVS5OPIcYRTbAGBYt/fT6Cj5cCQ0OvkW2WkjzmM0yXBJlQ5vHlgOV678fXKH87E4=
X-Received: by 2002:a1c:b155:: with SMTP id a82mr64405533wmf.35.1564665785848; Thu, 01 Aug 2019 06:23:05 -0700 (PDT)
MIME-Version: 1.0
References: <mailman.83.1564599616.17604.bmwg@ietf.org>
In-Reply-To: <mailman.83.1564599616.17604.bmwg@ietf.org>
From: Raphael Vicente Rosa <raphaelvrosa@gmail.com>
Date: Thu, 01 Aug 2019 10:22:54 -0300
Message-ID: <CAD-XRrXemqdPHTn2gwBcMN72fmJUiz1=CMUsnRoNE2GPgOWw3A@mail.gmail.com>
To: bmwg@ietf.org
Content-Type: multipart/alternative; boundary="0000000000005b9c92058f0e24f7"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/j3miHAMyQ-ZyzdpnlMDqHGPRP3E>
Subject: Re: [bmwg] Review comments of draft-rosa-bmwg-vnfbench (Luis M. Contreras)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Aug 2019 13:23:12 -0000

Luis, thanks a lot for the great review!
We appreciate the engagement, it sure will help us clarify and improve the
draft content for the next version, a path for the bmwg adoption.

Best regards,
(The authors)

On Wed, Jul 31, 2019 at 4:01 PM <bmwg-request@ietf.org> wrote:

> Send bmwg mailing list submissions to
>         bmwg@ietf.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://www.ietf.org/mailman/listinfo/bmwg
> or, via email, send a message with subject or body 'help' to
>         bmwg-request@ietf.org
>
> You can reach the person managing the list at
>         bmwg-owner@ietf.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of bmwg digest..."
>
>
> Today's Topics:
>
>    1. Review comments of draft-rosa-bmwg-vnfbench (Luis M. Contreras)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 31 Jul 2019 17:33:01 +0200
> From: "Luis M. Contreras" <contreras.ietf@gmail.com>
> To: bmwg@ietf.org
> Cc: LUIS MIGUEL CONTRERAS MURILLO
>         <luismiguel.contrerasmurillo@telefonica.com>
> Subject: [bmwg] Review comments of draft-rosa-bmwg-vnfbench
> Message-ID:
>         <CAE4dcxnUdkm=zmtmy+Ve+G=
> CFNTAVnasfC6_Oda+yvgxSNxAMg@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi all,
>
> as committed last week during BMWG session, I have performed a review
> of draft-rosa-bmwg-vnfbench.
>
> These are the comments coming from my review.
>
> /General comments/
>
> ..- it would be good to have some text in the draft indicating how this
> benchmarking methodology relates with the activities in ETSI NFV TST
> working group
>
> ..- an special case for benchmarking could be the redundancy. There are
> different schemas of redundancy (e.g., VNFs with active / standby VNFCs,
> M:1 redundant VNFCs, non-redundant VNFCs, etc). Would the redundancy be
> part of the scope of the draft/methodology that you describe? And if so,
> how this could be included as part of the descriptors / setup that you
> describe?
>
> ..- Can we in general terms assume that Agents represent active probing
> while Monitors represent passive probing? If so, it would be probably good
> to make it explicit
>
>
>
> /Specific comments/
>
> ..- Section 5, bullet on VNF (after Fig. 1): with relation to the VNFCs,
> can
> be those components tested individually or should them be always tested as
> part of the comprehensive VNF?
>
> ..- Figure 1: the Monitor box, as depicted, it is not too much clear. The
> boundaries of the box overlap with the boundaries of VNF component and
> Execution environment. This could be done on purpose, but the figure
> becomes a bit confusing, at least to me. Additionally, I can see arrows
> to/from the agents, but no arrows to/from the monitor. From/to where the
> information is send to Monitor box?
>
> ..- Section 6.1: should it be included information about the kind of VIM,
> MANO, etc to use for on-boarding, managing and running the VNF? Should the
> NMS of the VNF part of the tests (maybe assisting on the
> configuration/collection of information)? Should it be also declared the
> usage of VM, containers, etc in the setup? Where?
>
> ..- Section 6.3.1: should it be included there the duration of the tests?
>
> ..- Section 6.2.2: the VNF processing / Active metrics, are those
> equivalent
> to the kind of metrics can be obtained from a PNF? Any difference? If not,
> it could be maybe convenient to reflect that fact since same metric
> description could be reused by the operators for performing the tests (and
> for comparison, as well)
>
> ..- Section 6.3.2: if the Manager collects all measurements, then it has to
> support some kind of interface for information retrieval, hasn?t it? If so,
> it could be maybe convenient to reflect it in figure 1 and in the text.
>
> ..- Section 6.4.3: failure handling can be considered as active or passive
> testing?
>
> ..- Section 8: during the VNF benchmarking, it could be considered the
> running of security tests? For instance DDoS, etc. If so, it could be
> mentioned.
>
>
>
> /Editorial comments/
>
> ..- The Agent/Prober and Monitor/Listener bullets should be better aligned,
> maybe using different levels of bullets and skipping existing space lines
>
> ..- section 6.3.2, bullet 1: s/ ? compose the all the permutations ? / ?
> compose all the permutations ?
>
> ..- section 6.4.1, 1st paragraph:  s/ ? to mitigateside effects ? / ? to
> mitigate side effects ?
>
>
>
> I would like to tahnks the authors for the very good document produced so
> far.
>
>
> Best regards
>
>
> Luis
>
>
> --
> ___________________________________________
> Luis M. Contreras
> contreras.ietf@gmail.com
> luismiguel.contrerasmurillo@telefonica.com
> Global CTIO unit / Telefonica
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://mailarchive.ietf.org/arch/browse/bmwg/attachments/20190731/85826251/attachment.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg
>
>
> ------------------------------
>
> End of bmwg Digest, Vol 178, Issue 14
> *************************************
>