Re: [bmwg] New Version for draft Methodology for VNF Benchmarking Automation

Raphael Vicente Rosa <raphaelvrosa@gmail.com> Tue, 17 November 2020 14:25 UTC

Return-Path: <raphaelvrosa@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A84A3A1272 for <bmwg@ietfa.amsl.com>; Tue, 17 Nov 2020 06:25:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wC0ua-laC1KX for <bmwg@ietfa.amsl.com>; Tue, 17 Nov 2020 06:25:00 -0800 (PST)
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 20D393A1375 for <bmwg@ietf.org>; Tue, 17 Nov 2020 06:25:00 -0800 (PST)
Received: by mail-wm1-x32c.google.com with SMTP id w24so3414510wmi.0 for <bmwg@ietf.org>; Tue, 17 Nov 2020 06:25:00 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=nZkNb7t3zAag4AEvuAr7/82KJ54swjjlHkzs7Ivzhow=; b=UV5ADGKFdT6tYKsvwRZeLehjTaIuAUNgeKN2oZ6rYvOwg05BawHLmAm+oT1T+a7lvm ldXJcWuhx6LS1wwv9Tvc/D2F2lV+ASIN7ZOpO0Pfv5QfMqgU19ZPfxi/XtqxJzqCiR8K oG1AAqMUA48LTNtxgTL8XTgMnNITdk1OY8WWhHBilx/I42zX9i77WXtTK7Vm7W3I5ngH 1U3qaUS1vY6S8NHuoOdbtoq1OxECphCtPIspXU7t8fvpP0+bOmCIPUURaDJbI6YNBcMG zdZLqZAAqg//DizKCIvxtfx5vKmOTOsg/uW6OtaXP72mATJ9fKSCiNXwpdq5n20ivmgP wAAA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=nZkNb7t3zAag4AEvuAr7/82KJ54swjjlHkzs7Ivzhow=; b=YZIQHI5qEW3ub8RB/bUM0KqPOm6m52y0dQWbS8BgsOhCJDYfPeVnADfZDMOYq/mgIN GAU7p6K1eZwcz3BUi6CUIOIfQiTZJrmBP6m5Au3yMmk6bYCTTvX4rtICWPnuySEo9J4D VGQrd+MiUQjtSs1Tc0UtdoP85+Xh2d3eBjtNZ7mSX/LiQ+Cm2XGyKMmm47cgl9tLFIEV s+CClGsxdlrP6NxTl9sB+3avKcI6lf+bHKgY86whpDDgJOxsayhMbfW0jCNz6fqEPVXU U4ynbZB1HhlleFXL5twx/LhrTwqkNTl7FBTVkdfRBvBiMQTbMbrYQg/v7a3s4D1ZOePB m7VQ==
X-Gm-Message-State: AOAM5326M4nKGke82Fm5d5dyMz7tcdl96B3h+icyEGuqTsF3P2lZy3xU tCVscszfx6qlYRV+Ztg7ZASlmx5tjuoYJy/B/KwzRpKel/I=
X-Google-Smtp-Source: ABdhPJy1oneAnBDG5eKhWcL+NrE5FAn7VkHuwHcBaRXVlWlJSyf9LLMP+XoRXgz4p2SGBcSdtOID+TkD4YbxIQWnQHM=
X-Received: by 2002:a1c:a9c4:: with SMTP id s187mr4749432wme.180.1605623098670; Tue, 17 Nov 2020 06:24:58 -0800 (PST)
MIME-Version: 1.0
References: <CAD-XRrWqr1uNoHX9km7UA_LiL5hV4FMjRgLmLRe0HK=i5+5CGg@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF0147618556@njmtexg4.research.att.com> <CAD-XRrV9YFTO7i7bpWkypU3cithM3E1G4+5Qag-4obmg3GUSXg@mail.gmail.com> <4c828aaf-9443-d3f2-d080-768316a8ca48@lightside-instruments.com>
In-Reply-To: <4c828aaf-9443-d3f2-d080-768316a8ca48@lightside-instruments.com>
From: Raphael Vicente Rosa <raphaelvrosa@gmail.com>
Date: Tue, 17 Nov 2020 11:24:47 -0300
Message-ID: <CAD-XRrUrfWsWkUpsBzmsLVo2m3z0Bb=qdvWQU-aezEN7LbAKmA@mail.gmail.com>
To: Vladimir Vassilev <vladimir@lightside-instruments.com>
Cc: "bmwg@ietf.org" <bmwg@ietf.org>
Content-Type: multipart/alternative; boundary="00000000000070809805b44e42f1"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/SObPYkhfgdrWEJ7gjslBzVxIiRY>
Subject: Re: [bmwg] New Version for draft Methodology for VNF Benchmarking Automation
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 17 Nov 2020 14:25:02 -0000

Hi Vladimir,

Thanks for the comments, I'm reading your draft too.

I really appreciate the comments about the YANG modules, indeed I'm not
experienced with them. With your comments I'm going to extensively review
them and update the draft.

Regarding the examples of parsed YANG modules, you can check the link
https://github.com/raphaelvrosa/gym/tree/master/examples/results
The experiments realized by gym with vnf-br[1,2,3] contain demonstrations
of simple prototypes using containers only. That is sufficient to elaborate
the draft content and showcase the realization of the proposed methodology.
Since gym is almost stable, I'm going to implement cases using more robust
traffic generators (moongen, t-rex, etc) and different scenarios, besides
containers.
You can also try the experiment by yourself,
https://github.com/raphaelvrosa/gym.

When I come up with the draft updates I'll properly answer your email
suggestions.

Thanks,
Raphael




On Sun, Nov 1, 2020 at 11:41 PM Vladimir Vassilev <
vladimir@lightside-instruments.com> wrote:

> Hi Raphael,
>
> On 20/10/2020 21.16, Raphael Vicente Rosa wrote:
> > ...
> > 7. The reference implementation realizes all the methodology (getting
> > some results to present).
>
> I am in the process of reviewing the draft. Can you post a link to some
> results (XML instance data) of a benchmark report generated according to
> the YANG data model?
>
> That said here are some notes I made so far:
>
> * I am a bit uncertain about what "VNF-BD YANG path of the variable" is?
> Have you considered using "type instance-identifier;"?
>
>         leaf path {
>           type string;
>           description "The VNF-BD YANG path of the
>           variable.";
>         }
>
> * IMO The short description in "5.3.  VNF Benchmarking Report (VNF-BR)"
> can be improved with some examples. Appendix with complete example
> report data instance file.
>
> * IMO rfc8345 should be used for the topology definitions that are
> significant part of the /vnf-bd:scenario - nodes, connection-points,
> links etc. ietf-network-topology@2018-02-26.yang can be augmented with
> additional context specific data.
>
> * ETS14a,b,c,e reference links have typos and do not work. ETS14d, f  work.
>
> * use "ietf-yang-types:date-and-time" instead of "string" for timestamps.
>
> * use "ietf-yang-types:uuid" types instead of "string"
>
> * Add YANG tree diagrams:
>
>     $ pyang -f tree  --path
> /usr/share/yuma/modules/ietf:/usr/share/yuma/modules/ietf-drafts:.
> vnf-bd.yang
>
>     module: vnf-bd
>       +--rw experiments
>       |  +--rw trials?   uint32
>       |  +--rw tests?    uint32
>      +--rw scenario
>       |  +--rw nodes* [id]
>
> ...
>
> * Drafts with YANG modules should use <CODE BEGINS> file ... <CODE ENDS>
> so that the modules are detected and validated by the IETF tools.
>
> /Vladimir
>
>