Re: [bmwg] New Version for draft Methodology for VNF Benchmarking Automation

Vladimir Vassilev <vladimir@lightside-instruments.com> Mon, 02 November 2020 02:41 UTC

Return-Path: <vladimir@lightside-instruments.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E025A3A0C53 for <bmwg@ietfa.amsl.com>; Sun, 1 Nov 2020 18:41:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.148
X-Spam-Level:
X-Spam-Status: No, score=-2.148 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, MSGID_FROM_MTA_HEADER=0.001, NICE_REPLY_A=-0.247, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=netorgft4991094.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hh5xReFc0XJr for <bmwg@ietfa.amsl.com>; Sun, 1 Nov 2020 18:41:24 -0800 (PST)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150043.outbound.protection.outlook.com [40.107.15.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7F0103A0C4F for <bmwg@ietf.org>; Sun, 1 Nov 2020 18:41:24 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SEE/n//MTkCQ4PyA1NMvpewJ409W9Ssbr0Ii63OdnZdfHVbEd0Fcn8I8oWUQtDR3Bji+CHltQqiExgZ1CrPld9MOxQkYYWhtyreU3+TMVjQJienRjXocgdvV3Rpwt/ihieFnWH9VoqLFUUtPrvRF7E/KuV/9L/BOtePs/sfc8na1+V7hY9nC5JNWF6PI+WNEybrNn7dRf994ZltPhgi6BOaxg1G9O+3Hv2x+vxMZpIfOb3Al6nWONaUucCwBmPZ9lM5hbJ0joma+2WHAMIvEVYngz8oW+wbeyDp0aIpnl98B2CyNLkIJZ3j177eDqghMV2KGVgkc87EGiIWr8IfpSw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A9oHSPCpomyY4eLE92FQ05IKl8Cs5Dldz+V21MR8qEY=; b=F3uBkDdq1XkozIORQDtpSCIPJocSqc6EoOnR6F8Y+uI3/+Q3ZZ5SDWLgfc+Re3FrKepBGB/fdoRXRDxS8sKClpnyPfQmYo2HisSQXgfM1UN19Uk7WDkNKPGunwVTf0tcuuvYFtdsX4kNBzLDtue0QhPWOEVFFxXHRyqydmdwrLoeBv8uCioSQwogUi2I7jWco87s0eeSgaBoh3CUVMgvtMXtT3KsIIek5qMxiSVvqtpMnhmAHKn+lnFsLYdUy9Ff4Iw9i7qBg0OqQ8bDONCZ7/GHTaDGt0U+kTFlx//UDE9Txgypn0dZSs9GiotJVVsmo85MQRYGYS5+bz1HSv8hKg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=lightside-instruments.com; dmarc=pass action=none header.from=lightside-instruments.com; dkim=pass header.d=lightside-instruments.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NETORGFT4991094.onmicrosoft.com; s=selector2-NETORGFT4991094-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A9oHSPCpomyY4eLE92FQ05IKl8Cs5Dldz+V21MR8qEY=; b=KXfBj0cJRdoe/wEFFekibThJxn6pHNxWsZxgNdfuDXEU8DahEeovRQlZNDSrTjg52XDwGcpK9Hjglpx/auy5+/bsfxT6kw3RroJ5n4nlDiTYsTd9aa3TXwnFWnmeFWS+S4OU4M9bJLboO+AyXnXqc9+foVVj147p+ALuwHultTE=
Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=lightside-instruments.com;
Received: from AM0PR08MB4084.eurprd08.prod.outlook.com (2603:10a6:208:129::25) by AM9PR08MB6241.eurprd08.prod.outlook.com (2603:10a6:20b:281::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3499.18; Mon, 2 Nov 2020 02:41:10 +0000
Received: from AM0PR08MB4084.eurprd08.prod.outlook.com ([fe80::ed8a:57b3:81d6:5d19]) by AM0PR08MB4084.eurprd08.prod.outlook.com ([fe80::ed8a:57b3:81d6:5d19%5]) with mapi id 15.20.3499.030; Mon, 2 Nov 2020 02:41:10 +0000
To: Raphael Vicente Rosa <raphaelvrosa@gmail.com>
References: <CAD-XRrWqr1uNoHX9km7UA_LiL5hV4FMjRgLmLRe0HK=i5+5CGg@mail.gmail.com> <4D7F4AD313D3FC43A053B309F97543CF0147618556@njmtexg4.research.att.com> <CAD-XRrV9YFTO7i7bpWkypU3cithM3E1G4+5Qag-4obmg3GUSXg@mail.gmail.com>
From: Vladimir Vassilev <vladimir@lightside-instruments.com>
Cc: "bmwg@ietf.org" <bmwg@ietf.org>
Message-ID: <4c828aaf-9443-d3f2-d080-768316a8ca48@lightside-instruments.com>
Date: Mon, 2 Nov 2020 03:41:05 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0
In-Reply-To: <CAD-XRrV9YFTO7i7bpWkypU3cithM3E1G4+5Qag-4obmg3GUSXg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Originating-IP: [2001:840:4b0b:1337:c68e:8fff:fef3:82a7]
X-ClientProxiedBy: HE1PR0301CA0020.eurprd03.prod.outlook.com (2603:10a6:3:76::30) To AM0PR08MB4084.eurprd08.prod.outlook.com (2603:10a6:208:129::25)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [IPv6:2001:840:4b0b:1337:c68e:8fff:fef3:82a7] (2001:840:4b0b:1337:c68e:8fff:fef3:82a7) by HE1PR0301CA0020.eurprd03.prod.outlook.com (2603:10a6:3:76::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3499.22 via Frontend Transport; Mon, 2 Nov 2020 02:41:08 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: d9d87328-d5fb-4f0c-2fe5-08d87ed8c0d3
X-MS-TrafficTypeDiagnostic: AM9PR08MB6241:
X-Microsoft-Antispam-PRVS: <AM9PR08MB624112D65DCC2776FAE1414A9B100@AM9PR08MB6241.eurprd08.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:6790;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: Pny8TFcZrVTSaUw1QVFm5ZhYZp3N6Yon7SSQ/X5JpzWpWLCm248tbygcxRbzYnaJLV3TOSxE9tG/LEN1ORPopilWcADNfr/bkKJnaBHQ3D9kaJLnzDdSoyx6oZ94hZOa4XfW+qJUo75lvSDRlhCCm5HDxk5cxn7DEURZrAtojjRJT4Bfw2ReAr5vSiX364dS10fvAKgQC1SDvjFXPQQy78UxnU0Je7aFc7Maxsz0Nb5157l42C4uEEl4SbM9TUhua/iDwkxKxOGujFxTECF+Q8EKgKqkv5j+XV5sknEZh2nI/mg2gmCm9kQ59C8lsyt9wZMK4OvjHb5hbhi2u1wkd6uT7Wb1vXdmyq+DxJlIOmvhLtM4BsYyeGGOJnxaX5/+
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR08MB4084.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(396003)(39830400003)(366004)(346002)(136003)(376002)(4326008)(16526019)(478600001)(31686004)(8936002)(83380400001)(2616005)(6666004)(186003)(2906002)(86362001)(8676002)(52116002)(31696002)(36756003)(66946007)(66556008)(66476007)(316002)(6916009)(6486002)(5660300002)(43740500002); DIR:OUT; SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: TvmAgp6Xg2RMxM5MDRn8HKJVjjf1zAdhWRNpLFM6sUmFJ68Z37Y/iF1dhXRPjZwWCuXil8DMiOJc8QgfDNiEZfx+8i4PHFOSgvSFbQyijCb7Wt+2K8AJUWDQDhcsI8apB4XjMmo1xFulRonLPEjfPTE5PcmLguC+Oc7mqz7mFjY4rvQnf5u9MV0hAGoHiyEJwz8jWAvdC98n7PMEmt/G2MIbquFdQHn14W9bKBw4K6nE8rCw6Sf+YUdBAbIk1DFsDgdqohfR9aWxfFgWjNqzY83NOtGHdZT5PtqfEzOLtA1E0rX+Jd1JTQI9sRDIU4bs7b08G8dmEZUChAt4WSkhmr5RuU8+Kvs5sRe+YkfMxXq4kuqsD73hTehP9c0VxyozRuBbwS2mEikmBiicZgN1+gmRC9RKCTmPJVOTK8qmzgJ+eT8D0ZBeL+xNHahXAQoC7FhpKipp2sXR8y5C5Rpa26nE1IdSFAF7iB90glQK+E5RcVxkyiEvqp4TdxGCrtBLoy4x7OJ2Z8/I1iDGetF+pb/0JRZ6nYfA/451S5b7kWLlPwdlafrkZFB50a7vXMixvdNu+yrdo/TLCp21o3tdqqX04hopNtOdJfnpRi2sGSTa/y/hMcN3C3o6+AYFRcmY4nFbR3TEa7TN+k/o7oWs7z1vBJla1TlM3KY0Sih5gGAYsdwjrK9wkncKFoSRWWKrHTBHz3qKqtWbO/NWVwtN2g==
X-OriginatorOrg: lightside-instruments.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d9d87328-d5fb-4f0c-2fe5-08d87ed8c0d3
X-MS-Exchange-CrossTenant-AuthSource: AM0PR08MB4084.eurprd08.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Nov 2020 02:41:10.0208 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: c0326317-f373-4461-a96f-7946e0abb603
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: T6r8D2vfPwn1U7R58CRy5ufM3EieWkUKVgA2V4Ut8223tnUiiuqF5buFfXmEljPnp3gjziqTMkmx3tpP4+bBqeEiI05TkDWRImm3olepk39Svf6EPE7AXa7qOKWIuAvk
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6241
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/OySZIak99RN9Nwj4PyO1yetoeZo>
Subject: Re: [bmwg] New Version for draft Methodology for VNF Benchmarking Automation
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Nov 2020 02:41:26 -0000

Hi Raphael,

On 20/10/2020 21.16, Raphael Vicente Rosa wrote:
> ...
> 7. The reference implementation realizes all the methodology (getting 
> some results to present).

I am in the process of reviewing the draft. Can you post a link to some 
results (XML instance data) of a benchmark report generated according to 
the YANG data model?

That said here are some notes I made so far:

* I am a bit uncertain about what "VNF-BD YANG path of the variable" is? 
Have you considered using "type instance-identifier;"?

        leaf path {
          type string;
          description "The VNF-BD YANG path of the
          variable.";
        }

* IMO The short description in "5.3.  VNF Benchmarking Report (VNF-BR)" 
can be improved with some examples. Appendix with complete example 
report data instance file.

* IMO rfc8345 should be used for the topology definitions that are 
significant part of the /vnf-bd:scenario - nodes, connection-points, 
links etc. ietf-network-topology@2018-02-26.yang can be augmented with 
additional context specific data.

* ETS14a,b,c,e reference links have typos and do not work. ETS14d, f  work.

* use "ietf-yang-types:date-and-time" instead of "string" for timestamps.

* use "ietf-yang-types:uuid" types instead of "string"

* Add YANG tree diagrams:

    $ pyang -f tree  --path 
/usr/share/yuma/modules/ietf:/usr/share/yuma/modules/ietf-drafts:. 
vnf-bd.yang

    module: vnf-bd
      +--rw experiments
      |  +--rw trials?   uint32
      |  +--rw tests?    uint32
     +--rw scenario
      |  +--rw nodes* [id]

...

* Drafts with YANG modules should use <CODE BEGINS> file ... <CODE ENDS> 
so that the modules are detected and validated by the IETF tools.

/Vladimir