[bmwg] Comments on draft-vpolak-mkonstan-bmwg-mlrsearch-02

"MORTON, ALFRED C (AL)" <acm@research.att.com> Mon, 15 July 2019 12:50 UTC

Return-Path: <acm@research.att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id B288B1200F4 for <bmwg@ietfa.amsl.com>; Mon, 15 Jul 2019 05:50:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id VIg-MTD6KpGF for <bmwg@ietfa.amsl.com>; Mon, 15 Jul 2019 05:50:23 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 10E1B12008B for <bmwg@ietf.org>; Mon, 15 Jul 2019 05:50:23 -0700 (PDT)
Received: from pps.filterd (m0048589.ppops.net []) by m0048589.ppops.net-00191d01. ( with SMTP id x6FCjEqJ008528 for <bmwg@ietf.org>; Mon, 15 Jul 2019 08:50:20 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com []) by m0048589.ppops.net-00191d01. with ESMTP id 2trqsp2b0v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for <bmwg@ietf.org>; Mon, 15 Jul 2019 08:50:20 -0400
Received: from enaf.dadc.sbc.com (localhost []) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id x6FCoJM3012284 for <bmwg@ietf.org>; Mon, 15 Jul 2019 07:50:19 -0500
Received: from zlp30495.vci.att.com (zlp30495.vci.att.com []) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id x6FCoEiQ012072 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <bmwg@ietf.org>; Mon, 15 Jul 2019 07:50:16 -0500
Received: from zlp30495.vci.att.com (zlp30495.vci.att.com []) by zlp30495.vci.att.com (Service) with ESMTP id D9F5F400AE37 for <bmwg@ietf.org>; Mon, 15 Jul 2019 12:50:14 +0000 (GMT)
Received: from clpi183.sldc.sbc.com (unknown []) by zlp30495.vci.att.com (Service) with ESMTP id BBF0B400AE35 for <bmwg@ietf.org>; Mon, 15 Jul 2019 12:50:14 +0000 (GMT)
Received: from sldc.sbc.com (localhost []) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id x6FCoEjm015882 for <bmwg@ietf.org>; Mon, 15 Jul 2019 07:50:14 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com []) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id x6FCo6sg014972 for <bmwg@ietf.org>; Mon, 15 Jul 2019 07:50:07 -0500
Received: from exchange.research.att.com (njbdcas1.research.att.com []) by mail-green.research.att.com (Postfix) with ESMTP id 00AA4E10C2 for <bmwg@ietf.org>; Mon, 15 Jul 2019 08:48:34 -0400 (EDT)
Received: from njmtexg4.research.att.com ([fe80::8cd:baa3:219e:5bd4]) by njbdcas1.research.att.com ([fe80::8c6b:4b77:618f:9a01%11]) with mapi id 14.03.0468.000; Mon, 15 Jul 2019 08:50:05 -0400
From: "MORTON, ALFRED C (AL)" <acm@research.att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Comments on draft-vpolak-mkonstan-bmwg-mlrsearch-02
Thread-Index: AdU7BZcv3hlDbgvyQHmdymYl814Aig==
Date: Mon, 15 Jul 2019 12:49:18 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CFA0ACDDC7@njmtexg4.research.att.com>
Accept-Language: en-US
Content-Language: en-US
x-originating-ip: []
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-07-15_03:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=941 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1907150154
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2YlN8IjjB7nfXQckAmKW9dnXsNU>
Subject: [bmwg] Comments on draft-vpolak-mkonstan-bmwg-mlrsearch-02
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Jul 2019 12:50:25 -0000

Hi Maciek, Vratko, and BMWG,

Here are some comments on the latest version:

Section 3 is as far as I could go today,
(as a participant)

Section 1 jumps into terms and definitions, 
relying that the reader will read the abstract first.
It's OK, and typical, that text from the Abstract
becomes part of the Introduction (and Motivation).
I would also expect to hear about FD.io CSIT in
this introduction - these are compelling experiences
in the CI/CD category of testing for NFV.

Section 1 defines many terms, but we should be re-using
as much terminology as we can, from:
(and RFC 2544 and 2889, where some definitions were expanded).

For example, in 1242:
3.5  Data link frame size

                The number of octets in the frame from the first octet
                following the preamble to the end of the FCS, if
                present, or to the last octet of the data if there
                is no FCS.
overlaps with your :
      Frame size: size of an Ethernet Layer-2 frame on the wire,
      including any VLAN tags (dot1q, dot1ad) and Ethernet FCS, but
      excluding Ethernet preamble and inter-frame gap.  Measured in

The template used in 1242 and 2285 is also helpful
during reviews. Several of the sentences you wrote would
go in the "Discussion" section of the template.

I think we'll find that you've defined many new terms that
are useful, but let's concentrate on those after removing 
the overlapping terms.

Section 2
In this background section, I think it would be appropriate
to mention the ETSI GS NFVTST009, since one of its key 
contributions is on search algorithms.

Section 3

   *  Search:

         +  Start with a large (lower_bound, upper_bound) interval
            width, that determines measurement resolution.
The "interval width" is not_equal to step size, right?
The (lower_bound, upper_bound) define the interval width, AFAICT.

It seems that you need to specify the number of rows in a table
of search values, and then the "measurement resolution" in terms of
the acceptable range of offered loads will be clear (or some similar
conclusion reached through deciding the step size first, which 
determines the number of rows in the search space table).

This terminology is important to nail down, especially when 
you include the geometric search parameter variability.