Re: [bmwg] I-D Action: draft-ietf-bmwg-mlrsearch-01.txt

"Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco)" <vrpolak@cisco.com> Mon, 12 July 2021 18:32 UTC

Return-Path: <vrpolak@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 084F13A0033 for <bmwg@ietfa.amsl.com>; Mon, 12 Jul 2021 11:32:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.696
X-Spam-Level:
X-Spam-Status: No, score=-7.696 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com header.b=idc5gs1Q; dkim=pass (1024-bit key) header.d=cisco.onmicrosoft.com header.b=qRXNpWj+
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QbXl14kyLycG for <bmwg@ietfa.amsl.com>; Mon, 12 Jul 2021 11:32:31 -0700 (PDT)
Received: from alln-iport-4.cisco.com (alln-iport-4.cisco.com [173.37.142.91]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 456613A0DA0 for <bmwg@ietf.org>; Mon, 12 Jul 2021 11:32:31 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4308; q=dns/txt; s=iport; t=1626114751; x=1627324351; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=vdGdlyKXsSArFKFxhfX/a0iBwfwyRngCl6mDY8Wvggk=; b=idc5gs1Q8QuA6glCQSTBNfp4jdcVsZAyE23ZKBFUwoZhf5oir/aZewn7 f7/XGEe20E+SatQrhauL0UlxhhUryw98HP3PqlbITlpEPAik8vYnsID8S 45ah5R7OV4ZJlYi9BK5pmuU7YxEqE4aetth/UjQGkhuKCaJ83MQYuo/H3 8=;
X-IPAS-Result: A0AQBAASiuxgl40NJK1agQmDLFF+WjcxhEiDSAOFOYhXA5oqglMDVAsBAQENAQEqCwwEAQGBYIJ0AheCYQIlOBMCBAEBAQEDAgMBAQEBBQEBBQEBAQIBBgQUAQEBAQEBAQFohWgBDIZFAQEBBAEBEBERDAEBLAwLBAIBCBEEAQEDAiYCAgIlCxUICAIEEwgagk8BglUDLwEOnH8BgToCih96gTKBAYIHAQEGBASBOQIOQYMuGIIyAwaBECqCe4QOgmeDeyccgg2BWIJiPoJiAQECAReBSIMVNoIugiAKgS8mBFECFEMBAyAKE4EmOZRulWWSEwojgwGKMZQcEqZSlgKMLphSAgQCBAUCDgEBBoFyIoFbcBUaIYJpUBkOjjiDV4UUhUpzOAIGCgEBAwmMMwEB
IronPort-PHdr: A9a23:t+ZFPhOnRUEAgB9QjtEl6ncfWUAX0o4cdiYU54YpzbVUfffr85fjO RnZ4vNgxB/MUJ7A4v1Jw+zRr+j7WGMG7JrA1RJKcJFFWxIfz8lDmQsmDZ2EBFH1avnwYH9yE MFLTlQw+Xa9PABcE9r/YFuHpHq04HYSFxzzOBAzKP7yH9vZjt+80Ka5/JiACzg=
IronPort-HdrOrdr: A9a23:TS3CBaxFKuxtdcT39VZrKrPwQr1zdoMgy1knxilNoH1uE/Bw+P rEoB1273LJYVUqNk3I++rrBEDoexq1yXcf2+Ys1NmZMDXbhA==
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-AV: E=Sophos;i="5.84,234,1620691200"; d="scan'208";a="714280988"
Received: from alln-core-8.cisco.com ([173.36.13.141]) by alln-iport-4.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 12 Jul 2021 18:32:29 +0000
Received: from mail.cisco.com (xbe-aln-007.cisco.com [173.36.7.22]) by alln-core-8.cisco.com (8.15.2/8.15.2) with ESMTPS id 16CIWTdT029119 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=OK) for <bmwg@ietf.org>; Mon, 12 Jul 2021 18:32:29 GMT
Received: from xfe-rtp-002.cisco.com (64.101.210.232) by xbe-aln-007.cisco.com (173.36.7.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.792.15; Mon, 12 Jul 2021 13:32:29 -0500
Received: from xfe-aln-005.cisco.com (173.37.135.125) by xfe-rtp-002.cisco.com (64.101.210.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.792.15; Mon, 12 Jul 2021 14:32:28 -0400
Received: from NAM10-BN7-obe.outbound.protection.outlook.com (173.37.151.57) by xfe-aln-005.cisco.com (173.37.135.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.792.15 via Frontend Transport; Mon, 12 Jul 2021 13:32:28 -0500
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EziFuyEhPtzBGS4SOq2s700iBuPkQv2YPlJrkOIIYtXlqD9BxiOA7FWa4HmoAqThvb97A8rn2rYi9HwnTuybatIOPNeoAHrHgK/69tqnVwRQ7kRYVLThgyp3yRc6ngqTHtuShbmr/oLugBfkNIfZMGtBfSEE5Qv+QKRYTYoB4rZphNjLwrfhMGnyP9MIUkdvon8wA+tF5m1fAPLucGcEq1qQ+y3sYMxjREWDIIf5VhA9k4PgRyW3JkVSiArlD/5a6aeKP8LrcNBgEkPZG9HTP/RpLdxIrvmJ/VEGwwzPlPN233KDtexRvSnkVKch8QpQoWIpYZzxcu2md9MbjWSA6g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vdGdlyKXsSArFKFxhfX/a0iBwfwyRngCl6mDY8Wvggk=; b=BxcZrDDJZsneCiQMmbBSxhOtl4bmykyV392zp+tYzkUZPttmPrPbwASWMzVrps6dz1ZtdwPMDgiw556z1yaiDEnsZJw6cV54EwAMrMxtqJ6XVV7AVfOB9hytvVqErb5yuYBiWsLHwgCqT2HuGJ5qD0yRyM0OcpykFdlMAuou/oMITBC8RA9VcezF9XYSqSAfM/MtaACgc6VH9T/rmUZXPsJbbEER2AuqaRHZqUrKcrhH5u1aACK58oVax5H6NIu5ra2lp6CPREm5p4JAkAIbe2wqK4dtDV8gdTd9WWBz1zduHdSk5ecTdvNcuIehdvLYuXS4iU4zIuSti5z3NgcyMw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vdGdlyKXsSArFKFxhfX/a0iBwfwyRngCl6mDY8Wvggk=; b=qRXNpWj+4uxmgsT+XNK5ff7rPZmS4jO7rOVmUl4yJq5vqgB15YaCVHk6NgFv5xCaq76kbgYEAntxoL6ZEhvNDUFjyR1HTV/bB8eX7YlyklX81UruUcZcFawjI9D2JB+C69UAx7Im2RNJNknwRdEdkb2pP5xeCXOzd+qRbtVYsKo=
Received: from CO6PR11MB5650.namprd11.prod.outlook.com (2603:10b6:5:35a::9) by CO6PR11MB5652.namprd11.prod.outlook.com (2603:10b6:5:35e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.21; Mon, 12 Jul 2021 18:32:26 +0000
Received: from CO6PR11MB5650.namprd11.prod.outlook.com ([fe80::fd90:fbb1:14bc:df68]) by CO6PR11MB5650.namprd11.prod.outlook.com ([fe80::fd90:fbb1:14bc:df68%2]) with mapi id 15.20.4308.026; Mon, 12 Jul 2021 18:32:26 +0000
From: "Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco)" <vrpolak@cisco.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] I-D Action: draft-ietf-bmwg-mlrsearch-01.txt
Thread-Index: AQHXd0tAS1NEXMEcy0eqGftfbU+5h6s/qBPg
Date: Mon, 12 Jul 2021 18:32:25 +0000
Message-ID: <CO6PR11MB56506F33FC0BB7673341277FBD159@CO6PR11MB5650.namprd11.prod.outlook.com>
References: <162611420967.8886.18192167043875198661@ietfa.amsl.com>
In-Reply-To: <162611420967.8886.18192167043875198661@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=cisco.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 7629c1b5-986d-4a47-acc9-08d9456365bf
x-ms-traffictypediagnostic: CO6PR11MB5652:
x-microsoft-antispam-prvs: <CO6PR11MB5652158FD77F1309D5E0DA09BD159@CO6PR11MB5652.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: n4efeMFNCJnKK9ItbUj7axpc0uqamMYmcMsBfn5dOTGSn8Fd6XKRqgYCjPb8kUE2TxdrfztmzXiF/BQepMgN0XXvpWrN8wvJufc0WCRHXLuAJmkLJes0pCU7aPvsIqc/n8ABFlO1zeXGnOioffKrdU2JJo2IXcrwzZNKmqFm70tnTk2oTYWaXBc8ECvYt7hXSb1qimVEd7woSqSMV374D68P4jJsjUOGwPK3MXku4t9vb2eJuIQ6T1KX8mJ4kJ3Y+CgbgDZNWavhy63QZ0VqcQph5slw8dby4+HsENrHvb0XjT2lxLKsdiMdM4sJOAQ3wQcxRGhVXcCfZBWSCrkDCmUibiqmuXq6ftzSIsObObeOa8NA0TQMXHOI0+MYoNEUdjeRW83i8PGVdEvnIHPtlo0Tvp9z45dog7X0VHsPjzP+RPen5ICPmWDLrSZ3j2aQAUba9f2RNcgtI1fFQL20gRrbxVOegl+V064NzGdOiNvSJOxFD8q196aehlTyGTE6SXszJhVoCq+znBioYfKzSmaFeUAV1N0Zl3nFuFL9+oQVDsTB5h92/OScExfHiZeIoLJUvfGHISrtDDu3mGKqWqHEECbE+HhoRGEm1LWdZos3FJG5BMzo+MFLPzQ6lSpjwct4yEznP4JZpb/Bo5sIe2XPfPxkirZEH1HikF9bMbdVIfkSFk6R/NqXG2cyzXUkwDFqUzU39hrIpE4BzM7D/8uumxhevZeu7nLacb3O/+utlHKIBMiAQ3uzr80+N1NN
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR11MB5650.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(346002)(376002)(136003)(39860400002)(396003)(366004)(966005)(52536014)(66946007)(33656002)(66446008)(122000001)(6916009)(5660300002)(8936002)(478600001)(83380400001)(316002)(186003)(38100700002)(86362001)(26005)(76116006)(8676002)(64756008)(66556008)(71200400001)(66574015)(7696005)(53546011)(6506007)(9686003)(66476007)(2906002)(55016002); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 4xjc90W+o/w9k//i8uYaRXxX8AKnMoWb2H3Ah/Q/hMtLxeMUD2ETbmt4V2eTut1eIDj/IHPK++sUCRmkI7FI0BJePXnTRnElBPFBPTaPMwJipRemJgmYzrn+uw/0IiqS4csxgNgBvydEsvKFbAraE40HLOE65lxhibPj5mQ2r7MNm3pypCJjsPoYD7y43lQZ7pFQ/o09Lckr5VSTkt4NB9bDoKany7b1y92myPGY5/GU9oIquTJhPVMGY6WkLAcXKhIbcH4gmrxEl2rqg+y+9GjtAhVdGkwN1USAcSHfjilUQTg+zkFM/G4/40JjUmY5ifJMareBdGCMfpNDRue48OHzhgxVgp0mmc3TKbd6myuvJ8dnM3cRV457q+k7X5klMhbEOKaiSeDlWaerl5nHGoZF5bUFMljySSINr9Hi9kz7+setEyxXvvvaEJ05qlbVLYEwDZ3GvzYXa4lPNNJfgX2Kbt0tEP/rHHJNivKkbm95ebLmMOlcsxu2Fi1VFjje7UGffSoqY2wufrNfOcZ2JYASI6E1J28Trmjq+qIUhlVdKfVTjhgZcY3Vs0YGIvO0/qQvKmBzaYdMBGsaf44RQHb2pg7QJ4d4g+nL5dDstc072zXiFzsMtVU8iAIQ71qHIKx5y1qPEnR726xjSU6Ws/IU7o+Hmet9uk1lEVVCqF+7yqix/GK6PfW9v+G/rU2TayoiK58k/4LGAXp9mETuyDbAdh2A0JkwAljsWCOx47Q/qNFEekG2rmWOp08tvitgvn/lXrSeEXQ85TBYRCWE0y+1VwED52kf0GLkTd6WyFBo0wrZrL4TUOtrKnXkx0A5HH7GtIq2hTj8mb3/Nplz/96RHtP7AcY6qrHdTjyEzhnB6Js9v0WT4AaVehhWGlNk9nOUJL/q7E4lDbjnh5idGvewSE4JV+QMAlb1zPil+uMFJ7ArPcakmMkrN9U9hmskoSNY2IUAPen+CzHB4NAjfOLvSELC6s1I8BqiDcc7h1o6pF/+KHAaATPB2O8S8EHNY9oeL03+Dma8HfVF+NUZqwBPWo9Dc+RcXpu0lCFgNMOzxSp+RQTnRbVs4fQqH0gblJAxEefUuJCRpeQkYsvcS5RLAE/lWflTXt8e0KbH4lcNCsuSANP6WZs2sD6j4ueQXo/FURJOJFRGzHI+VsMCPnox6eeYTAQsjXGNHheW5JAgT7FkSMyZ3sD051hULf4r96780RJ91WD+c/NG+nfvfuzQTvKA7s5SxUbhu7Sbcw526s3KOBgCD6QE+CwEoJxoYgJN2++rGHdLCjl/fwe1NHiiOWtL4dPVRByAC6l+c7I=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CO6PR11MB5650.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7629c1b5-986d-4a47-acc9-08d9456365bf
X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jul 2021 18:32:25.9510 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vhcXRC6clCDIghLFKJgbnSZbDXMgiScG31EWhuHvRWpE9TCbHl6/sZkGOqM1yDofPBZfSSwiU59H2P9VUbMvlg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR11MB5652
X-OriginatorOrg: cisco.com
X-Outbound-SMTP-Client: 173.36.7.22, xbe-aln-007.cisco.com
X-Outbound-Node: alln-core-8.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/WrLWniRNnyYTugjtxetJiPPos_g>
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-mlrsearch-01.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Jul 2021 18:32:36 -0000

This version describes the new logic
(few improvements since my last e-mail),
and gives some examples on how to use for finding throughput
in transactions per second, not just packets.

There is at least one small logic improvement left,
but more work is needed on terminology side.
E.g. the current draft still contains "transmit rate" and "offered load"
referring to the same thing, now called "intended load".

So there will definitely be at least one more draft version,
hopefully in not too distant future.

Vratko.

-----Original Message-----
From: bmwg <bmwg-bounces@ietf.org> On Behalf Of internet-drafts@ietf.org
Sent: Monday, 2021-July-12 20:23
To: i-d-announce@ietf.org
Cc: bmwg@ietf.org
Subject: [bmwg] I-D Action: draft-ietf-bmwg-mlrsearch-01.txt


A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Benchmarking Methodology WG of the IETF.

        Title           : Multiple Loss Ratio Search for Packet Throughput (MLRsearch)
        Authors         : Maciek Konstantynowicz
                          Vratko Polak
	Filename        : draft-ietf-bmwg-mlrsearch-01.txt
	Pages           : 21
	Date            : 2021-07-12

Abstract:
   This document proposes changes to [RFC2544], specifically to packet
   throughput search methodology, by defining a new search algorithm
   referred to as Multiple Loss Ratio search (MLRsearch for short).
   Instead of relying on binary search with pre-set starting offered
   load, it proposes a novel approach discovering the starting point in
   the initial phase, and then searching for packet throughput based on
   defined packet loss ratio (PLR) input criteria and defined final
   trial duration time.  One of the key design principles behind
   MLRsearch is minimizing the total test duration and searching for
   multiple packet throughput rates (each with a corresponding PLR)
   concurrently, instead of doing it sequentially.

   The main motivation behind MLRsearch is the new set of challenges and
   requirements posed by NFV (Network Function Virtualization),
   specifically software based implementations of NFV data planes.
   Using [RFC2544] in the experience of the authors yields often not
   repetitive and not replicable end results due to a large number of
   factors that are out of scope for this draft.  MLRsearch aims to
   address this challenge in a simple way of getting the same result
   sooner, so more repetitions can be done to describe the
   replicability.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-mlrsearch/

There is also an htmlized version available at:
https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-mlrsearch-01

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-mlrsearch-01


Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/


_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www.ietf.org/mailman/listinfo/bmwg