Re: [bmwg] Throughput definition discussions - volunteers needed

"Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco)" <vrpolak@cisco.com> Wed, 24 March 2021 19:41 UTC

Return-Path: <vrpolak@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E73163A342E for <bmwg@ietfa.amsl.com>; Wed, 24 Mar 2021 12:41:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.595
X-Spam-Level:
X-Spam-Status: No, score=-9.595 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com header.b=AvCi0bo7; dkim=pass (1024-bit key) header.d=cisco.onmicrosoft.com header.b=jjlm6yta
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1wo9Bv92g7b0 for <bmwg@ietfa.amsl.com>; Wed, 24 Mar 2021 12:41:55 -0700 (PDT)
Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CE7373A342D for <bmwg@ietf.org>; Wed, 24 Mar 2021 12:41:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=57828; q=dns/txt; s=iport; t=1616614914; x=1617824514; h=from:to:subject:date:message-id:references:in-reply-to: mime-version; bh=RVjoKGh3LNvW22ZSzoSQr/EufjqR/PEz3ZJBqtd68vo=; b=AvCi0bo7CGJ0u7pU37mb99wYJUoAlU+dOHoWceTRS2kIi5yetD7mZHDa KxzqExPcVmj1tmmOrl15/SqBUguE2ZHUbHqZbdkYTVfcU5c6772CHO1+8 B1aDnaVwmOxnmiekLB9u65NyjFkBOp1qP64i7Eo3qETX7juUiEXacs3zA I=;
X-Files: image001.png : 10733
IronPort-PHdr: =?us-ascii?q?A9a23=3Akfh/jxK/OVl2CocgX9mcuZcyDhhPgJ39IxIV5?= =?us-ascii?q?5w7irlHbqWk+dH4MVfC4el25HfIUJnVrfVehLmev6PhXDkG5pCM+DAHfYdXX?= =?us-ascii?q?hAIwcMRg0Q7AcGDBEG6SZyibyEzEMlYElMw+Xa9PBtUFdrwIVrIrS764TsbA?= =?us-ascii?q?B6qMw1zK6z8EZLTiMLi0ee09tXTbgxEiSD7b6l1KUC9rB7asY8dho4xQps?= =?us-ascii?q?=3D?=
IronPort-HdrOrdr: =?us-ascii?q?A9a23=3A6LjUf6NKFU1kNcBcT5vx55DYdL4zR+YMi2?= =?us-ascii?q?QD/3taDTRIb82VkN2vlvwH1RnyzA0cQm0khMroAsi9aFvm39pQ7ZMKNbmvGD?= =?us-ascii?q?PntmyhMZ144eLZrQHIMxbVstRQ3aIIScdDIfX7B1RikILe6A63D94vzLC8gd?= =?us-ascii?q?yVrM3ZyGpgShwvTqlm4RtwBArzKDwIeCBtA50lGJ2AovdWvjbIQwVZUu2XJF?= =?us-ascii?q?kgG9LCqdrCiY78bXc9ZyIPxQGSgVqTmcbHOjeC2BN2aUIr/Z4D9iz/nxX99u?= =?us-ascii?q?GftZiAu2Lh/kvyy7gTp9f70NtEA6W3+4YoAxHhkByhaogke5DqhkFPnMiV5F?= =?us-ascii?q?wnkMbBrn4bVqwZgR68DwLFwyfF4AXu3C0j7HXv0zaj8AHeiPbkTzE3Adcpv/?= =?us-ascii?q?M/TjLl6lEttNw58KVH03PxjesvMTr8nT/w79WNahdylkDcmwtFrccviRVkIO?= =?us-ascii?q?8jQY4UibZa0FJeEZ8GEi6/wpsgCvNSAMbV4+sTWU+GbljC11MfgeCEbzAWJF?= =?us-ascii?q?OrU0ICssua33x9h3Zi1XYVw8QZgzMp6I89cZ9Z/O7JW54Y1I1mf4szV+ZQFe?= =?us-ascii?q?0BScy4BijmWhTXKl+fJlzhCeUhN2/SrYX0pJE4/vujdpBN7JZaouWDbHpo8U?= =?us-ascii?q?oJP27+A8yH25NGtjrXRn+mYDjrwsZCo5djvLnxQ6fqLD2DRFgin9DImYRbPu?= =?us-ascii?q?TrH9KIfL5GCf7qKmXjXaxT2RflZpVUIX4CFM0Z0+xLH26mk4buEMnHp+bbeP?= =?us-ascii?q?HcKP7GCjA/QF7yBXMFQXz2LMVE5Ua7R2/gjHHqKjTQU3262ag1PLnR/uAVxo?= =?us-ascii?q?RIHJZLqBIphVOw4dzOLyZDvKwwdE53O6jmjau/uGmz8Q/znidUEysYKnwQzK?= =?us-ascii?q?TrUntMqwNPGVjza6w/t9KWfn0XwGGKPQZlT8TdEBdWolN+/a7fFe3X+QkST/?= =?us-ascii?q?acdk6KhXoao3yHC6oGkqqY/MH/Z9cTFZA9QpF8Eg3NCj14kQtns314dQcBX0?= =?us-ascii?q?PTfwme0JmNvdgxPqX/f8M5qBq3KcRUwEivy3m0lIUKfD8neBKAFeSQmh0jQj?= =?us-ascii?q?JIgEYZyd5vvJOw3RC1KWU+h+wkNkZrc2r/OsMfMC21IKNJh7vsZAZ8CUCNiD?= =?us-ascii?q?DysWBoRkPas2MPm2fmMSqYPcvuP2MYkHVZ3qH2mWkELVm1d15sa3x8rI12HX?= =?us-ascii?q?nHvHE2yuOQeq+vyQKqGys/6/BYPzffbTQIJARyg9ixyR6OgT6HUW4r35M0I4?= =?us-ascii?q?Xmff4eWqCW3nOmM4uTk64aW/dS4ZZ+Ldjr29V7Gd63akuQLDniDfku1BHQrn?= =?us-ascii?q?E5ODNsoH1hlf/zwhXq4Cy523E4aMCia2hOVvUeI9uG6XLjSOvN2JJljcgtte?= =?us-ascii?q?/1K37vcLe9uNbqRi8GLgmWrX+9Tukup5wRtaUutKFrF52eVTfTznlI0Bg3Mc?= =?us-ascii?q?+crjJSfI1rpLTafoN/dc0bfCxUukAkk9mCN0MnuA37CO1WRyBls1bLe9eSp7?= =?us-ascii?q?bYo7smBUOM4BbqMV6E6itH4rPLWTCA2bNyMdN9HU1GLEwnrHJs8+OJe9eOVE?= =?us-ascii?q?Gkd+Rf8EG7NXH4erlHU6SBEagRqBE/49zgpZ7jSwPonATL+T18KeZS9mziR8?= =?us-ascii?q?W4CgeFA/RJ/Ny3Ik7kuNrd3OejyDPsDSKmYEEZj5BffUMebs5fmiAv5bdHoB?= =?us-ascii?q?SaW+jyuAY5iFNQ7jFsi0711oWn6GndG1taMQexuOQjYRBDdn6SjcrE9uCE1H?= =?us-ascii?q?PypDhdsKOzZ3ttQg=3D=3D?=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0CiCQDClVtg/49dJa1XAx0BAQEBCQE?= =?us-ascii?q?SAQUFAYIQgSMwIy4Hdlo2MQqEOINIA4U5hWOCYAOZNIJTA1QEBwEBAQoBAgE?= =?us-ascii?q?BKAoCBAEBhFACF4FlAiU4EwIDAQELAQEFAQEBAgEGBHGFYQEMhkQBAQEDAQU?= =?us-ascii?q?BHQIIARIBAS4KBAkCAgEIEQQBAQYBAQEYAQYDAgICBQ8BAQ4MFAkIAQEEEgE?= =?us-ascii?q?GAgaCY4JVAw4hAQMLoH0Cih53gTKDBAEBBoE3AoFXggcYggwHAwYFgTSCdjW?= =?us-ascii?q?DQw8BAYFAgRGDcyYcgguBEkOCKy4+gQQBgVsBAQIBgV0VCgwJEYJPNYIrgVi?= =?us-ascii?q?BSw9RAlcPHwVFRDiQYQgzgwqHWzKBMIFsiSKGBYMgiDYKIYJlh2wCgWqTKIN?= =?us-ascii?q?IimqWFoZEjEuBdoIOiVKSQwSEXgICAgIEBQIOAQEGgWsjgVMGcBWDJFAXAg2?= =?us-ascii?q?OHwwWg02FFIVFcwIBNQIGAQkBAQMJfIR4LYEHAYEOAQE?=
X-IronPort-AV: E=Sophos;i="5.81,275,1610409600"; d="png'150?scan'150,208,217,150";a="851307828"
Received: from rcdn-core-7.cisco.com ([173.37.93.143]) by rcdn-iport-3.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 24 Mar 2021 19:41:38 +0000
Received: from mail.cisco.com (xbe-aln-001.cisco.com [173.36.7.16]) by rcdn-core-7.cisco.com (8.15.2/8.15.2) with ESMTPS id 12OJfcvO000993 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=OK) for <bmwg@ietf.org>; Wed, 24 Mar 2021 19:41:38 GMT
Received: from xhs-rtp-001.cisco.com (64.101.210.228) by xbe-aln-001.cisco.com (173.36.7.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.3; Wed, 24 Mar 2021 14:41:37 -0500
Received: from xfe-rcd-004.cisco.com (173.37.227.252) by xhs-rtp-001.cisco.com (64.101.210.228) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 24 Mar 2021 15:41:37 -0400
Received: from NAM11-BN8-obe.outbound.protection.outlook.com (72.163.14.9) by xfe-rcd-004.cisco.com (173.37.227.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.792.3 via Frontend Transport; Wed, 24 Mar 2021 14:41:36 -0500
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DH8d0+Nsoqs1mTVQMnIBTUsg7B7NWSRHkX1ahpWoEejZb2gxiXQyORvUCy/VcMbMiN9m/sSpt51x0bS8blvmO/j+DTJVCcFF2oTPCJBL4UnTWQ5zaymVO4rK+M0LdhOt9x7g6HLKHX7EhSc5L4lhLJ4O3LTQXs4UJ1EuvgFsx5H2/O439qcLRXEmzGFCLI/T8WmnczUZZlC/LiLfzuo5PcE8swefiEjTodPLzd5Doo4AEOmm2WOjrxpa+3pd2yK6LF8qEncwzvjDSMengBYBZGYrfcEHHeAe+68aAKDCmD7z4bAcqQqI06vUK8aNoTH/Z+y0gzpdms9mVnUCkexSnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pt3NgqacZUphFxBIzh7441Mo/mtpTiDN4as5gxQOL+k=; b=Xh1jZogYbNpdkC7JbzMmhqdf/K07Qrr7pAdHUc+TEIdu7y98ISPTn08wr+AAoJHz1hqvzEKEJ7KO3jspxTdP1Z9UxhiSAc93EGWiWxG0ls0Fy02bxmKoLwkTzH8p84DwvFU1fEtaQNNpi+dP6ZyEazu91bJNU/nc5FeaNXGICNYXb8cMUCpxmYbRMi5BEFCVVggSHp+f0yEKqM11BkI+/st7sO2z6wMl5j54uf5cBIKUNKXmfb+nLxHwyI55yLJtMAxUORUrKIBttxYlD6AOKKId7OoCPcKoSHO29m7r6Tr+P6Onk15i6wC+PkOhk98ELie1W8J3up1buvbQpmZkPA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pt3NgqacZUphFxBIzh7441Mo/mtpTiDN4as5gxQOL+k=; b=jjlm6ytaJE+dX7cnRr+s6EVlmJRWT+UX624XHPft9FTAlxuh+BUuhukev2zFzC1di2LREJAZl5hONK1rDS79Bbxhdun9ZvEj5sgEp+GBISK1BHoqsx83SLqbNvScXPgBopE//QxUMdYslVurnzV5pWHOERKMZx46V/f3lNkkvGY=
Received: from BYAPR11MB3430.namprd11.prod.outlook.com (2603:10b6:a03:89::16) by BYAPR11MB3288.namprd11.prod.outlook.com (2603:10b6:a03:7e::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.23; Wed, 24 Mar 2021 19:41:35 +0000
Received: from BYAPR11MB3430.namprd11.prod.outlook.com ([fe80::fcbb:6f61:5361:973c]) by BYAPR11MB3430.namprd11.prod.outlook.com ([fe80::fcbb:6f61:5361:973c%7]) with mapi id 15.20.3955.027; Wed, 24 Mar 2021 19:41:35 +0000
From: "Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco)" <vrpolak@cisco.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Throughput definition discussions - volunteers needed
Thread-Index: AdcXXRVGD1VADmtoSFOA5isvSCuDVwCSGGGQAc0Q5VA=
Date: Wed, 24 Mar 2021 19:41:34 +0000
Message-ID: <BYAPR11MB343011C5780714D71E463E69BD639@BYAPR11MB3430.namprd11.prod.outlook.com>
References: <002801d7175e$36505d00$a2f11700$@netsecopen.org> <BY5PR11MB40382EB5C803BEBB1B95A258BD6C9@BY5PR11MB4038.namprd11.prod.outlook.com>
In-Reply-To: <BY5PR11MB40382EB5C803BEBB1B95A258BD6C9@BY5PR11MB4038.namprd11.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=cisco.com;
x-originating-ip: [195.91.4.16]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ea9575be-40fd-4f05-0687-08d8eefcd55d
x-ms-traffictypediagnostic: BYAPR11MB3288:
x-microsoft-antispam-prvs: <BYAPR11MB3288637F7FBACD8B97C4FB59BD639@BYAPR11MB3288.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 7RCuUQ+Ne1rbD72rbNni+roEP/JBElHzuQX0f9wlmGtoI+bfhpgoY2W5XXfAIeVS5cmpg0HY0Gvn30NSO0Fyr7IJKKhNRVsmuMuGXy3Uw8k9qJ7gtRC7lAld+zZWbASbT8rUncIeELGV0C2lG28Qg03OrQt9AikuQOixPrk3IQqMh8UGFOWwu828sNpBbl17VS8XSPAqC1YnTntbfQ7iukA4yMPmz6ZGrT/y1InZWlLdNJt8MnXOsJV7vpmEDQ90df+vU7ffMRrFYvOA7NgsIvX5IK8nEWJI9gUxABDFWmsp+gNfQCxFckMIfIR+x5SrWwhDODE9wpVIkHGvXAfN6/dkU1NXMKMwCl8TQpli0FDGZ6rJYXBULhCZcjOjPJVzhtrCtrKBUBE4aKLjfON9qrQKOTN2r61e9yzBjWeeHeEM7WbfCnFo/BH++HqKlWFtdkVW9ijhVG6+jeBAb/BAJx1TcZKgMfdKXASL7xWAvjlmRvCDvECpv5ppj3XM5eq2fWBAKeulpsGqqVc1xpspUdsHuXXIjX7i56toKHoUom/XAneWIYEIliisunPfnnquH0z/K5hGIXsdfJ3Ho+fEaAGsynViRwMwk8X9n9L45hX9NPxrXnff9gH4yoJg4miEynJ7CMOvSXtUxEABSPutup16Sr6TBangNDGq6KGLBCek7MewKXjGzciHdo7GkAIbq3+Bl+1FLjfrspjyDobyTcDIukfplreInGHdVLQkv+I=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3430.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(396003)(39860400002)(346002)(376002)(136003)(366004)(7696005)(478600001)(8936002)(5660300002)(52536014)(966005)(6506007)(86362001)(8676002)(26005)(99936003)(186003)(53546011)(66556008)(66616009)(166002)(9686003)(66476007)(55016002)(66446008)(83380400001)(66946007)(2906002)(38100700001)(316002)(33656002)(64756008)(76116006)(6916009)(71200400001); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: =?utf-8?B?bE1WY0VXcG1WbGFxcHlXTWNNVXRVWnY4NHRFNGpuN25wMmtzbG04T2VZbnYr?= =?utf-8?B?bmxUb0JBOFJ5ME1kTVVKUlltSVcxV2VNTElzMUtnUUpjVUJKemp4YW1GaUhw?= =?utf-8?B?bW9LdEpyOVZTTlluaEt1T2QrMnpHVUwwNUNGT29oOHJ6dmhoOThGTkMwZkJR?= =?utf-8?B?d3c5ckFtSkphcmlEUG01UTZuVElJV2VCSkU5ZHVucnZJbWY1TmVKTUJDUFgv?= =?utf-8?B?SVA5UVVWY0lSM01meEVoMlNDWXQ0djhIYkMvWnNjMGdsZngvVzBxL3NMQUJx?= =?utf-8?B?eERuY0tRS2VCV1ZjUHk2dGpDTjhSNDFWcGFROFE5ZVN2dkJNL09oNzNwOUV2?= =?utf-8?B?Qmt5eVFoWTNZQlJNNHc1Q2Facm1UdW12cUtiZjBPT2doMTZxTzV2TWRQaWQw?= =?utf-8?B?eEM0VHpYcnBTTTJSL1AzbG4wWDZ5N2ZwUUpjcnFPRDdGMGVRNkRzYnMwWWdp?= =?utf-8?B?TEhFQUxUTjgzWWVETlE4dklsS0lpbXYrK0x2UXVua2pIQnNyUTJZcjFOQ0U5?= =?utf-8?B?TG8rVERscjRXclpLalBjaG9IZGlCQWRXVktHM1phd256a3ZzWE0rcnc3K0FZ?= =?utf-8?B?UlpDYlRhcWlQZCt2VnV5VEZJNnBCQjd4enF5VlFXMkJYS0tNWjE4L211VENN?= =?utf-8?B?cllkaWdYalR6Q0NWU2JuYjBaWnBHOWdjMnIyWldNeUNoZmRWeDA1VW9yeFhk?= =?utf-8?B?Z2l3VFZVQzF5WHNHRnpFY1pZK1BBSmp1Q0lLQ2ozU0pMbHNDWUlnZVVvZkFS?= =?utf-8?B?cFNOc3E3L0ZiQ05wUTlRTWE5ejdTOWgwUkJRQWUzNGl5T2E3RzRUN0ZueG1G?= =?utf-8?B?M3YzckdtMEpUNFFDTjFFeFpHL2phaURUcDIwRVNaZEpXMzJQUVRLWERHdjFE?= =?utf-8?B?Rk81eTl5VEFUbFNBVlM1TmRvRGl2b0pKbjlBV2w4anM0bDJCM2lXZ2g5ZkZD?= =?utf-8?B?NUh1NlBwYmg2UUxNUlA1aVNnVWVmdUR3S2R4UmZvbm5NQkZ3UERLOVpBUmZG?= =?utf-8?B?cWlPbTczbUpBc0gzVEw4cDJpUUY3WEQzWUlTQTdDZWcrTGlvUUVKTVkzVXJl?= =?utf-8?B?YVc2R01RUFlkdjZ0RXlvVDdRS3R5U3JhY0N1czZldStmN2Q2eFlXTjduNnJQ?= =?utf-8?B?M09heWhsMkZIVFUwWTFsdjZ1RGVvWDN1M3NETi9jWDJXaks4ZTFLWVB0VlBs?= =?utf-8?B?TVZPNWhHZmZLSzBaWURzRk5xbkJRNUVna1dKaHY3ZE9TRXNtMWtWK1BUb2pr?= =?utf-8?B?bzFIVlRBME4rdXFxeEVzQmdIdHlvZ0pLK3RPc1lON0RtNitXOExVK0s0ejdY?= =?utf-8?B?U1Y5SW1LZGtOSmU2cmwvRGlZckpHVVl2VHJiZmF1alZWZmJmNURkK1RDc1dY?= =?utf-8?B?S3dJR3pzcm1EMEtLYWZwckhPUTRZTU41Y2VkcWhPRXZIS3BWdmVlSHlaSkRu?= =?utf-8?B?SXNFN3h0Sk0xQ29PSnFxYi9pUER2bWNialRGRlJiYXROT2ZuaTBLbS9KRE94?= =?utf-8?B?cjBaOHFRZHdjUlNadW94MCtTdWhtSlhSOEpPTVJWTzNTYy80VFB6c3E0NzZ6?= =?utf-8?B?S3hpbkdUeVdYbmFJSWd1OU1jdVJXTklseTN3ejN2SVpvNXhsTURiekcvVERE?= =?utf-8?B?dVhCWC9TN0dDcXZtWEJYdDVaOWJ4N0Q3UDBMdTdoWFBVUmdoeWV6VjBjVjdI?= =?utf-8?B?QTZSM09oNUdJT055aEVxbS9nazZHZmpkdTB3MDVYNHRDejNQVmpnNnVkWjg0?= =?utf-8?Q?4XHYMkZu+jPs8e4/lw=3D?=
x-ms-exchange-transport-forked: True
Content-Type: multipart/related; boundary="_004_BYAPR11MB343011C5780714D71E463E69BD639BYAPR11MB3430namp_"; type="multipart/alternative"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3430.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ea9575be-40fd-4f05-0687-08d8eefcd55d
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Mar 2021 19:41:34.9169 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: z/exC4CZBjdw/7eMmZi6yM98epJ1E9YlFYxJhGkcHqgM4AkO/dh6qbPIqJg8FnHm8ZwDcblMPqEehJEMW76rpA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3288
X-OriginatorOrg: cisco.com
X-Outbound-SMTP-Client: 173.36.7.16, xbe-aln-001.cisco.com
X-Outbound-Node: rcdn-core-7.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/JJGjCkwB3L7w0jUUoAJ9_RE1-oE>
Subject: Re: [bmwg] Throughput definition discussions - volunteers needed
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 24 Mar 2021 19:42:00 -0000

TL;DR:
1. Define "load" (the one quantity test equipment varies between trials within a search).
2. Define validity criterion for trial measurement result.
3. Confirm we are searching for max valid load (not for some medium load optimum).
4. Define which quantity of the best trial ends up in the spreadsheet.
(5. I care about definitions, I do not have strong opinions about names.)

The long version:

I had no comments during the meeting,
but here is my updated understanding.

You can generalize trial [3] in various ways
and get corresponding generalizations of throughput [4]
by selecting best trial that satisfies some condition.

We are interested in "continuous traffic" trials,
where trials have configurable duration and "load" as input values.
(Other trial type has configurable "size"
and duration is the result of measurement,
but we are not interested in that trial type yet.)
Of course, test equipment has more inputs than just duration and load,
but we group them under "traffic profile" and keep constant.

A direct result of a (continuous traffic) trial is a bunch of counters,
different vendors may use different counters (so we are not standardizing those).
Indirect results are numerous, but two are important.
Validity criterion, and "forwarding rate".

For the classic throughput, "load" is the Offered Load (in pps),
validity criterion is zero loss, and forwarding rate is again Offered Load.

The general approach for finding "throughput" is to vary the load,
and select the best "forwarding rate" from result that passed validation.
Usually, valid trial at increased load implies increased forwarding rate,
so it is enough to do binary search to find max load that still gives valid result.
Other algorithms (such as MLRsearch) can be used.
I will call this approach "the search".
(An example where increased load can decrease forwarding rate
is Maximum Forwarding Rate [5], as it has no validation criterion
related to packet loss.)

I think the generalization for NGFW would look like this.
Traffic profile defines what a "transaction" is.
Test equipment has "transaction rate" as its "load" parameter.
It initiates new transactions with a constant frequency,
regardless of status of previous transactions.
Transaction rate is given in TPS (transactions per second).
Test equipment has counters that can evaluate transaction success percentage (at least after the trial).
Validation criterion is that only 0.0001% of transactions can be other than success.
Test equipment also has some L2 counters.
Forwarding rate is computed from those counters, (probably in bits per second,
not sure about inter frame gap and other details).
The search is then finding the highest transaction rate that still leads to valid trial result.
The "throughut" is then the forwarding rate at highest valid transaction rate.
Even if slightly smaller loads happen to achieve higher forwarding rate
(which does not happen, L2 load usually accelerates more rapidly than transaction rate).

Even if all of the above is true, we still need to name two quantities.
One is the "highest load still leading to valid trial results".
(In PLRsearch draft I am calling that a "critical load",
but maybe we can come up with something less generic.)
Another one is the "previously reported as throughput",
an L2-centric quantity measured at the highest valid load.
Ideally, this name would refer to the first name in some way.
Examples: "Critical Throughput" or "Forwarding Rate at Maximal Usable Load (FRaMUL)".

Vratko.

[3] https://tools.ietf.org/html/rfc2544#section-23
[4] https://tools.ietf.org/html/rfc2544#section-26.1
[5] https://tools.ietf.org/html/rfc2285#section-3.6.3

From: bmwg <bmwg-bounces@ietf.org> On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco)
Sent: Monday, 2021-March-15 15:43
To: bmwg@ietf.org
Subject: Re: [bmwg] Throughput definition discussions - volunteers needed

> Let me know if you are interested

I am interested.

I think we already mentioned (during the meeting)
the need to distinguish allowed vs rejected traffic and packets vs bits.
RFC 2647 also mentions traffic orientation and goodput.
I think (for future proofing) we should think also about situations
when the amount of data going in differs from the amount of data coming out
(e.g encapsulation/decapsulation, encryption/decryption, compression/decompression).

> RFC 1242, 3.17 – Throughput

That is defined as a load with some additional properties.
This assumes constant load [0] in packets per second.
Stateful traffic often leads to non-constant loads,
affecting different layers differently.
I think we first need to update or replace intended load [2] (and offered load),
and base the "application throughput" on that.

Finally, I repeat my earlier comment: Counters examined during the traffic
cannot distinguish an in-flight "transaction" from a failed one.

Vratko.

[0] https://tools.ietf.org/html/rfc1242#section-3.4
[1] https://tools.ietf.org/html/rfc2285#section-3.5.1

From: bmwg <bmwg-bounces@ietf.org<mailto:bmwg-bounces@ietf.org>> On Behalf Of bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org>
Sent: Friday, 2021-March-12 17:39
To: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: [bmwg] Throughput definition discussions - volunteers needed

Folks,

The NGFW performance draft (https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-06) was reviewed in the virtual BMWG meeting yesterday. One item that proved to be a bit interesting  the discussion was around the definition of “Throughput” in section 6.3. We used what we thought was appropriate wording from RFC 2647, section 3.4 – Bit Forwarding Rate. The concern raised was specifically our use of the word Throughput. There was general consensus from the BMWG that this was confusing and for clarity sake we should use something else as the accepted BMWG definition of Throughput was defined well in RFC 1242, 3.17 – Throughput.  This discussion devolved into a circular one. It was suggested that some folks from NetSecOPEN and some folks from the BMWG get together on a call to come up with proposed wording that would be acceptable to everyone. I am looking for a few people from here to volunteer. Please let me know if you are interested.

Maciek Konstantynowicz and Al Morton have already volunteered.  The plan is to setup a call with the BMWG volunteers and a small number from NetSecOPEN to discuss. Our goal is to reach consensus on suitable wording that we can bring back to the group.

Let me know if you are interested and I will reach out to discuss timing.

Brian


---------
Brian Monkman
Executive Director, NetSecOPEN
Office: +1-717-610-0808
Fax: +1-717-506-0460
Mobile: +1-717-462-5422

[cid:image001.png@01D720E2.470533E0]
https://www.netsecopen.org