Re: [bmwg] draft-morton-bmwg-b2b-frame-05

"Maciek Konstantynowicz (mkonstan)" <mkonstan@cisco.com> Mon, 24 June 2019 11:32 UTC

Return-Path: <mkonstan@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 395F812003F for <bmwg@ietfa.amsl.com>; Mon, 24 Jun 2019 04:32:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.501
X-Spam-Level:
X-Spam-Status: No, score=-14.501 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com header.b=ma+bmGqd; dkim=pass (1024-bit key) header.d=cisco.onmicrosoft.com header.b=SNejOEIw
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ydn-o1DtV5uC for <bmwg@ietfa.amsl.com>; Mon, 24 Jun 2019 04:32:08 -0700 (PDT)
Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 34B52120136 for <bmwg@ietf.org>; Mon, 24 Jun 2019 04:32:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2939; q=dns/txt; s=iport; t=1561375928; x=1562585528; h=from:to:cc:subject:date:message-id:content-id: content-transfer-encoding:mime-version; bh=7Y1JqQdk6o6DDfdKl2SgcoCn9+DDApDuCpCbSxgUnp4=; b=ma+bmGqdquqRzoXlS1OWii9nHMVJESqMv3vqZG8dkP/eIlbYsqCZ60BA HGSAwHNAuMdFHP4I+4GVXZoEP7k5xd3dqgQKpYGJk6SO1dKzz+224S/oY sDB2V7tDqwMJCPX8nTVxNAJEJTVxjXyuKxzCyEk5hCTBB5wgHhOmhOSlB M=;
IronPort-PHdr: =?us-ascii?q?9a23=3AGBX8eh85dv9k6v9uRHGN82YQeigqvan1NQcJ65?= =?us-ascii?q?0hzqhDabmn44+8ZR7E/fs4iljPUM2b8P9Ch+fM+4HYEW0bqdfk0jgZdYBUER?= =?us-ascii?q?oMiMEYhQslVcuGDkvgMPfsRyc7B89FElRi+iLzPA=3D=3D?=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0BkBQDUsxBd/5FdJa1kHAEBAQQBAQc?= =?us-ascii?q?EAQGBZ4FEJCwDalUgBAsoCodTA45hmhOCUgNUCQEBAQwBARsSAgEBgUuCdYJ?= =?us-ascii?q?sIzgTAQMBAQQBAQIBBW2KNwyFSwIEEigGAQE3AREBCDZCJwQOJ4MAAYFqAx0?= =?us-ascii?q?BApgkAoE4iF+CIoJ5AQEFgkeCMxiCEQmBNIteF4FAP4ERJx+CTD6IAIImnBy?= =?us-ascii?q?NcQkCghSGTY0VG5dHpCkCBAIEBQIOAQEFgWchgVhwFWUBgkEJgjiDcIJkh24?= =?us-ascii?q?BcoEpjWYBgSABAQ?=
X-IronPort-AV: E=Sophos;i="5.63,412,1557187200"; d="scan'208";a="583985515"
Received: from rcdn-core-9.cisco.com ([173.37.93.145]) by rcdn-iport-2.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 24 Jun 2019 11:32:07 +0000
Received: from XCH-RCD-008.cisco.com (xch-rcd-008.cisco.com [173.37.102.18]) by rcdn-core-9.cisco.com (8.15.2/8.15.2) with ESMTPS id x5OBW6JJ007831 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Mon, 24 Jun 2019 11:32:06 GMT
Received: from xhs-rcd-001.cisco.com (173.37.227.246) by XCH-RCD-008.cisco.com (173.37.102.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 24 Jun 2019 06:32:06 -0500
Received: from xhs-rtp-002.cisco.com (64.101.210.229) by xhs-rcd-001.cisco.com (173.37.227.246) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 24 Jun 2019 06:32:05 -0500
Received: from NAM01-BN3-obe.outbound.protection.outlook.com (64.101.32.56) by xhs-rtp-002.cisco.com (64.101.210.229) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Mon, 24 Jun 2019 07:32:05 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=M1jtYyCdAoDqkpL1Zmg2/yGu9TqZgFEtjesCpEXEZJA=; b=SNejOEIwhQ99x9kRvZ/XCLU08Sw0QJke1yMHlFv2UQSpr4MKAz4l2U+sTsV2YpZ8QHSyygJlQDUM3zHgZ5HWOGYHWsQLCI/STdwjHpsIRxx2F79wnVSNdOX8TwvUiytcDw6gOUOdZsEVScYx/BBgm6lQ+E77DD76DAXjcLGN4Qo=
Received: from MN2PR11MB3856.namprd11.prod.outlook.com (20.178.251.203) by MN2PR11MB3805.namprd11.prod.outlook.com (20.178.254.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2008.16; Mon, 24 Jun 2019 11:32:05 +0000
Received: from MN2PR11MB3856.namprd11.prod.outlook.com ([fe80::d16a:3acd:c648:9f2c]) by MN2PR11MB3856.namprd11.prod.outlook.com ([fe80::d16a:3acd:c648:9f2c%7]) with mapi id 15.20.2008.014; Mon, 24 Jun 2019 11:32:05 +0000
From: "Maciek Konstantynowicz (mkonstan)" <mkonstan@cisco.com>
To: "Alfred C. Morton" <acmorton@att.com>
CC: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: draft-morton-bmwg-b2b-frame-05
Thread-Index: AQHVKoByNbM/rQa8vkyTnQVsC3xjaA==
Date: Mon, 24 Jun 2019 11:32:04 +0000
Message-ID: <81391894-FC37-4048-9371-453BCE6E4EF8@cisco.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=mkonstan@cisco.com;
x-originating-ip: [173.38.220.60]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2f0c4b2f-0710-4de6-2cbe-08d6f897956c
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:MN2PR11MB3805;
x-ms-traffictypediagnostic: MN2PR11MB3805:
x-microsoft-antispam-prvs: <MN2PR11MB3805E177F60F9154B9B27813CFE00@MN2PR11MB3805.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5236;
x-forefront-prvs: 007814487B
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(136003)(346002)(39860400002)(396003)(189003)(199004)(6506007)(6486002)(66946007)(229853002)(2906002)(99286004)(53936002)(7736002)(73956011)(71190400001)(71200400001)(4326008)(68736007)(86362001)(186003)(476003)(6916009)(81156014)(81166006)(102836004)(26005)(6436002)(486006)(8936002)(76116006)(8676002)(316002)(36756003)(91956017)(14444005)(6246003)(305945005)(256004)(2616005)(478600001)(66476007)(66066001)(66556008)(66446008)(33656002)(5660300002)(64756008)(25786009)(6512007)(6116002)(14454004)(3846002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR11MB3805; H:MN2PR11MB3856.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: cisco.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: iZMD3CELC7IcI7Ja6Kk2sASx0hbhJNcu6cDyNjwx7FOJTwGpAZ+MeTDPu/DPyOGdF8fQzMsi9BJwbOBCG+d5O0oPachkuBGBkqlh386MZBu8MJuWTitteYJ56m2cXOywEZvBu8o/zTcD9lrGFrF8yC1nxJfplbLEvQivDtvP9+/9FjyO+rf+2fN7xdaOQ1r1JFZ9Xa01pv0VgAlmtYvY1lTK+S6RlY6CiG0pi8Gc7hjI7Z/TwY7B2dkiLfltlB3jTYyyh0Gjoi45E0ya/WD4fO0GHLxBaabM7m5u3ybQqMKBmHNvWAgEACc7OH8WKzjyekAbDXJWBN45JjSUSFQoTz1TnCc+rhbuBBIZVJpfJo7pbUy1+v5Tj+R4okBQkCdXoYLyyTVhnBUvotjg+8sWwwWisWuejutxBgIFzPCJtrU=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <4FF0FFC854560A43A3462CFE2F249A09@namprd11.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 2f0c4b2f-0710-4de6-2cbe-08d6f897956c
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jun 2019 11:32:04.8799 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: mkonstan@cisco.com
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3805
X-OriginatorOrg: cisco.com
X-Outbound-SMTP-Client: 173.37.102.18, xch-rcd-008.cisco.com
X-Outbound-Node: rcdn-core-9.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/-o7OEEX3ConIEv1m2UfdvdOvKs4>
Subject: Re: [bmwg] draft-morton-bmwg-b2b-frame-05
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Jun 2019 11:32:10 -0000

Hi Al,

I finally had time to properly review this draft.

It is a very well written benchmarking specification that does address
an important area of measuring ingress packet buffer capacity used for
adapting packet arrival rate to not-always-steady packet processing rate
capability in any packet processing system.

It does take into consideration specifics of NFV systems (i.e.
vswitches, other VNFs) where this capability is of elevated importance
based on experience from OPNFV VSPERF benchmarking (backed up by a
number of measurement references).

The draft rightly notes that the size of this buffer matters as it
enables storing packets disruptions in the software packet processor
operation due to some external system interference. This a very
important aspect of NFV and as such I would suggest to make this aspect
more prominent in the draft, instead of burying it at the end of section
3, as it is a case now.

Two other general observations:

- The goal is measuring (ingress) buffer size in front of HeaderProc,
   per DUT definition in section 3.
   - This works if there is no other buffering in the system under test.
   - Suggest to add a paragraph dictating the setup where no egress
     queue build up is possible.

- Proposed methodology works for setups where the DUT's (composed here
   of "Buffer" and "HeaderProc" functions) behaviour can be measured
   with external tester:
   - This requires that any "noise" impacting DUT's behaviour is
     identified and isolated.
   - Potential sources of "noise":
     - In-path active components, other than DUT, noted in draft as
       "Ingress", "Egress".
     - Operating system environment interrupting DUT operation.
     - Shared resource(s) access collisions between DUT and some
       off-path component(s), impacting DUT's behaviour, a.k.a. "noisy
       neighbour" problem.
   - To deal with this e.g. for NFV DUT, the draft suggests to use
     enhanced Binary Search with Loss Verification as specified in
     [TST009], sec. 5.2. Plus repeating the test N times, sec. 5.3.
     - Agree this is the right way to isolate the DUT behaviour.
   - But I am puzzled when it comes to proposed calculation of
     "Corrected DUT Buffer Time", sec. 5.4.
     - There "Measured Throughput" is measured as per RFC2544, instead
       of referring to Binary Search with Loss Verification [TST009], or
       possibly using MLRsearch [draft-vpolak-mkonstan-bmwg-mlrsearch]
       to find NDR (non drop rate) and PDR (partial drop rate) and use
       those to calculate the actual DUT buffer time.
     - Another potential candidate for the "Measured Throughput" is the
       maximum measured throughput regardless of loss, metric popular
       with academics dealing with software network processing, also
       defined in FD.io CSIT as Maximum Receive Rate.

Hope this makes sense.

Cheers,
-Maciek