[bmwg] Comments on draft-mkonstan-nf-service-density-01

"MORTON, ALFRED C (AL)" <acm@research.att.com> Mon, 11 November 2019 18:48 UTC

Return-Path: <acm@research.att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 06B351201C6; Mon, 11 Nov 2019 10:48:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id xIqxb2p3L4Iy; Mon, 11 Nov 2019 10:47:59 -0800 (PST)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7C80E1200DF; Mon, 11 Nov 2019 10:47:56 -0800 (PST)
Received: from pps.filterd (m0049463.ppops.net []) by m0049463.ppops.net-00191d01. ( with SMTP id xABIbGYQ031943; Mon, 11 Nov 2019 13:47:51 -0500
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com []) by m0049463.ppops.net-00191d01. with ESMTP id 2w7cprs1jt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Nov 2019 13:47:51 -0500
Received: from enaf.dadc.sbc.com (localhost []) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id xABIloGE073090; Mon, 11 Nov 2019 12:47:51 -0600
Received: from zlp30496.vci.att.com (zlp30496.vci.att.com []) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id xABIlmDd073032 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 11 Nov 2019 12:47:48 -0600
Received: from zlp30496.vci.att.com (zlp30496.vci.att.com []) by zlp30496.vci.att.com (Service) with ESMTP id 790EB404B842; Mon, 11 Nov 2019 18:47:48 +0000 (GMT)
Received: from clpi183.sldc.sbc.com (unknown []) by zlp30496.vci.att.com (Service) with ESMTP id 56FAD404B841; Mon, 11 Nov 2019 18:47:48 +0000 (GMT)
Received: from sldc.sbc.com (localhost []) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id xABIlmkW014566; Mon, 11 Nov 2019 12:47:48 -0600
Received: from mail-azure.research.att.com (mail-azure.research.att.com []) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id xABIlg45014183; Mon, 11 Nov 2019 12:47:44 -0600
Received: from exchange.research.att.com (njbdcas1.research.att.com []) by mail-azure.research.att.com (Postfix) with ESMTP id 871F7E26FF; Mon, 11 Nov 2019 13:46:34 -0500 (EST)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njbdcas1.research.att.com ([fe80::8c6b:4b77:618f:9a01%11]) with mapi id 14.03.0468.000; Mon, 11 Nov 2019 13:47:25 -0500
From: "MORTON, ALFRED C (AL)" <acm@research.att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>
CC: "draft-mkonstan-nf-service-density@ietf.org" <draft-mkonstan-nf-service-density@ietf.org>
Thread-Topic: Comments on draft-mkonstan-nf-service-density-01
Thread-Index: AdWYvl706Qc01QNrSSCAvzSUalfudw==
Date: Mon, 11 Nov 2019 18:47:24 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CFA6EF4A7F@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
x-originating-ip: []
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-11-11_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1910280000 definitions=main-1911110164
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/0-AznrFlDPaOgbWl9z1Dh7SEwV4>
Subject: [bmwg] Comments on draft-mkonstan-nf-service-density-01
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Nov 2019 18:48:02 -0000

Hi Maciek and Peter,

My comments - promised at September ONS-EU - follow.

(as a Participant)



This is a well-written draft, in formative stages. I'd like to 
see other authors working in the network virtualization space
look it over - see where you agree and not, etc.
Our WG's products are better if they are consistent w.r.t.
terminology, and avoid overlap as much as possible
(through cross references).

bmwg co-chair


1.  Terminology
Section 1 should be a proper introduction.

I would like to see this draft align its many terms and definitions
with existing definitions.  ETSI NFV has defined most, if not all
in some form, NFV003 and TST009 are good places to start, but so are
RFC8174 and RFC8204. Obviously, we want to reference RFC2544 and RFC2889
where applicable, too.

Perhaps this section should become a Terminology Draft, expanded to cover 
and avoid overlap with other work in progress:

             Figure 1. NFV software technology stack.

   Proposed methodology is complementary to existing NFV benchmarking
   industry efforts focusing on vSwitch benchmarking [RFC8204], [TST009]
   and extends the benchmarking scope to NFV services.
Some of the TST009 Set-ups encourage testing of the "smaller"
services envisioned here, with scale of many serial and parallel

3.3.  Packet Path(s)

I think there is one variable possibly over-looked here,
that is the number of ports on the VF used to attach to 
the different data planes.
For example, it's very common to have containers/pods
with a single port/IP addrs on the Host data plane.
A practical example is a small Web server, only needs 
one port and IP addrs.
- This question is kind of a test of how abstract 
  the Network Service is, and how we answer is an 
  important part of the NS definition.

5.  Host Networking

   Host networking data-plane is the central shared resource that
   underpins creation of NFV services.  It handles all of the
   connectivity to external physical network devices through physical
   network connections using NICs, through which the benchmarking is


   o  Linux Kernel-Mode Networking.

   o  Linux User-Mode vSwitch.

   o  Virtual Machine vSwitch.

   o  Linux Container vSwitch.

   o  SRIOV NIC Virtual Function - note: restricted support for chain
      and pipeline topologies, as it requires hair-pinning through the
      NIC and oftentimes also through external physical switch.
   Analysing properties of each of these options and their Pros/Cons for
   specified NFV topologies and configurations is outside the scope of
   this document.

   From all listed options, performance optimised Linux user-mode
   vswitch deserves special attention.  Linux user-mode switch decouples
   NFV service from the underlying NIC hardware, offers rich multi-
   tenant functionality and most flexibility for supporting NFV
   services.  But in the same time it is consuming compute resources and
   is harder to benchmark in NFV service density scenarios.
Could you say more about why this is harder to benchmark?
I guess that DUT resources used and other unavoidable processes
have conflicts during measurements, are there more problems?
Also, I think that there may be potentially greater security
risks for some choices, meaning fewer or redesigned options
when it comes down to production deployment.

7.  Compute Resource Allocation
         +  PC - physical core, with SMT/HT enabled has many (mostly 2
            today) logical cores associated with it.
CPU Pinning to accomplish this, right?

         +  LC - logical core, if more than one lc get allocated in sets
            of two sibling logical cores running on the same physical
CPU Pinning and additional vCPU assignment to accomplish this, right?

   NFV Service Density - Core Usage View
   vSwitch-1c, NF-1c

   SVC   001   002   004   006   008   010
   001     2     3     6     9    12    15
Struggling to figure-out why the row below is not the answer.
It must have something to do with the size of the NUMA node or ?
   001     2     3     5     7     9    11
   002     3     6    12    18    24    30
   004     6    12    24    36    48    60

9.2.  Benchmarking MRR Throughput
Probably the sample results should be in an Appendix.
Put the graphs in slides of a BMWG presentation that
can be referenced (IETF URLs are fairly persistent).