Re: [bmwg] Agenda for BMWG session at IETF-109

Vladimir Vassilev <vladimir@lightside-instruments.com> Wed, 11 November 2020 08:43 UTC

Return-Path: <vladimir@lightside-instruments.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5760D3A0EF0 for <bmwg@ietfa.amsl.com>; Wed, 11 Nov 2020 00:43:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.701
X-Spam-Level:
X-Spam-Status: No, score=-1.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, MSGID_FROM_MTA_HEADER=0.001, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=fail (1024-bit key) reason="fail (body has been altered)" header.d=netorgft4991094.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Cod9d8GoyZnW for <bmwg@ietfa.amsl.com>; Wed, 11 Nov 2020 00:43:37 -0800 (PST)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150049.outbound.protection.outlook.com [40.107.15.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 507B33A0FBD for <bmwg@ietf.org>; Wed, 11 Nov 2020 00:43:36 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZjwSov5W4MKUa6F1eqgcJTgMrbKFnS+KcKAPWEGXxXeaqYGLKKgvX6vqjsJwTSMqltbNG9uNFO/4KnTnK2EDn1rJEC8FSHLJxZ6UZSPzYPjz61MkQ/gDTx4li8Vfz6a862BREpVFGcGA0Tkr0aLOxCbIN9CT/1gTms1PQ1qVaJ2TopRGngKkp9ghQPZaYE8OXCE0ycoIN7+6N63kvJoWXQ9dox/Ik4pBlZIQcQSKQaw4F0pno4SD6oJO5/QguNn8DILR8q/7q+jYvmZWu6Au9aaPu4npoKcp7omTIwKgDddeu8iLXe2dFI9ZLThl1p0oGzuGRc40v7yVo/B27ZaSrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OISZkq5qD7kScGpr673NfhMx+zcI1q8sjcbP58NjUNw=; b=bqJkbberJIl+KB27MhalvUJEnM+A+CX+S5e627XHVleJ3TfWTcVhnffe4Xk4zI1xdUnfGn5yvtko32HGI343Zdx7+i3ZM8nMQc90DTcHJg+R9+zL2zxggVdh/cSw8VUfhZssmh5KYD6/TMO+lJzwH2gqrVsexYdT1KfT/GnXBw5Rp98Fd/VGg8qfN3Y6Hs7UX/FtcnvU2Hi6+Sju4aLaoJZALjUicixQdFBT//ZzuZ/6TgohzfyiqEsZiHYX/8KCaVstvxfRZVdKrXu3Xh+HwAotiAjioEAHVvmm4XuApa9SWz035mRF6E4PnIX2DONSrxjO8LxRwur/juS+hVJmcg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=lightside-instruments.com; dmarc=pass action=none header.from=lightside-instruments.com; dkim=pass header.d=lightside-instruments.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NETORGFT4991094.onmicrosoft.com; s=selector2-NETORGFT4991094-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OISZkq5qD7kScGpr673NfhMx+zcI1q8sjcbP58NjUNw=; b=t/r1bJxHW/W2kyB/bJ8S+fa5vOuGdbJGKTeM4fUoYQt6MF2YniDZoXiPM8EO6mOU59WTO5+mrdDqZwx9KBAH9D8J8LMHxjfzG+LvgIl9lfYeKzcfpgiSIJvPb5lmqJH02Y1lx6Lit7+W+SZV9uBeN0X2rA58lFqPIsU519/qvJk=
Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=lightside-instruments.com;
Received: from AM0PR08MB4084.eurprd08.prod.outlook.com (2603:10a6:208:129::25) by AM9PR08MB5954.eurprd08.prod.outlook.com (2603:10a6:20b:287::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.22; Wed, 11 Nov 2020 08:43:34 +0000
Received: from AM0PR08MB4084.eurprd08.prod.outlook.com ([fe80::218b:2fc5:90f6:537c]) by AM0PR08MB4084.eurprd08.prod.outlook.com ([fe80::218b:2fc5:90f6:537c%6]) with mapi id 15.20.3541.025; Wed, 11 Nov 2020 08:43:34 +0000
To: =?UTF-8?Q?Lencse_G=c3=a1bor?= <lencse@hit.bme.hu>
References: <4D7F4AD313D3FC43A053B309F97543CF0147645848@njmtexg5.research.att.com> <665bd46b-f8b6-6d4f-bcbe-92c958987b79@hit.bme.hu>
Cc: bmwg@ietf.org
From: Vladimir Vassilev <vladimir@lightside-instruments.com>
Message-ID: <dc02d621-89c8-a631-ac09-4833cbb5291e@lightside-instruments.com>
Date: Wed, 11 Nov 2020 09:43:32 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0
In-Reply-To: <665bd46b-f8b6-6d4f-bcbe-92c958987b79@hit.bme.hu>
Content-Type: text/plain; charset=iso-8859-2; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Originating-IP: [84.209.6.28]
X-ClientProxiedBy: OL1P279CA0034.NORP279.PROD.OUTLOOK.COM (2603:10a6:e10:13::21) To AM0PR08MB4084.eurprd08.prod.outlook.com (2603:10a6:208:129::25)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [192.168.0.27] (84.209.6.28) by OL1P279CA0034.NORP279.PROD.OUTLOOK.COM (2603:10a6:e10:13::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.21 via Frontend Transport; Wed, 11 Nov 2020 08:43:34 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e4275410-a7de-4242-8d07-08d8861de006
X-MS-TrafficTypeDiagnostic: AM9PR08MB5954:
X-Microsoft-Antispam-PRVS: <AM9PR08MB59542FB5BA9D4485F2CCD3919BE80@AM9PR08MB5954.eurprd08.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: MQH3ezowqacsFQkhFq1fdUD4yg7ZLlKMa03fWTfOccQ6xT+t+2sQ8d1zFgJlNSbgydfSpxXDlQxr0fewO0LomT4rCleEVx7bvJBWzIxf17q0WJ//lg6K0X75oZ8CffklfpewIS1Hjku39oUK7d8iHf0Csat9SqD+m8fIc2HXpaQ2OTrEhzGaKttPZM/5EAyFWGFpOxaIVHCTyZQtPIZJa992E6GE4NDgkNxpA2ygeAFcBIR/XLQQuBGRGZ7+x5YNqmRdZCwde8ULXiLR9seCXXjoKo/nqh+CgXf40X7+O5eEXRMFuNPnPy7ShqCQ2Tm0f3IMJZ1bzreQNmzTa2EujES3paNGAmviRW/tw/M8tSPfc0IrWzg9mfHKx9fQZh6Q4JI4H6TgIdbRgYuVYz39S2OpWlqfUOW81ySWuk0HUBSp0Fs7a/AI+U37mneS3iGdD6wOiIhyDnJFc0uxfZe07A==
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR08MB4084.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(39830400003)(136003)(366004)(346002)(376002)(396003)(66476007)(31686004)(26005)(16576012)(478600001)(66946007)(66556008)(6486002)(316002)(186003)(4326008)(8936002)(31696002)(36756003)(52116002)(16526019)(5660300002)(966005)(956004)(2906002)(8676002)(2616005)(86362001)(83380400001)(6916009)(43740500002); DIR:OUT; SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: OhRoQeyAOnhH6EjzAHMldvhD1MkeRlonUdpznh6EOQAC5eqp4JLcb7j1MJHjR3TXQN9I+G2gOt7RfystOQjVkFdY/URntyIK+g8EISpSSUnn+DyyKaUw4NtuJd2iRZHq9+2qblk67hwcUC5hVmONArREK+iGkS7A1AThE7G7nXqjG1AqMJGizKiwyvPUaNVLWSyi7DbjBI7s75f+isdO2onYrfAJ4ZsuWAXjspdlux94oKlk/c4hrKiw5DKABjS2Q3nCvuKQy9YGmhyb/eeNne5u4quhlVkJps9gJ6Oj3UBMNVwPHcJM80RVtfp/P5wNxUzDHLEsuXbypst9MuijvUrSfBYZmSs1+CR+yw6kRMC+Z7wVl8SiU5Sqqq0nKk0vAPjJm88DjESeaCItk9hxFpaTs06v3j4b60JdECVLaW1WSLEmZaHRJUkl4JSBDZl7DLlLPQRNanIELCmBQfoduSBUeGS8RowIj94AOxXFu6yLNBmiVtLgs5ant+0kVY0hdDtWaBN2UQj7tf9ZNrClUh1B4Q+tEQcCuGCgmxAqHRLl/D8xjOkXRhUnEIJQfx2xV1xn1kzOqnuW8JcFSNzwq75Y09YkI76huSgCXiS2P1gaNhE1Qiauyxh7DQR321qKZQmfwYZxfQAciw+KfKXlMwX3nbYxpdc6tRUNua8h5K1EFOLLdJwEIX6Y46Gt9juWzisa00yMycThWaeYV6eTEUrstCnyNQR/DP/3zbiBqU+fP/LhwmOLYf2HP0H+fJmZMl7nAyD58ScyiaHKjtrbqvcLrCFprBREwV7QVgWVdxIq7FOkU8KOpz2Y7faZazbFVwVxl4s7lLFs97kN+tD2pWVF5Gw2JjxNTP1BO2gTXsKQq/wN4j+HEFAHecvv/Iz6
X-OriginatorOrg: lightside-instruments.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e4275410-a7de-4242-8d07-08d8861de006
X-MS-Exchange-CrossTenant-AuthSource: AM0PR08MB4084.eurprd08.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Nov 2020 08:43:34.6292 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: c0326317-f373-4461-a96f-7946e0abb603
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HePY3KgPKU+9PSmnt9bSJ79dwMbL1tFZf1ha40ARBmS+yDYyO61EXlm/q7JD8tkZ1FH2T9UHgtMkZ246njdVGvYe22dwEcNYm3egmD3F+vwMGks+lXa1fYabMdxxKxnG
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB5954
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/vqujrvDcHNPQJFZ6DPIygiDI0CY>
Subject: Re: [bmwg] Agenda for BMWG session at IETF-109
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Nov 2020 08:43:46 -0000

Hi Gábor,

On 08/11/2020 19.21, Lencse Gábor wrote:
> However, I have results regarding siitperf, my RFC 8219 compliant SIIT 
> tester. (The usage of random port numbers is working well. I'm 
> currently working on calibrating it with a legacy RFC 2544 tester by 
> benhmarking the same DUT with the two devices and comparing their 
> results.) If there is interest in the WG for it, I would be happy to 
> present. But I do not want to push it.
>
> So, what do you think?


I looked into the implementation of siitperf. It is a well designed 
reference implementation for RFC 8219 and a useful DPDK API usecase and 
example in general.

Maybe it is a long shot but I was wondering if you are participating in 
the ongoing Hackathon? We are working on a relevant project that 
implements a general network interconnect tester based on a YANG model 
[1]. Instead of a single executable implementing a benchmark test (e.g. 
RFC2544, RFC8219 etc.) one can split the design into control plane 
(search logic, reconfiguration of the generator for different trials, 
reading status) and data plane (the timing critical frame input/output 
operations and timestamping). The control plane is managed with a python 
script over NETCONF [2] and the data plane workload is processed by 
software (native code in C) [3] or hardware logic (Verilog) [4] that is 
started/enabled by the NETCONF server for each interface when traffic 
generator configuration is committed.  The command line looks like this 
(based on the YANG model for a generator with single stream):

$ ./traffic-generator --interface=eth0 --frame-size=64 
--interframe-gap=20 --interburst-gap=124999852 \
    --frames-per-burst=2 --total-frames=4 
--realtime-epoch="2020-11-10T13:00:00.000000000Z" 
--interface-speed=1000000000
--frame-data="123456789ABCDEF01234567808004500002E000000000A112CBCC0000201C0000202C0200007001A0000000102030405060708090A0B0C0D0E0F10119CD50E0F"

We have an alternative traffic-generator-gmii tool [4] that instead of 
using software to generate the packets writes to the registers of a 
hardware core we have designed in Verilog. So replacing the command name 
results in deterministic generation for the systems that have the core 
available.

We were thinking of a third option based on DPDK 
(traffic-generator-dpdk) could be a compromise that is more 
deterministic then the default (Unix userspace) option and would not 
require programmable logic like the hardware dependent one.

We could augment the YANG model adding choice and an alternative to the 
static frame-data specifying the range and algorithm for generation of a 
pseudorandom port numbers in the specified range as a second reference 
implementation and do some testing of common DUTs and compare the 
results produced by the different implementations.

References:

[1] 
https://datatracker.ietf.org/doc/draft-vassilev-bmwg-network-interconnect-tester/

[2] 
https://github.com/vlvassilev/litenc/blob/master/tntapi/example/ietf-network-interconnect-tester/test-rfc2544-throughput.py 
(seems the script diverged from the initial  throughput sec. 26.1 target)

[3] 
https://github.com/vlvassilev/yuma123/blob/master/example-modules/ietf-traffic-generator/traffic-generator.c

[4] 
https://github.com/vlvassilev/network-interconnect-tester-cores/blob/master/lib/hw/lsi/cores/traffic_generator_gmii/hdl/traffic_generator_gmii.v


/Vladimir


>
> Best regards,
>
> Gábor