IETF Logo Mail Archive

Re: Impact of hardware offloads on network stack performance

Boris Pismenny <borisp@mellanox.com> Thu, 10 May 2018 13:57 UTC

Return-Path: <borisp@mellanox.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D9D5C12EAD8 for <quic@ietfa.amsl.com>; Thu, 10 May 2018 06:57:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level:
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=mellanox.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yUsRf1q2EmgW for <quic@ietfa.amsl.com>; Thu, 10 May 2018 06:57:41 -0700 (PDT)
Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20068.outbound.protection.outlook.com [40.107.2.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1FBCC12EACF for <quic@ietf.org>; Thu, 10 May 2018 06:57:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=JYdlsWfUrq5B2VHc3+Qf5HCJoKliIXh8G+XnteDITc4=; b=NIizpDt74vZway4U3VyMqMCZxgwFLs9vvTVcu8JVLRBFrelokuxs/WCV4g1lCeSOmOnLQp4sEAohQTkC3KtFimmhzTbId6zE24DSVGL4d8m3ysiT3gQb2qUiHZVpzBGggJVhJcASxil3uhnBbTO/bg4mSdmzDcwK6B4+LqscWhI=
Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=borisp@mellanox.com;
Received: from [10.8.2.53] (193.47.165.251) by DB7PR05MB4348.eurprd05.prod.outlook.com (2603:10a6:5:1a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.735.17; Thu, 10 May 2018 13:57:35 +0000
Subject: Re: Impact of hardware offloads on network stack performance
To: Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>, "Eggert, Lars" <lars@netapp.com>
Cc: Watson Ladd <watsonbladd@gmail.com>, craigt <c@gryning.com>, "Lubashev, Igor" <ilubashe@akamai.com>, Roberto Peon <fenix@fb.com>, Mike Bishop <mbishop@evequefou.be>, Ian Swett <ianswett=40google.com@dmarc.ietf.org>, Praveen Balasubramanian <pravb@microsoft.com>, IETF QUIC WG <quic@ietf.org>, "Deval, Manasi" <manasi.deval@intel.com>
References: <CY4PR21MB0630CE4DE6BA4EDF54A1FEB1B6A40@CY4PR21MB0630.namprd21.prod.outlook.com> <1F436ED13A22A246A59CA374CBC543998B60C7AD@ORSMSX111.amr.corp.intel.com> <CAN1APdc3y0EwFqeYVvZs7MtBHhS_9CzwGmcwRqi_6GHWzF3_2Q@mail.gmail.com> <1F436ED13A22A246A59CA374CBC543998B60C851@ORSMSX111.amr.corp.intel.com> <CAN1APdcPkO-HfXqqvjeee6K8U8KSZQdtx=6fW1vZo+H6pKfzzQ@mail.gmail.com> <CAKcm_gP8TRPT7yi1y=mU5Fvq7xB5-1ieyKFLQMPomfabYbkcxA@mail.gmail.com> <bf540ec1f6f045aca1cf2379380630b5@usma1ex-dag1mb5.msg.corp.akamai.com> <SN1PR08MB185446750FCDA2B58C9833C8DA890@SN1PR08MB1854.namprd08.prod.outlook.com> <CACsn0cmJcAxORo4Cd-qyGJL3ZOT5Yz0WgdMqv_AGi4DjedyzFA@mail.gmail.com> <A6F55755-D7E1-40E4-8EC3-D5FE77A2EDBE@fb.com> <CAK-1kemWAW+2Sk5aKJ+RXKUAyjFii4MRH=xK7SE0mr=tbCiRfw@mail.gmail.com> <CAN1APddgX6=CxKdUA718=95EeCUPS60vVxbGFDHa7-eRjw7BxA@mail.gmail.com> <98A67023-AE1D-48EE-BD75-7D03E05E980D@netapp.com> <CAN1APdf2VKXB-sO97S=n5rCC4t6YZL22T=xdxPba9SgRAPKwLA@mail.gmail.com>
From: Boris Pismenny <borisp@mellanox.com>
Message-ID: <5213b2f5-c795-9d54-72b8-01a4a6db03ee@mellanox.com>
Date: Thu, 10 May 2018 16:57:29 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0
MIME-Version: 1.0
In-Reply-To: <CAN1APdf2VKXB-sO97S=n5rCC4t6YZL22T=xdxPba9SgRAPKwLA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-Originating-IP: [193.47.165.251]
X-ClientProxiedBy: CWLP265CA0121.GBRP265.PROD.OUTLOOK.COM (2603:10a6:401:53::13) To DB7PR05MB4348.eurprd05.prod.outlook.com (2603:10a6:5:1a::21)
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-HT: Tenant
X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(48565401081)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DB7PR05MB4348;
X-Microsoft-Exchange-Diagnostics: 1; DB7PR05MB4348; 3:dkpbSMb+195o1nRG9DSP+kNAmERf5AYaMFeddXLAOoiAKQa3kalpiH26oFLRGywGKEpVSo9hZC+410U9lwfXLYqaahPAxd7BK4z0E5RVr7gRdrB8fIfy8nfoOWHgZy/9Wmjrg5otudcClYk3ytIyc45RhoMA6i372LZyBH7TxQIQXuln0BXbhoBjEfFhQEeD65EKOqOk2JTd7gVfk5g9VrykObZ4iMeq1leyzced8HdvK2VFdyi3ME/fjAvpBUXm; 25:5CvpGH2iL6lhXa0nLVgz3vyZ85httk32FfuVfDRLPJZUjggYRFlPzdKmE26r8hO21ow1bDSPwNNPrvjeLVRlFWs1UgKrUWcaJM5o1YPZ6epMlqJ2jbxIj0D5bhufFJkv1V4kqDnq5lRZ5M9BSmbb4kzicWsX/S9mVwiQAsKpBX71ZrljVanVZeG1RRN/yziqpHHe1+GdC0Z3f1OnseSq2r0UtSa61lM+kgYXjW4RAwaKkXio/b/n1fWJnZhpOZZO96YZ1uQ9fnVW+b7SfvFp5mbe6LF3pVW5uvWX9WxAPTn34x/TwA+r9qWUz7hWvLhgao6oGPe4aQ66qZpL+MvQZw==; 31:1gsIsLg1VooaNJvcmwNZfB33ojmz26dZGnwuaOqQRoy20PA6hluBlJQoKWrZs2BmQhkepRDaJvdrnJ4DiK8N/ZO2x0lT17+EgEIs8RQSvMmImNO7V+6nrn5heCiQSgFanVPakrho6BjtcaWR33dDMuT0b9fsOJsCYSQmQniJnf2TD947KkfbR6JcqtgHBczPqpyNQACpITbKddcvE/4tv2m1j0p+O8LIcutvxfo1Nio=
X-MS-TrafficTypeDiagnostic: DB7PR05MB4348:
X-Microsoft-Exchange-Diagnostics: 1; DB7PR05MB4348; 20:6rpH671RoCvJ7rXxvvHenRCllHOkw+bcYpgfjcIX1kpDHX3tvw3LIuyscB47Yn7AGRBKdWRLH5N8Ov3rFB1C9VeEuPcw7LUj9/Bf+RuJuP5pdG4t0YGQLBS69yALMDay5bq8AfFb27DqwRFyCcLK3DjaLRcUo4RUrXWb1LPlG56ikuKBGqwS5fznfSQ/KQgWp1wRG9t8A1dYjkY7t44DxiYZwGBAnoL9WffkHUmx3p68mNWACuudF2HvUVCcnIc+7agqTgIp08MoKMkhF2pkZwVULCex8lgKLHdbfmeQzBn0J7c6dkPeGQ0oqrBxP7CNjliEbM7fuMO4uh6Z/gW0RxUw0zfT0njlARbw5aw+fbAyZSFLEIPlrGb+8VDvudqIX/gZKx/NcV8byPR15bDgcZX8jdJMj+MmLhQErjJbYor23oum+jt2Dx7HuHMTcwB3OoF4z6Sz1faVvC8mwuG+5V1BQ/K19e//P9iZM/1kwNWTtPKe+PZdwp5YVsw2Yrq+; 4:cW3CCS8xdS8aMYA/SPz7VHmrsekmgzj6hsbc2Ut0mX/vKAze0l6YGr/U7gLD/zcKv5fJOzCEYerNxV42+slXRhjnyZ0gmUER6rDmd0/aFlrRoxT3/+EReZYiFQ64fE565qEeJ9pXmnSUI5r7G/rl067Z/edk8CezG/UHZQ1iL+ldgoSOZAIZNNK2GYkbn3LugoBPJYWL54OiIqiKrxjMrrfxhM1xfTVnLGkghS7OW0WI7szOEDTfUiTX+X5yorSOpPS85IIzu139MHYnV7NLMyT5/1zxPRTnW50WPDF+jqEnNZ4M3bdkuXx46rLWG940tXgr+1hs8OoOKOM9HlfXk9L+qBjJT3AX/BuJdK6oq3c=
X-Microsoft-Antispam-PRVS: <DB7PR05MB4348B04C764A50DD12D30AB3B0980@DB7PR05MB4348.eurprd05.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(178636050973902)(132712982866762);
X-MS-Exchange-SenderADCheck: 1
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3231254)(944501410)(52105095)(93006095)(93001095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(20161123562045)(20161123558120)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:DB7PR05MB4348; BCL:0; PCL:0; RULEID:; SRVR:DB7PR05MB4348;
X-Forefront-PRVS: 066898046A
X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6049001)(39380400002)(346002)(396003)(376002)(39860400002)(366004)(189003)(199004)(4326008)(3846002)(110136005)(97736004)(5660300001)(39060400002)(81156014)(81166006)(8666007)(54906003)(486006)(53936002)(316002)(186003)(7416002)(8936002)(16576012)(31686004)(8676002)(58126008)(305945005)(2906002)(2870700001)(16526019)(65826007)(6246003)(77096007)(26005)(93886005)(6116002)(52116002)(66066001)(65956001)(966005)(386003)(53546011)(105586002)(956004)(25786009)(47776003)(36756003)(86362001)(229853002)(106356001)(6486002)(2486003)(476003)(3260700006)(76176011)(50466002)(65806001)(52146003)(478600001)(446003)(68736007)(64126003)(2616005)(7736002)(67846002)(11346002)(23676004)(31696002)(6666003)(6306002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR05MB4348; H:[10.8.2.53]; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts)
X-Microsoft-Exchange-Diagnostics: 1;DB7PR05MB4348;23:dcbSyFxMTWSoV5s89ZUZtZ3xEIW87ZYoGW/h9lpRnlD2RkPde3qq5dtoSKKD+19eBlGVlb+q2puMfARXefmvQpWMGHHJ6yxMpi5BnRJ9TxHwKWzmI3ITEKbU5tIh03vuNiV+E+w4n/96Y96AiMBzPUFIaLs9mfi+9dxTId+DHFax/lvW86TCTkLzy8wCc9PPWpmVsZ3jMxHoX+CQo2UsrVggKapc33CuLia7SKj1CULKIdU4GXGerDLbkWe/LhIeHSz4Z16sn3V1LUiYdqeC4WMoCWGOVdDwQQ4Fq9vPqkZBiz+kh1xI/W8MQLnvTTQ5ihSJFDDWiVA4FeYYaGHt4HiXnhJdymttiv1iEp1AVw8xG8st7z9DgfenaNsycu7zo+GSwoI7cWx0jUKP8NHOThngppNAhvbDmsVBIdetX76Db4AEiTVofeOqPolCPaAcn/SbfYc/hz5PWXKjGzk5D1UcrXagh52sF9zMM9IzF4X8lZIyE9es6i6pbpgldlqZkzjLsg33DqgPo3iimTk29pXQNpv2iz8PR+nmJgd3j1yThXRVz4Xz/gGjy7Ypwb6QSPfM/DuG0Cd2neKYQyNb3S7wXPdKa+jOLZn0/exs8TX9EpcgYMeOCDC4EI3FXY3nROaxG+qCZCn+jmIM3D3fqtaWOFpKW81atIy6ZM3PSf3EvBhwvaw7JwJHD40lQDWXg7udTtqY0a1qJOIOGJeLcNCnWl1MFDo1AzQpaqvEW5v3RocjuI75McqXpCH7UhGBuXR113o++xRdCCJqRmMKNqcGcS/NLm466Ci0ZhgvStTJIkWfzIpNPt3bbgN5qM0wTPLsBW+FL25+rCdQL1lj4OZBjEltnJBdXGV7ezDktG1f3CTdsU39mgUvfZQEwq8372HddGaNFLWKoQc0gEXHe0UUkFfdTKWNOc+yadB5J69raufeIw/9CJuOof/crhmA7Y3OwLfGx5KZVTAGIpzXiuGH8ZOpLDOys7DJP0LZT7wznc8Ellkm6bcooBh3K7ySr9K7/hTuYOQEAlpC3rb3ZSU/qSWpDVdV9DIC9lFLN0803qoc/oAXtX6oKOvbh0oBqsT63yV1mMLwDJV9RTvW1CxdX1T7E+j0JnS9qiWrOMHuKqc0Vda30LlP+aZ1SrrmXmg7VBEUlxthp4qf46Fi9AVqa7eG84zY5hLtn5tl40GoE6o28xYEWO8+dR0H0omQ1zZJcCWCF/THLrlwMKDZSczah68jTV7rutyD2z9+ypKYCZOJI/EdoPsXXClEKYS48JU3ZNT2e476YQRNIINEytuYCqfoMqSc0NwxlL6AjbQEBZB+hZuf/fPFDJUZ88EG4hsRnobgX1oZb86QFJu+vIi92cP/q3Hx304pU2xklSJVSj0hZjQUrsAE1PjfUYF1QTNnIx0tdfsq/z+RyooDClnx7enjU+JzjgkFb5+aRv0dzmM0tSiL5EMfvOK8mp2Cld919G+THIFTRJGYr3htwAK8/TZ8cm/9rxv+PdftOgrRzi+YDvT/htS29TVjVqnPWD0bhQHpjEMBa6+1CwQ1XYFwHqpz4kXq5t9IujZV4wzzgFE9G5dNsWRndr1JVic66Q0hREsoKY7K7jzZ9zv09A==
X-Microsoft-Antispam-Message-Info: rLuzP7+BpAefwTyXCMa9gEPh7/QCDMIuYBHZYvggmS7dDJCVIGPRUAP5yENNAUyb1d1xBjUZYR4ma0NIOFbRUttqriPqRipHiKP+/s3Q2zG2M3gdSk18CuBuCN9qf0DkqGP0jEWyFQn5+vo6+qzcY8nLU01XfQ2Natur4RSGYh/FOSPE70AJVDmKZF8DgkVW
X-Microsoft-Exchange-Diagnostics: 1; DB7PR05MB4348; 6:4S7RedddlxFhutRWP08YX4l/bMgc9lmZDJ/OsiimIDvEtA9+EQcO84/lgX0MAgfsuJl2zCcA7axOWEAwfsnShVrcc3MACmrkDyA6RPZjZZy5NYQ5KrWEqtjeZdAQ/aXJctskusHXgh41oQOwi8VE9R+o7y+o3jkiQ0LJBPNtEeLVra8tFV8yE2nF+SO4eWmu2qT0ZsWdh6dS4EQPTnMLJR2q1lI1LRJj/qlBisvEwUW4jR7Zxe299po5LE/4zCCwBjI8Ve4nGqUUyZMKtsX01uZoqHWyktGWjf/HKygOXzEk7dYsYmKgTAuEpTFThjzK3sISw3Vq9c58wf1G4+cdN1gE/+nW+LBGrBdmWwDPd5xMyQ54E5gTdcTK3f1CWOIwbGaQvsUQZfQ5fddKpMu2MtLa9rjVNZJNddn19DNo6yDsaHVohHJLHL1gZxkuP3EszB3Jyj8uhN0nxgiprSjw+Q==; 5:j5vxmgmmo7I3aFudSSSiIMpLjfTrxafLOBBx9yNptHYuRJi33+jPWzexHY3OnZAySbYj8dUQSXQSlnznUvFEDQCbz8CvEVcjW8XaeMZWhykINm4/LeSGYQ9R0hESrowmyQhq3xztNW10b7eDlPXD0DXDzNAXktLmQOP7VqyfXTU=; 24:OceW/NuTcCikkUFnmhahmrmuTcwlGA94ls99F9Kfh+uBZXlIF7B8WQjpGf6amFqdQgvTJiJxivcSlYNPRtbyRAnQE1X3N2wGVdRz+rXaJM0=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1; DB7PR05MB4348; 7:Vea80yecQ6tmgKZkKBQUrhyM+FNJAL7VfWKpV6yhyC31yvdu4YMnXBcLc42M4M5DWuMX1hkOljn08SP5S3TH8Ajb6Q15FsvlQkIanO/D7KObIK5TV1Sesl6q4EfcxOC1DjbDOTd+b6NWxOXPdCVjphrGbETvdsIyVjNLQd3UhgNNPE7du+G6XHVcwyGb1LRkAApzfvu1yZVYqD/gDzPBCbW5UAm4vhVo7MwWMlKt1KPQ23iQj5kFa3+wqKyo7uNT
X-MS-Office365-Filtering-Correlation-Id: 79c306c8-7eda-4c6a-d412-08d5b67dfc38
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2018 13:57:35.1074 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 79c306c8-7eda-4c6a-d412-08d5b67dfc38
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR05MB4348
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/6RGieKIWDReYnKQcJie_B4eTpsg>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 10 May 2018 13:57:44 -0000

On 5/10/2018 4:46 PM, Mikkel Fahnøe Jørgensen wrote:
> On 10 May 2018 at 15..33.58, Eggert, Lars (lars@netapp.com 
> <mailto:lars@netapp.com>) wrote:
>> You can bind netmap to individual NIC rings.
> 
> 
> Yes, but is is not exactly simple since deployment gets rather elaborate.
> 

It is possible to use DPDK without taking over the entire network card:
https://www.slideshare.net/LF_DPDK/lfdpdkmellanox-bifurcated-driver-model

Also, is there a reason to prefer netmap over other userspace network 
stacks like DPDK/VPP/etc?

v2.23.0 | Report a Bug | By Email