IETF Logo Mail Archive

Re: Impact of hardware offloads on network stack performance

Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com> Thu, 10 May 2018 15:02 UTC

Return-Path: <mikkelfj@gmail.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D087124B18 for <quic@ietfa.amsl.com>; Thu, 10 May 2018 08:02:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.697
X-Spam-Level:
X-Spam-Status: No, score=-2.697 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6YjgoTFt63Vo for <quic@ietfa.amsl.com>; Thu, 10 May 2018 08:02:22 -0700 (PDT)
Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6364A1243FE for <quic@ietf.org>; Thu, 10 May 2018 08:02:21 -0700 (PDT)
Received: by mail-io0-x22b.google.com with SMTP id e78-v6so3381733iod.0 for <quic@ietf.org>; Thu, 10 May 2018 08:02:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=PNiGaSL/aUhqMPzM3/HPm4hba1UOnYFk1Tc5dVAszec=; b=mGpcq41ouAshDl/oaFkhqSPar6yTvK8V6hTDl1HuqobQ6FagCGZfE4tXEDftPnqw7x Z9J03PGDstL2CdpJMqiBTRE9PbOLWwQzUcJ1neQPCuBpR4AVNv7eRX5Ah30wl9xSzRxw /OkGXxsBgaPgnP/+l4CUVGVaFY1bCLcUShs4v7RYSClXBSY7Ya/HA80KOBiD4aQ8tpMB faXVDcCghJetUwEWG6zy0FoQhVvzLon90/epesCYLSPhH4xxBYa51MytoiKfIn320XtY rjkpLrQMCz3f8z3nATPezj68mdGjZp7PcaTvLIzoKTsPtASRJ0GaILomkhH81o6qqK4T RYHw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=PNiGaSL/aUhqMPzM3/HPm4hba1UOnYFk1Tc5dVAszec=; b=s82dY8sZmH0LejfD+Td/kxN8ZFMKf8so3nPymEEeUHQg5+F9Llz5aFpN86Fuaq2L0K nf4uwQOoTGYxCmWItKKZjBEP9hUPPBBtO6EZl1MuwdfSidU58q8V8KvhNngiWE353JHn 92mnc/u0MdRv6eVud4HaHuM1mpMnxppTfClwoDYfiGNMqiEhRWKxeQw7sQwRPSQKnR2K nbQS96wnTf3v+tIxR/XDuwhVV+69d3gqrp0LIKmYKQrbBpnp6G+0iXDOrD5YDYpHdEll k12vAamsyibZnSIQ6T+IX0fEokSzwye24HIFff1pyPcyMlAc1w+5wSm7SEH3HAMTWado HFsw==
X-Gm-Message-State: ALKqPwfr9rHf//FsDcUY+198gjW/NRsEklEW3zsuvM8w9xiz07S+THSG 1Xypv7f70d0XcY7YPMNAeVaGA0dpJHnIePhO/jw=
X-Google-Smtp-Source: AB8JxZrFNCMeIgTCQbWwRAgVu5ABxkXxvuGt1yDYmqUNpZwpm6Vmse0Ux73/18E/I4HUXnGAd3CbInGIOdEmtKDfrUo=
X-Received: by 2002:a6b:5010:: with SMTP id e16-v6mr1819977iob.274.1525964540858; Thu, 10 May 2018 08:02:20 -0700 (PDT)
Received: from 1058052472880 named unknown by gmailapi.google.com with HTTPREST; Thu, 10 May 2018 08:02:20 -0700
From: Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>
In-Reply-To: <CAN1APdcZfs5UwNsyeqb=C3oodC6GzcoPM2EOY8nK1ofV+yqevA@mail.gmail.com>
References: <CY4PR21MB0630CE4DE6BA4EDF54A1FEB1B6A40@CY4PR21MB0630.namprd21.prod.outlook.com> <1F436ED13A22A246A59CA374CBC543998B60C7AD@ORSMSX111.amr.corp.intel.com> <CAN1APdc3y0EwFqeYVvZs7MtBHhS_9CzwGmcwRqi_6GHWzF3_2Q@mail.gmail.com> <1F436ED13A22A246A59CA374CBC543998B60C851@ORSMSX111.amr.corp.intel.com> <CAN1APdcPkO-HfXqqvjeee6K8U8KSZQdtx=6fW1vZo+H6pKfzzQ@mail.gmail.com> <CAKcm_gP8TRPT7yi1y=mU5Fvq7xB5-1ieyKFLQMPomfabYbkcxA@mail.gmail.com> <bf540ec1f6f045aca1cf2379380630b5@usma1ex-dag1mb5.msg.corp.akamai.com> <SN1PR08MB185446750FCDA2B58C9833C8DA890@SN1PR08MB1854.namprd08.prod.outlook.com> <CACsn0cmJcAxORo4Cd-qyGJL3ZOT5Yz0WgdMqv_AGi4DjedyzFA@mail.gmail.com> <A6F55755-D7E1-40E4-8EC3-D5FE77A2EDBE@fb.com> <CAK-1kemWAW+2Sk5aKJ+RXKUAyjFii4MRH=xK7SE0mr=tbCiRfw@mail.gmail.com> <CAN1APddgX6=CxKdUA718=95EeCUPS60vVxbGFDHa7-eRjw7BxA@mail.gmail.com> <98A67023-AE1D-48EE-BD75-7D03E05E980D@netapp.com> <CAN1APdf2VKXB-sO97S=n5rCC4t6YZL22T=xdxPba9SgRAPKwLA@mail.gmail.com> <5213b2f5-c795-9d54-72b8-01a4a6db03ee@mellanox.com> <9E549481-9B80-4DED-AC37-DC0C91AEBCDE@netapp.com> <CAN1APdcZfs5UwNsyeqb=C3oodC6GzcoPM2EOY8nK1ofV+yqevA@mail.gmail.com>
X-Mailer: Airmail (420)
MIME-Version: 1.0
Date: Thu, 10 May 2018 08:02:20 -0700
Message-ID: <CAN1APdejvqPWi-E14PiO6Pts7BpBv9BODMBjRoSDzwRD_hiruw@mail.gmail.com>
Subject: Re: Impact of hardware offloads on network stack performance
To: Boris Pismenny <borisp@mellanox.com>, "Eggert, Lars" <lars@netapp.com>
Cc: Roberto Peon <fenix@fb.com>, Mike Bishop <mbishop@evequefou.be>, IETF QUIC WG <quic@ietf.org>, "Lubashev, Igor" <ilubashe@akamai.com>, craigt <c@gryning.com>, Praveen Balasubramanian <pravb@microsoft.com>, Watson Ladd <watsonbladd@gmail.com>, Ian Swett <ianswett=40google.com@dmarc.ietf.org>, "Deval, Manasi" <manasi.deval@intel.com>
Content-Type: multipart/alternative; boundary="00000000000065d4ef056bdb4eff"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/-5-Df5NSNbmFA8Njk-Qptsr39nU>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 10 May 2018 15:02:24 -0000

Some of you probably have inside knowledge of this:

https://www.bbc.co.uk/rd/blog/2018-04-high-speed-networking-open-source-kernel-bypass


On 10 May 2018 at 16.57.35, Mikkel Fahnøe Jørgensen (mikkelfj@gmail.com)
wrote:

Netmap is has a fairly low line count and complexity for what it achieves,
it is not tied to any vendor specific hardware, it patches do seem to
arrive for various cards.

I recently did a user space emulation of netmap where separate user threads
drains and fills the tx and rx rings respectively. I’m not sure this is
worth it in terms of performance since it adds more synchronization (netmap
makes certain assumptions because some updates are in the kernel). However,
it does provide a uniform buffering layer and it makes it easier to plug in
proper netmap when available. It does move some per message latency out of
the way.


On 10 May 2018 at 16.03.28, Eggert, Lars (lars@netapp.com) wrote:

On 2018-5-10, at 14:57, Boris Pismenny <borisp@mellanox.com> wrote:
> Also, is there a reason to prefer netmap over other userspace network
stacks like DPDK/VPP/etc?

For me personally, the netmap API is a lot more minimal and therefore
easier to integrate. DPDK really wants to take over scheduling, memory
management and all kinds of other stuff besides packet I/O. YMMV.

Lars

v2.23.0 | Report a Bug | By Email