IETF Logo Mail Archive

Re: Impact of hardware offloads on network stack performance

Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com> Thu, 10 May 2018 14:57 UTC

Return-Path: <mikkelfj@gmail.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 43E071242EA for <quic@ietfa.amsl.com>; Thu, 10 May 2018 07:57:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.697
X-Spam-Level:
X-Spam-Status: No, score=-2.697 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FyRXkSNHwdc4 for <quic@ietfa.amsl.com>; Thu, 10 May 2018 07:57:36 -0700 (PDT)
Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8EBF0124205 for <quic@ietf.org>; Thu, 10 May 2018 07:57:36 -0700 (PDT)
Received: by mail-io0-x235.google.com with SMTP id p124-v6so3358462iod.1 for <quic@ietf.org>; Thu, 10 May 2018 07:57:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=vjxJvLD5pzK9fH/nL5TjmW0aPUIpSJiA5bOiuEXWNGA=; b=p+AjfFwrn58gruRzyUtdUQPGr3syK4jNfcNHF7OjflKG0uPhELkI2ypQHbmQUpeZKQ QPqVfs/nksthug2V2DGkTAiMkz2enANSMIQHsysLXZHbzY+j4G/iFIUnXN087aS6XizJ EYTWajB38wDQi2ySLIv5TuPjpOE/c6/fE5DtcScmliFwTHdHiDtBazEHcRDODGThHhqr om/KIvWhhFo2D+nUIlD5l+q+Cnp4yaw9RxHVH58m/ozHreoZnNNWicivbggHvsT+XvrZ WaoPRHU80eaBDhOznIFzHpr2Vdq7LuLen45fRlCwkZeC0iiTxKVnZCa8CiSW5rqZaJXw eTQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=vjxJvLD5pzK9fH/nL5TjmW0aPUIpSJiA5bOiuEXWNGA=; b=LiOH15nhIfYxlyCkIYSOyZ2XIg4IxnMnE3mXqn8Oh3Cy+wklmsnWRXsLXX3jD/aCgt alzF1/aZNApNa4swr5NAvp1MUv4C9rcQihTbgTRYIylLiAKSodocj6UVqtYqjpQQSBXS ELznYLg8P/DtTQybiIHcc9HdW6Z/eEk3nsmcOFMB08/x9AB4C1mMtiS86EbJVS92krAX o6noem/8gtKOpho8g8TY6+kc1HkFhGWE+40PQLFiaEIDYCeiHHNDIeBNnGej/jKVapA6 shDdFLK5xkJqy3+5tSmDb/+bammgaCMKC/wEfxvrcZzcb09jnlz0/TKXbpe2l8Oaj034 qApA==
X-Gm-Message-State: ALKqPwfvwswcXQG31Ytjg+kEOlEFV+Lq+7tRN67sMMtiFLjhHd3HiACn p7zSb/WJoBB2I37E+Aia7E+xxnbml+INPI2+Oqc=
X-Google-Smtp-Source: AB8JxZqHdyBef3FR8/YgR3/5wHnrjfbI3dLFWaCgIwbUiuSWcisWj1LoML0gxlooXqzxWT4M/fI/2Vg1rnGQlQFhib0=
X-Received: by 2002:a6b:b60a:: with SMTP id g10-v6mr1765560iof.209.1525964256041; Thu, 10 May 2018 07:57:36 -0700 (PDT)
Received: from 1058052472880 named unknown by gmailapi.google.com with HTTPREST; Thu, 10 May 2018 07:57:35 -0700
From: Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>
In-Reply-To: <9E549481-9B80-4DED-AC37-DC0C91AEBCDE@netapp.com>
References: <CY4PR21MB0630CE4DE6BA4EDF54A1FEB1B6A40@CY4PR21MB0630.namprd21.prod.outlook.com> <1F436ED13A22A246A59CA374CBC543998B60C7AD@ORSMSX111.amr.corp.intel.com> <CAN1APdc3y0EwFqeYVvZs7MtBHhS_9CzwGmcwRqi_6GHWzF3_2Q@mail.gmail.com> <1F436ED13A22A246A59CA374CBC543998B60C851@ORSMSX111.amr.corp.intel.com> <CAN1APdcPkO-HfXqqvjeee6K8U8KSZQdtx=6fW1vZo+H6pKfzzQ@mail.gmail.com> <CAKcm_gP8TRPT7yi1y=mU5Fvq7xB5-1ieyKFLQMPomfabYbkcxA@mail.gmail.com> <bf540ec1f6f045aca1cf2379380630b5@usma1ex-dag1mb5.msg.corp.akamai.com> <SN1PR08MB185446750FCDA2B58C9833C8DA890@SN1PR08MB1854.namprd08.prod.outlook.com> <CACsn0cmJcAxORo4Cd-qyGJL3ZOT5Yz0WgdMqv_AGi4DjedyzFA@mail.gmail.com> <A6F55755-D7E1-40E4-8EC3-D5FE77A2EDBE@fb.com> <CAK-1kemWAW+2Sk5aKJ+RXKUAyjFii4MRH=xK7SE0mr=tbCiRfw@mail.gmail.com> <CAN1APddgX6=CxKdUA718=95EeCUPS60vVxbGFDHa7-eRjw7BxA@mail.gmail.com> <98A67023-AE1D-48EE-BD75-7D03E05E980D@netapp.com> <CAN1APdf2VKXB-sO97S=n5rCC4t6YZL22T=xdxPba9SgRAPKwLA@mail.gmail.com> <5213b2f5-c795-9d54-72b8-01a4a6db03ee@mellanox.com> <9E549481-9B80-4DED-AC37-DC0C91AEBCDE@netapp.com>
X-Mailer: Airmail (420)
MIME-Version: 1.0
Date: Thu, 10 May 2018 07:57:35 -0700
Message-ID: <CAN1APdcZfs5UwNsyeqb=C3oodC6GzcoPM2EOY8nK1ofV+yqevA@mail.gmail.com>
Subject: Re: Impact of hardware offloads on network stack performance
To: Boris Pismenny <borisp@mellanox.com>, "Eggert, Lars" <lars@netapp.com>
Cc: Roberto Peon <fenix@fb.com>, Mike Bishop <mbishop@evequefou.be>, Ian Swett <ianswett=40google.com@dmarc.ietf.org>, IETF QUIC WG <quic@ietf.org>, craigt <c@gryning.com>, "Deval, Manasi" <manasi.deval@intel.com>, Watson Ladd <watsonbladd@gmail.com>, "Lubashev, Igor" <ilubashe@akamai.com>, Praveen Balasubramanian <pravb@microsoft.com>
Content-Type: multipart/alternative; boundary="0000000000006bde41056bdb3d9f"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/HeEOqM6jTRtJ_43oq5qpR_zKJ2g>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 10 May 2018 14:57:46 -0000

Netmap is has a fairly low line count and complexity for what it achieves,
it is not tied to any vendor specific hardware, it patches do seem to
arrive for various cards.

I recently did a user space emulation of netmap where separate user threads
drains and fills the tx and rx rings respectively. I’m not sure this is
worth it in terms of performance since it adds more synchronization (netmap
makes certain assumptions because some updates are in the kernel). However,
it does provide a uniform buffering layer and it makes it easier to plug in
proper netmap when available. It does move some per message latency out of
the way.


On 10 May 2018 at 16.03.28, Eggert, Lars (lars@netapp.com) wrote:

On 2018-5-10, at 14:57, Boris Pismenny <borisp@mellanox.com> wrote:
> Also, is there a reason to prefer netmap over other userspace network
stacks like DPDK/VPP/etc?

For me personally, the netmap API is a lot more minimal and therefore
easier to integrate. DPDK really wants to take over scheduling, memory
management and all kinds of other stuff besides packet I/O. YMMV.

Lars

v2.23.0 | Report a Bug | By Email