IETF Logo Mail Archive

RE: Impact of hardware offloads on network stack performance

Mike Bishop <mbishop@evequefou.be> Tue, 24 April 2018 16:28 UTC

Return-Path: <mbishop@evequefou.be>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E75E12D77B for <quic@ietfa.amsl.com>; Tue, 24 Apr 2018 09:28:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.909
X-Spam-Level:
X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=evequefou.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HbEvci27S0D5 for <quic@ietfa.amsl.com>; Tue, 24 Apr 2018 09:28:54 -0700 (PDT)
Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0709.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe48::709]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3E4F312702E for <quic@ietf.org>; Tue, 24 Apr 2018 09:28:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=evequefou.onmicrosoft.com; s=selector1-evequefou-be; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+8WjPrB63XbFidZ2zxZ2aH6RUZnkc2Bjb/qKaULwjI8=; b=hqWVJTh79dg3C522JdTsF3UXlz56X0mg+oICGMBTJEWJj/y7cTFEvwPuVSLv/te/zkFh+3JDSg/gxjbUb3pPL7nc/z4ZK4yFq1DCcIvs9ev744OyPXTHWvxdJlNQpa89SWeaedVxz3MkXcQS7wcooJAOUt0xMF7RkeLag90TWRM=
Received: from SN1PR08MB1854.namprd08.prod.outlook.com (10.169.39.8) by SN1PR08MB1744.namprd08.prod.outlook.com (10.162.133.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.15; Tue, 24 Apr 2018 16:28:50 +0000
Received: from SN1PR08MB1854.namprd08.prod.outlook.com ([fe80::dd26:af46:4549:f472]) by SN1PR08MB1854.namprd08.prod.outlook.com ([fe80::dd26:af46:4549:f472%13]) with mapi id 15.20.0696.019; Tue, 24 Apr 2018 16:28:50 +0000
From: Mike Bishop <mbishop@evequefou.be>
To: Watson Ladd <watsonbladd@gmail.com>
CC: "Lubashev, Igor" <ilubashe@akamai.com>, Ian Swett <ianswett=40google.com@dmarc.ietf.org>, Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>, Praveen Balasubramanian <pravb@microsoft.com>, IETF QUIC WG <quic@ietf.org>, "Deval, Manasi" <manasi.deval@intel.com>
Subject: RE: Impact of hardware offloads on network stack performance
Thread-Topic: Impact of hardware offloads on network stack performance
Thread-Index: AdPMUbL7T8nZAW2ySiO289IPTC0ttgAApm4AADSbFZAACUWYgAAFkLQAACA3b/AAAajkgANRfL2AAAA3SQAAAHMwgAAAR/kAAAEuygAAAJMTgAAAxh6AAAICTwAAKOV38A==
Date: Tue, 24 Apr 2018 16:28:50 +0000
Message-ID: <SN1PR08MB1854A941E8A703913ECBE821DA880@SN1PR08MB1854.namprd08.prod.outlook.com>
References: <CY4PR21MB0630CE4DE6BA4EDF54A1FEB1B6A40@CY4PR21MB0630.namprd21.prod.outlook.com> <CAN1APde93+S8CP-KcZmqPKCCvsGRiq6ECPUoh_Qk0j9hqs8h0Q@mail.gmail.com> <SN1PR08MB1854E64D7C370BF0A7456977DABB0@SN1PR08MB1854.namprd08.prod.outlook.com> <CAKcm_gNhsHxXM77FjRj-Wh4JxA21NAWKXX3KBT=eZJsCdacM7Q@mail.gmail.com> <DB6PR10MB1766789B25E31EBE70564BD3ACBA0@DB6PR10MB1766.EURPRD10.PROD.OUTLOOK.COM> <SN1PR08MB18548155722C665FFD45CC4DDABA0@SN1PR08MB1854.namprd08.prod.outlook.com> <CAKcm_gMYf6AbHgvg+WKv-woQCtzz9WztiEDf-iRUjKw63Qx=9Q@mail.gmail.com> <1F436ED13A22A246A59CA374CBC543998B60C7AD@ORSMSX111.amr.corp.intel.com> <CAN1APdc3y0EwFqeYVvZs7MtBHhS_9CzwGmcwRqi_6GHWzF3_2Q@mail.gmail.com> <1F436ED13A22A246A59CA374CBC543998B60C851@ORSMSX111.amr.corp.intel.com> <CAN1APdcPkO-HfXqqvjeee6K8U8KSZQdtx=6fW1vZo+H6pKfzzQ@mail.gmail.com> <CAKcm_gP8TRPT7yi1y=mU5Fvq7xB5-1ieyKFLQMPomfabYbkcxA@mail.gmail.com> <bf540ec1f6f045aca1cf2379380630b5@usma1ex-dag1mb5.msg.corp.akamai.com> <SN1PR08MB185446750FCDA2B58C9833C8DA890@SN1PR08MB1854.namprd08.prod.outlook.com> <CACsn0cmJcAxORo4Cd-qyGJL3ZOT5Yz0WgdMqv_AGi4DjedyzFA@mail.gmail.com>
In-Reply-To: <CACsn0cmJcAxORo4Cd-qyGJL3ZOT5Yz0WgdMqv_AGi4DjedyzFA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=mbishop@evequefou.be;
x-originating-ip: [38.134.241.6]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; SN1PR08MB1744; 7:vTGDtddGS2XjOxe3x2gqRoBA/jG5GeFWkQeJcAraNnyjMFLnc1CuO57k2/KbfUNhEPw7PKzZ5R/pZKoXlJe61yozsBG96t5q0qNetTCjGaycgZYCXXGAwD78GwGOctv64+Xz1O8gb5y1so8ZUg5eUGEXOaPqn3k4z3KFYpadkApkEevjdV+pPfVMnyk6HDYwemkPDabbUozCU4e2mCaX33nWJ/kzBhbRagtoiZlYNkR5kHKyOuWBLJRmAjX8SD02
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(7021125)(5600026)(4534165)(7022125)(4603075)(4627221)(201702281549075)(7048125)(7024125)(7027125)(7028125)(7023125)(2017052603328)(7153060)(7193020); SRVR:SN1PR08MB1744;
x-ms-traffictypediagnostic: SN1PR08MB1744:
x-microsoft-antispam-prvs: <SN1PR08MB1744FB6253A78237FCB3E9EBDA880@SN1PR08MB1744.namprd08.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(28532068793085)(89211679590171)(85827821059158)(21748063052155)(266576461109395)(228905959029699);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(3231232)(944501410)(52105095)(10201501046)(93006095)(93001095)(3002001)(6041310)(20161123562045)(20161123558120)(20161123564045)(2016111802025)(20161123560045)(6043046)(6072148)(201708071742011); SRVR:SN1PR08MB1744; BCL:0; PCL:0; RULEID:; SRVR:SN1PR08MB1744;
x-forefront-prvs: 0652EA5565
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(376002)(396003)(39830400003)(366004)(39380400002)(189003)(199004)(26005)(3660700001)(81156014)(86362001)(81166006)(8676002)(8936002)(6436002)(74316002)(7736002)(55016002)(105586002)(6246003)(8666007)(6306002)(9686003)(2906002)(6916009)(19609705001)(33656002)(5660300001)(316002)(229853002)(3280700002)(39060400002)(53936002)(97736004)(236005)(54896002)(54906003)(106356001)(25786009)(11346002)(476003)(93886005)(102836004)(53546011)(2900100001)(4326008)(45080400002)(446003)(478600001)(6506007)(14454004)(76176011)(6116002)(74482002)(3846002)(790700001)(66066001)(5250100002)(68736007)(1411001)(99286004)(186003)(7696005)(486006); DIR:OUT; SFP:1102; SCL:1; SRVR:SN1PR08MB1744; H:SN1PR08MB1854.namprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:0;
received-spf: None (protection.outlook.com: evequefou.be does not designate permitted sender hosts)
x-microsoft-antispam-message-info: aiLKplXOvzvhcn4DHI/LXttYHcD+AfF+v2+uza0JS/NLUDqoJNzZKbKZ5qYCNFJ43f1oDfgTuFy6EUgfm3N7mSZEE+TgR/fogB1h28TmjwWh62ln6gBKrJCwt5L2kWlQ9s4U/wCafTKTGbrxPtuddCAh40z4HcUVyrZozSuhg2XTPehaamVAvh9DOZm8tIKC
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_SN1PR08MB1854A941E8A703913ECBE821DA880SN1PR08MB1854namp_"
MIME-Version: 1.0
X-MS-Office365-Filtering-Correlation-Id: 8958b7dd-71c9-4008-cee4-08d5aa007645
X-OriginatorOrg: evequefou.be
X-MS-Exchange-CrossTenant-Network-Message-Id: 8958b7dd-71c9-4008-cee4-08d5aa007645
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Apr 2018 16:28:50.1457 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 41eaf50b-882d-47eb-8c4c-0b5b76a9da8f
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR08MB1744
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/p71DIXUDSqGUZ9_QR-bIPB5pZVQ>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Apr 2018 16:28:57 -0000

Yes, that would be sufficient.  My point is that the offload probably needs to leave packetization to the software and not attempt to parcel them out as part of the offload.

From: Watson Ladd [mailto:watsonbladd@gmail.com]
Sent: Monday, April 23, 2018 1:56 PM
To: Mike Bishop <mbishop@evequefou.be>
Cc: Lubashev, Igor <ilubashe@akamai.com>; Ian Swett <ianswett=40google.com@dmarc.ietf.org>; Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>; Praveen Balasubramanian <pravb@microsoft.com>; IETF QUIC WG <quic@ietf.org>; Deval, Manasi <manasi.deval@intel.com>
Subject: Re: Impact of hardware offloads on network stack performance



On Mon, Apr 23, 2018 at 1:06 PM, Mike Bishop <mbishop@evequefou.be<mailto:mbishop@evequefou.be>> wrote:
I agree – the point of offloading the crypto is not only to spare that cost on the CPU, though that can be useful, it’s to enable further offloads that can’t be done on the encrypted packet.

LRO that could simply group packets by CID and deliver a single batch of packets from a single CID in a single invocation seems like a great first step, and doesn’t even require crypto offload.  Once you have crypto offload, of course those packets per CID could be delivered already decrypted.  I’d want to see data on whether there’s value in making the received data look like one huge packet when there’s not a single bytestream, but that’s not something we need to specify here.

LSO is more interesting, and the challenge there is that the transport stack still needs to know what data was in which packet so that it can do loss recovery properly.  QUIC explicitly disallows fragmentation of the UDP datagram, for the “painful and potentially non-deterministic” reasons you referenced.

Forgive my complete ignorance, but doesn't it just need to know what packet it sent?

More explicitly the sender would maintain a queue of sendable packets, and mark which have been sent. Acknowledged packets are removed from the queue. In event of loss, they simply retransmit the same packet. It's up to higher layers to pick the right size of packet and take data from the individual QUIC streams. If you wanted to expose the stream structure, that sounds a lot more complicated.

Sincerely,
Watson

v2.23.0 | Report a Bug | By Email