Re: Impact of hardware offloads on network stack performance
craigt <c@gryning.com> Thu, 10 May 2018 12:50 UTC
Return-Path: <c@gryning.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A1E4A12D883 for <quic@ietfa.amsl.com>; Thu, 10 May 2018 05:50:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.609
X-Spam-Level:
X-Spam-Status: No, score=-2.609 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gryning-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cXAK9dusNDL9 for <quic@ietfa.amsl.com>; Thu, 10 May 2018 05:50:14 -0700 (PDT)
Received: from mail-lf0-x235.google.com (mail-lf0-x235.google.com [IPv6:2a00:1450:4010:c07::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A78E1275AB for <quic@ietf.org>; Thu, 10 May 2018 05:50:14 -0700 (PDT)
Received: by mail-lf0-x235.google.com with SMTP id t129-v6so2807959lff.3 for <quic@ietf.org>; Thu, 10 May 2018 05:50:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gryning-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0s/vCrkmQGVG9bnxM6blDr3P4Q97LYzmEdCDZgFtXYs=; b=Kcaj/YfwC0jMMQGqo1PxX9tp5wUnY2I2e6OhhIgEOgceClQN46pgm9xS76pxK3asDi lW0uncXLcl0FLbrrq07xiMY8Mhx8KvkMmt/+JcgQQFe6IndxcfzJ2AU2k9+VhOhu6tcd nHenNVoCsjyLXvB7ESGJ+aSOSEPddhyHWjwqFyjitV690l0P4OILBZQBHcHS+bLEts2D jCaSdJxfWp8cOS1pYooufzE0XIueaCX5U/6jTdaYpsoaZ0I/CCwhAhpOMl5W7XshnYxV i+jlz0G2qnYwf2wF8N0Ra+k9eGK+hHL5PGQCtNvlC6w8wEdRV+5/Rql2CA+VJHbXrfmB BiqA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0s/vCrkmQGVG9bnxM6blDr3P4Q97LYzmEdCDZgFtXYs=; b=gNGQkhhGoekssM248/p56AkpiaXxOiBpaToPnEDaySpGleG4l+nykd4poqtw/pky0M EtX0M8m+J7CDle2xaQQmvp1XbppSfnNqGSmeKlyvnKOCUjm5sFgKBLb5yQ2h/zhWF/6F nTY0iWQTF+dCj5+W1jMt+sWD69Ab7ShHuR0lKAqEsPOSV0jQvd6mrzA6O26xwHciLv04 UWI88jEsl4UahaR3w2K9+QiLCwQ8nZ89Xr5skAR+bKRtza4S+hWi7xKZPpMd5rtS56TO xCW7Uvo+QWfFfla92WI2srqx8mA2Rpk+bsUEc+NJ+nOTPpn8jTa6l0MrRvPdCj7UkkfB SRnw==
X-Gm-Message-State: ALKqPwct5xbZACRJiCIhApBpT/fBqzBx5QsOZFtlbOBnB7TD6tlrGERl zIp40e19yfgac1Tq9IAxQdYsAtcNTWhCs9RLSIcOnw==
X-Google-Smtp-Source: AB8JxZptleoyHeF2lusWWbh9NQBpvs4c8NVt674zZkgBpOIo6qFKSjXit653VmqOp/Ek4SG5g7Q3OVHzq7aC9ROEbbg=
X-Received: by 2002:a19:d853:: with SMTP id p80-v6mr1019026lfg.36.1525956612330; Thu, 10 May 2018 05:50:12 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a19:d012:0:0:0:0:0 with HTTP; Thu, 10 May 2018 05:50:11 -0700 (PDT)
X-Originating-IP: [2001:41c1:4:8000:186e:9af4:4c85:721a]
In-Reply-To: <A6F55755-D7E1-40E4-8EC3-D5FE77A2EDBE@fb.com>
References: <CY4PR21MB0630CE4DE6BA4EDF54A1FEB1B6A40@CY4PR21MB0630.namprd21.prod.outlook.com> <CAN1APde93+S8CP-KcZmqPKCCvsGRiq6ECPUoh_Qk0j9hqs8h0Q@mail.gmail.com> <SN1PR08MB1854E64D7C370BF0A7456977DABB0@SN1PR08MB1854.namprd08.prod.outlook.com> <CAKcm_gNhsHxXM77FjRj-Wh4JxA21NAWKXX3KBT=eZJsCdacM7Q@mail.gmail.com> <DB6PR10MB1766789B25E31EBE70564BD3ACBA0@DB6PR10MB1766.EURPRD10.PROD.OUTLOOK.COM> <SN1PR08MB18548155722C665FFD45CC4DDABA0@SN1PR08MB1854.namprd08.prod.outlook.com> <CAKcm_gMYf6AbHgvg+WKv-woQCtzz9WztiEDf-iRUjKw63Qx=9Q@mail.gmail.com> <1F436ED13A22A246A59CA374CBC543998B60C7AD@ORSMSX111.amr.corp.intel.com> <CAN1APdc3y0EwFqeYVvZs7MtBHhS_9CzwGmcwRqi_6GHWzF3_2Q@mail.gmail.com> <1F436ED13A22A246A59CA374CBC543998B60C851@ORSMSX111.amr.corp.intel.com> <CAN1APdcPkO-HfXqqvjeee6K8U8KSZQdtx=6fW1vZo+H6pKfzzQ@mail.gmail.com> <CAKcm_gP8TRPT7yi1y=mU5Fvq7xB5-1ieyKFLQMPomfabYbkcxA@mail.gmail.com> <bf540ec1f6f045aca1cf2379380630b5@usma1ex-dag1mb5.msg.corp.akamai.com> <SN1PR08MB185446750FCDA2B58C9833C8DA890@SN1PR08MB1854.namprd08.prod.outlook.com> <CACsn0cmJcAxORo4Cd-qyGJL3ZOT5Yz0WgdMqv_AGi4DjedyzFA@mail.gmail.com> <A6F55755-D7E1-40E4-8EC3-D5FE77A2EDBE@fb.com>
From: craigt <c@gryning.com>
Date: Thu, 10 May 2018 13:50:11 +0100
Message-ID: <CAK-1kemWAW+2Sk5aKJ+RXKUAyjFii4MRH=xK7SE0mr=tbCiRfw@mail.gmail.com>
Subject: Re: Impact of hardware offloads on network stack performance
To: Roberto Peon <fenix@fb.com>
Cc: Watson Ladd <watsonbladd@gmail.com>, Mike Bishop <mbishop@evequefou.be>, Praveen Balasubramanian <pravb@microsoft.com>, "Lubashev, Igor" <ilubashe@akamai.com>, Ian Swett <ianswett=40google.com@dmarc.ietf.org>, Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>, IETF QUIC WG <quic@ietf.org>, "Deval, Manasi" <manasi.deval@intel.com>
Content-Type: multipart/alternative; boundary="000000000000d22b22056bd97523"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/JXWQV1yEQiSJzSrP4zbKQr3Mj6M>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 10 May 2018 12:50:19 -0000
Has anyone experimented with the zerocopy socket interfaces for exactly this use case? https://netdevconf.org/2.1/papers/netdev.pdf It would seem that is a more natural fit to the assertions made by several of the authors surrounding packetisation. I'm unaware if Windows/BSD have similar interfaces/implementations. c On 30 April 2018 at 21:37, Roberto Peon <fenix@fb.com> wrote: > QUIC (mostly) doesn’t do that (rexmit a packet verbatim)—you’d need a new > packet# for the rexmitted packet. > > -=R > > > > *From: *QUIC <quic-bounces@ietf.org> on behalf of Watson Ladd < > watsonbladd@gmail.com> > *Date: *Monday, April 23, 2018 at 1:56 PM > *To: *Mike Bishop <mbishop@evequefou.be> > *Cc: *Praveen Balasubramanian <pravb@microsoft.com>, "Lubashev, Igor" < > ilubashe@akamai.com>, Ian Swett <ianswett=40google.com@dmarc.ietf.org>, > Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>, IETF QUIC WG <quic@ietf.org>, > "Deval, Manasi" <manasi.deval@intel.com> > *Subject: *Re: Impact of hardware offloads on network stack performance > > > > > > > > On Mon, Apr 23, 2018 at 1:06 PM, Mike Bishop <mbishop@evequefou.be> wrote: > > I agree – the point of offloading the crypto is not only to spare that > cost on the CPU, though that can be useful, it’s to enable further offloads > that can’t be done on the encrypted packet. > > > > LRO that could simply group packets by CID and deliver a single batch of > packets from a single CID in a single invocation seems like a great first > step, and doesn’t even require crypto offload. Once you have crypto > offload, of course those packets per CID could be delivered already > decrypted. I’d want to see data on whether there’s value in making the > received data look like one huge packet when there’s not a single > bytestream, but that’s not something we need to specify here. > > > > LSO is more interesting, and the challenge there is that the transport > stack still needs to know what data was in which packet so that it can do > loss recovery properly. QUIC explicitly disallows fragmentation of the UDP > datagram, for the “painful and potentially non-deterministic” reasons you > referenced. > > > > Forgive my complete ignorance, but doesn't it just need to know what > packet it sent? > > More explicitly the sender would maintain a queue of sendable packets, and > mark which have been sent. Acknowledged packets are removed from the queue. > In event of loss, they simply retransmit the same packet. It's up to higher > layers to pick the right size of packet and take data from the individual > QUIC streams. If you wanted to expose the stream structure, that sounds a > lot more complicated. > > > Sincerely, > > Watson > > > > >
- Impact of hardware offloads on network stack perf… Praveen Balasubramanian
- Re: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- RE: Impact of hardware offloads on network stack … Mike Bishop
- Re: Impact of hardware offloads on network stack … Ian Swett
- Re: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- RE: Impact of hardware offloads on network stack … Mike Bishop
- Re: Impact of hardware offloads on network stack … Ian Swett
- Re: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- RE: Impact of hardware offloads on network stack … Praveen Balasubramanian
- RE: Impact of hardware offloads on network stack … Deval, Manasi
- RE: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- RE: Impact of hardware offloads on network stack … Deval, Manasi
- RE: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- Re: Impact of hardware offloads on network stack … Ian Swett
- RE: Impact of hardware offloads on network stack … Lubashev, Igor
- RE: Impact of hardware offloads on network stack … Mike Bishop
- Re: Impact of hardware offloads on network stack … Watson Ladd
- Re: Impact of hardware offloads on network stack … Jana Iyengar
- RE: Impact of hardware offloads on network stack … Mike Bishop
- Re: Impact of hardware offloads on network stack … Roberto Peon
- Re: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- Re: Impact of hardware offloads on network stack … craigt
- Re: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- Re: Impact of hardware offloads on network stack … Eggert, Lars
- RE: Impact of hardware offloads on network stack … Deval, Manasi
- Re: Impact of hardware offloads on network stack … Boris Pismenny
- Re: Impact of hardware offloads on network stack … Eggert, Lars
- Re: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- Re: Impact of hardware offloads on network stack … Mikkel Fahnøe Jørgensen
- Re: Impact of hardware offloads on network stack … craigt