RE: hardware offload (was: Packet number encryption)

Praveen Balasubramanian <pravb@microsoft.com> Fri, 09 February 2018 23:35 UTC

Return-Path: <pravb@microsoft.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B8F312D84D for <quic@ietfa.amsl.com>; Fri, 9 Feb 2018 15:35:34 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level:
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w_pMwg3JV-IN for <quic@ietfa.amsl.com>; Fri, 9 Feb 2018 15:35:31 -0800 (PST)
Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0093.outbound.protection.outlook.com [104.47.41.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 79BF312D838 for <quic@ietf.org>; Fri, 9 Feb 2018 15:35:31 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hLzK3xt3U6VkR31A6kACwuECC0w62gnkPaIHVUnoy0c=; b=bH0Tv918BDBJIR8Xzw/iD9BCEicH+U/KEpepPPVAFi5jxGMjg2Dp/yA4iZW2CUqmUUwbnyIbaNGsBpAKiEYacFzaUqd66dH1pKSqedxxhCzIvNRySCDjnYcO06DQmV7oq+oZN65jap6gZdIPbFjt9feeb6FJ8+2jsJMR/NTg1w0=
Received: from CY4PR21MB0133.namprd21.prod.outlook.com (10.173.189.15) by CY4PR21MB0149.namprd21.prod.outlook.com (10.173.189.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.506.3; Fri, 9 Feb 2018 23:35:29 +0000
Received: from CY4PR21MB0133.namprd21.prod.outlook.com ([fe80::f4bd:bebf:7ea5:7dc5]) by CY4PR21MB0133.namprd21.prod.outlook.com ([fe80::f4bd:bebf:7ea5:7dc5%6]) with mapi id 15.20.0506.011; Fri, 9 Feb 2018 23:35:29 +0000
From: Praveen Balasubramanian <pravb@microsoft.com>
To: Ian Swett <ianswett@google.com>, huitema <huitema@huitema.net>
CC: "Salz, Rich" <rsalz@akamai.com>, Eric Rescorla <ekr@rtfm.com>, "quic@ietf.org" <quic@ietf.org>
Subject: RE: hardware offload (was: Packet number encryption)
Thread-Topic: hardware offload (was: Packet number encryption)
Thread-Index: AQHToeMqYmtt/wnF60OrwLUYIJRfoKOcmxoAgAAYXpA=
Date: Fri, 09 Feb 2018 23:35:29 +0000
Message-ID: <CY4PR21MB01337BAE464D5FE1CDE85913B6F20@CY4PR21MB0133.namprd21.prod.outlook.com>
References: <CABkgnnVyo3MmWtVULiV=FJTnR528qfY8-OmKGWAs0bCvri-a_g@mail.gmail.com> <bdf88936-8edc-d56e-ee59-c9d597058edd@huitema.net> <CY4PR21MB01337C8A700E58B49D90B712B6FC0@CY4PR21MB0133.namprd21.prod.outlook.com> <119b3276-5799-1cc3-8982-7479171bbf27@huitema.net> <CAOYVs2pi8-NVuS+crNMfjsP-n5upK3=5tPeQ8OSGpOvL6RTrjA@mail.gmail.com> <CY4PR21MB0133A1117B2733BBCF049C5FB6FC0@CY4PR21MB0133.namprd21.prod.outlook.com> <MWHPR08MB24327A7BB5AE1AE70FE5CDB1DAF30@MWHPR08MB2432.namprd08.prod.outlook.com> <533a0a2e-3a87-b55f-84ce-c52bc03cd81c@huitema.net> <MWHPR21MB0144C68102972A668611E1FCB6F20@MWHPR21MB0144.namprd21.prod.outlook.com> <CY4PR21MB01332141C3563ABBA240C566B6F20@CY4PR21MB0133.namprd21.prod.outlook.com> <CABcZeBNeTT79nd+d7h-KFPpFYxpr5wt1KgwPY=M0_UQpCkKq1w@mail.gmail.com> <CY4PR21MB01337A5E81D8A8A1D7518D97B6F20@CY4PR21MB0133.namprd21.prod.outlook.com> <D3800B30-E1F5-4955-8F85-6FEF36AD2E23@akamai.com> <CAKcm_gO-2zejQnLCCzHvvG=gP70o9EAUQz8v2oYUiK=nFjyUCw@mail.gmail.com> <b8b848ba-e811-4ac0-5406-8d56fe7f7bae@huitema.net> <CAKcm_gNAUDqPxZ16nbSuzOS-815jJwiMasqwbyiiD9Cw5Gzpzg@mail.gmail.com>
In-Reply-To: <CAKcm_gNAUDqPxZ16nbSuzOS-815jJwiMasqwbyiiD9Cw5Gzpzg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [2001:4898:80e8:6::712]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; CY4PR21MB0149; 7:lnqgjSSkW4JIQiMDcugVcIAAWtERCbfIld1g3DtJCb0dq9BrXSHIW3lhGCJd0PyuCUO5FNrVOgmZ3UE6eSNm1d7ZXfV8L6DJCugOyD2NL6GaUr64QkQuz2KODXhOG7KWzvBhjkiBYthB7tU/ZYgTol1CGn/zQxPdeRT5RlEQYKhzjdVjbQ1Xo6yrwYsVHzbIYDeRbDTwQxW0zDS/k+ceL/2rc4nRwlQLHNM4O24S/GSYLLYYYA/Vuv8gfE+4MTyx
x-ms-exchange-antispam-srfa-diagnostics: SSOS;
x-ms-office365-filtering-ht: Tenant
x-ms-office365-filtering-correlation-id: 7f59eb76-aaa9-45ad-44bb-08d57015ce08
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(48565401081)(5600026)(4604075)(3008032)(2017052603307)(7193020); SRVR:CY4PR21MB0149;
x-ms-traffictypediagnostic: CY4PR21MB0149:
x-ld-processed: 72f988bf-86f1-41af-91ab-2d7cd011db47,ExtAddr
x-microsoft-antispam-prvs: <CY4PR21MB0149D2B792633E6A3E428E7AB6F20@CY4PR21MB0149.namprd21.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(275133514327357)(28532068793085)(89211679590171)(211936372134217)(21748063052155)(47284530071512);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425038)(6040501)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3231101)(2400082)(944501161)(3002001)(6055026)(61426038)(61427038)(6041288)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123558120)(20161123560045)(20161123562045)(6072148)(201708071742011); SRVR:CY4PR21MB0149; BCL:0; PCL:0; RULEID:; SRVR:CY4PR21MB0149;
x-forefront-prvs: 057859F9C5
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(396003)(366004)(39380400002)(346002)(39860400002)(376002)(199004)(189003)(22452003)(6436002)(229853002)(10290500003)(3660700001)(2900100001)(3280700002)(6346003)(86612001)(9686003)(53936002)(236005)(8990500004)(6306002)(55016002)(316002)(10090500001)(102836004)(54896002)(8936002)(105586002)(966005)(7696005)(106356001)(76176011)(6116002)(790700001)(14454004)(81156014)(81166006)(2906002)(74316002)(8676002)(478600001)(5250100002)(53546011)(6506007)(7736002)(68736007)(2950100002)(4326008)(97736004)(86362001)(19609705001)(5660300001)(186003)(99286004)(561944003)(25786009)(93886005)(33656002)(110136005)(606006)(6246003)(54906003); DIR:OUT; SFP:1102; SCL:1; SRVR:CY4PR21MB0149; H:CY4PR21MB0133.namprd21.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en;
received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts)
authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com;
x-microsoft-antispam-message-info: JMX28tctei///zQzu5ZxUrdyrP6xPwNR6wtqYHxa245ZW2hJi5+2wC7Q9xKjS9L94Ez4RtUJHRseq5ldl6Yg9A==
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_CY4PR21MB01337BAE464D5FE1CDE85913B6F20CY4PR21MB0133namp_"
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f59eb76-aaa9-45ad-44bb-08d57015ce08
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Feb 2018 23:35:29.4346 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR21MB0149
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/YNxqvLncMy133yaD5A4nTS1hujg>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Feb 2018 23:35:34 -0000

We have precedence in Windows with IPsec task offload which is symmetric crypto offload. The goal is to not have to make separate API calls to process crypto. The main advantage of crypto offload results from being able to make it a inline step in the NIC packet processing engine as part of the data path. This allows you to offload more mundane packet processing tasks going forward like we do with TCP. Some programmable NICs might be able to do a second pass but I am not a hardware expert to say if all NICs will be able to. There is precedence of some form of hardware crypto offload for TCP/TLS both as a side function and inline in network card. Some links from a quick web search:
https://www.chelsio.com/crypto-offload/
https://aws.amazon.com/documentation/cloudhsm/
https://netdevconf.org/1.2/session.html?boris-pismenny
I cannot comment on whether requiring an extra pass will render existing hardware inapplicable for QUIC. I have been trying hard to see if we can avoid an extra step regardless of crypto offload.

From: Ian Swett [mailto:ianswett@google.com]
Sent: Friday, February 9, 2018 1:46 PM
To: huitema <huitema@huitema.net>
Cc: Salz, Rich <rsalz@akamai.com>; Praveen Balasubramanian <pravb@microsoft.com>; Eric Rescorla <ekr@rtfm.com>; quic@ietf.org
Subject: Re: hardware offload (was: Packet number encryption)

I should say that I don't have a specific API, implementation, or NIC in mind, so hopefully someone(Praveen?) can provide some context about what is possible with existing APIs or what is relatively easy to add to them.

The API I had in mind was of the "encrypt this byte string" form, but in the NIC.  ie: When I write a sequence of packets I supply crypto info, as well as a minimum release time(or pacing rate).  The goal is to do the bulk crypto as the network stack is packetizing and copying into the NIC memory.  Ideally this can avoid an extra memcpy and avoids touching the data to be sent until the last possible moment.

Hardware acceleration of AES in modern CPUs is extremely good, so another approach is to do the above in a userspace networking stack or the kernel and use the CPU's AES acceleration.  If the networking code that was writing into NIC memory had a deep understanding of QUIC, possibly it could do the packet number encryption as a second pass?

On Fri, Feb 9, 2018 at 3:17 PM, Christian Huitema <huitema@huitema.net<mailto:huitema@huitema.net>> wrote:
On 2/9/2018 8:20 AM, Ian Swett wrote:

> One issue with the current encryption proposal is you can't use
> hardware crypto offload for the bulk encryption because the packet
> number has to be encrypted after the bulk data.
>
> Any suggestions on how to fix that?

Ian, could you be a bit more explicit about the type of hardware offload
that you have in mind?

I can think of two extreme types. One is embedding crypto offload in the
NIC, and is generally tied to TCP/TLS/TLO. That one type has no hope
whatsoever of working with UDP/QUIC. The other type is simply providing
an "encrypt this byte string" API, and can be trivially used with QUIC
-- much like we can use Intel's AES extended instructions today. So
obviously you have something else in mind, and maybe you should explain it.

-- Christian Huitema