Return-Path: X-Original-To: quic@ietfa.amsl.com Delivered-To: quic@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 903583A0A8F for ; Thu, 18 Jun 2020 10:35:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.096 X-Spam-Level: X-Spam-Status: No, score=-1.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLY=1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xVFlK4PrCpD0 for ; Thu, 18 Jun 2020 10:35:54 -0700 (PDT) Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B09F63A09F2 for ; Thu, 18 Jun 2020 10:35:54 -0700 (PDT) Received: by mail-yb1-xb29.google.com with SMTP id o4so3538069ybp.0 for ; Thu, 18 Jun 2020 10:35:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=mBNuN5B/fuaeZ6XT8QH1cy2XvzA+QdVSUay5dUf79II=; b=Ad9SgUOoTqQ6dkhRkx4DbXM9Y5ALAU0wRsT1POx1hcgvyB+0BkXclJmrxChN4o+RVV 3NvhkNgAIiW0w4kpKFwO7E3YDHKaFoPnAnOJ0b0zLd7v6ynhfqEQcDTbRt7HBtjj2hWe eRkaycrClVbjeh13kR3NrZlC1mu6D+RVBJsYLJZJBwuwaUkV3P/TGU0UHGJFm+mzsGFn Yld1Q31QWml0voLiCyoi+tavRViuq3CMIvxOzDvYjiDCP6ml4F7K+OE1NESclQhn5hFP 54Bk+cPdSuUfJD8LkWxPseku2Oa0EwaXWW7rAFYJp4m07BS8LuNqAqVyL3q8cEKTOpkz 3Eiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=mBNuN5B/fuaeZ6XT8QH1cy2XvzA+QdVSUay5dUf79II=; b=exAOZAVGRvKXWFzr8VfOAXsCUFHRwmEhAw9Y21/QjuPgSvAQ7mlfqqQW9hqmjFMM9v ZNMVWibhoGtsxr/A186Pz8cHtQ9MufipeTsMmWfROcvNBQ7FX7SN5+1iGDMpocMebkT9 42i65+RFLhoBcZX5wtkkUFxaRB1GO4ZddfJRFn+yoriO4f/fOstz6aNcu+3evLFt7HJH iLrh2sX6438oOMJzTIBJY2xXTMZC9i1/V2/p7B8TY1YbaOTcfEzLvfjURxgdjQjpZ3u+ oeu8++2uga9qpQco1Mn2r3vYpeEP+H6mhydmbLd8Vp2s18EEIDU0BcxTg9BiMmDyQ+6l fo+w== X-Gm-Message-State: AOAM532UksWcbgoOEbh/MYbOMQr/t4fBHi0hBxNIrF88ugtxtDIdVTPB ZiO2Zwvf6l9/AwkDZyMnah+hHWaRx2NBSSJiCb8= X-Google-Smtp-Source: ABdhPJx9oQFflTCqO6QSqpMEkZ+bzQ4cJKXRExidKtSVKOfBOD70950DIUYkp58pMT+ajNBvHTOK9D76X6TngQ9Ikcs= X-Received: by 2002:a25:31c6:: with SMTP id x189mr8506966ybx.264.1592501753793; Thu, 18 Jun 2020 10:35:53 -0700 (PDT) Received: from 1058052472880 named unknown by gmailapi.google.com with HTTPREST; Thu, 18 Jun 2020 10:35:53 -0700 From: =?UTF-8?Q?Mikkel_Fahn=C3=B8e_J=C3=B8rgensen?= In-Reply-To: References: MIME-Version: 1.0 Date: Thu, 18 Jun 2020 10:35:53 -0700 Message-ID: Subject: Re: AEAD and header encryption overhead in QUIC To: Kazuho Oku , Matt Joras Cc: IETF QUIC WG Content-Type: multipart/alternative; boundary="00000000000056d08905a85f35d7" Archived-At: X-BeenThere: quic@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Main mailing list of the IETF QUIC working group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2020 17:35:57 -0000 --00000000000056d08905a85f35d7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I agree, thanks for sharing. I am not too surprised by the results though. I am a bit surprised that it is possible to use bulk encrypt headers as Nick suggests as I would have thought the key / nonce was associated with the packet, but I haven=E2=80=99t looked closely recently. Kind Regards, Mikkel Fahn=C3=B8e J=C3=B8rgensen On 18 June 2020 at 19.21.44, Matt Joras (matt.joras@gmail.com) wrote: Wow, really interesting results, thanks for sharing! I was curious, since it's not mentioned on the PRs that I saw, were these tests done with any adjustments to the ACK frequency on the client? Or is this with the default ACK policy of ACKing every other? Matt Joras On Thu, Jun 18, 2020 at 1:24 AM Kazuho Oku wrote: > Hello folks, > > TL;DR: When existing crypto libraries are used, the AEAD cost of QUIC can > become somewhat greater than that of TLS over TCP, due to those crypto > libraries not optimized for handling short AEAD blocks. Crypto libraries > optimized for short AEAD blocks can tighten the gap. Also, crypto librari= es > optimized for QUIC can combine AEAD operation and header protection vecto= r > generation, effectively eliminating the cost of header protection. > > A bit longer story: > > Recently, when testing quicly [1] using a NIC with *hardware support* for > UDP GSO offload, we noticed that it uses +30% CPU cycles compared to TLS > over TCP. This was due to our underlying crypto not being optimized for > short AEAD blocks, and due to the call overhead of doing header protectio= n > one packet at a time. [2] > > We went on to implement our own AES-GCM library optimized for short AEAD > blocks (i.e. packet-sized), that also generates the header protection > vector while doing AEAD computation. By switching to the new crypto > library, the performance drawback of the crypto itself compared to large > AEAD blocks when down from 30% to 10% [3]. > > The overhead of header protection (relative to all the crypto cost) on th= e > send side is now between 0.3% to 1% when using packets sized around 1400 > bytes to 1500 bytes. The ratio against the total transport cost is going = to > be less than that, and in practice, I think that they are negligible. The > story might be a bit different on the receive-side due to the dependency > between header protection and AEAD, though I am sure there would also be > ways to optimize. > > In terms of CPU cycles spent by the QUIC stack, the overhead compared to > TLS-over-TCP came down to 5%. quicly can now sustain the send rate of 5Gb= ps > while spending about 30% of CPU cycles of one modern Intel CPU core [2]. > > These numbers are observations from one TLS / QUIC stack doing > micro-benchmarks, though I tend to think that sharing performance numbers > lead to people having better expectations regarding the CPU cost of QUIC. > Some of us were also concerned about the cost of header protection (a.k.a= . > packet number encryption) [4]. Hence posting here. > > [1] https://github.com/h2o/quicly > [2] https://github.com/h2o/quicly/pull/359 > [3] https://github.com/h2o/picotls/pull/310 > [4] > https://mailarchive.ietf.org/arch/msg/quic/Wy8JXwKyeYgWgRWizWDbyOLg0xc/ > > -- > Kazuho Oku > --00000000000056d08905a85f35d7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =
I ag= ree, thanks for sharing. I am not too surprised by the results though.

I am a bit surprised th= at it is possible to use bulk encrypt headers as Nick suggests as I would h= ave thought the key / nonce was associated with the packet, but I haven=E2= =80=99t looked closely recently.

=
Kind Regards,
Mikkel Fahn=C3= =B8e J=C3=B8rgensen


On 18 J= une 2020 at 19.21.44, Matt Joras (m= att.joras@gmail.com) wrote:

Wow, reall= y interesting results, thanks for sharing!

I was curious, sinc= e it's not mentioned on the PRs that I saw, were these tests done with = any adjustments to the ACK frequency on the client? Or is this with the def= ault ACK policy of ACKing every other?

Matt Joras

On = Thu, Jun 18, 2020 at 1:24 AM Kazuho Oku <kazuhooku@gmail.com> wrote:
Hello folks,

TL;DR:= When existing crypto libraries are used, the AEAD cost of QUIC can become = somewhat greater than that of TLS over TCP, due to those crypto libraries n= ot optimized for handling short AEAD blocks. Crypto libraries optimized for= =C2=A0short AEAD blocks can tighten the gap. Also, crypto libraries optimiz= ed for QUIC can combine AEAD operation and header protection vector generat= ion, effectively eliminating the cost of header protection.

<= /div>
A bit longer story:

Recently, when testi= ng quicly=C2=A0[1] using a NIC with *hardware support* for UDP GSO offload,= we noticed that it uses +30% CPU cycles compared to TLS over TCP. This was= due to our underlying crypto not being optimized for short AEAD blocks, an= d due to the call overhead of doing header protection one packet=C2=A0at a = time. [2]

We went on to implement our own AES-GCM = library optimized for short AEAD blocks (i.e. packet-sized), that also gene= rates the header protection vector while doing AEAD computation. By switchi= ng to the new crypto library, the performance drawback of the crypto itself= compared to large AEAD blocks when down from 30% to 10% [3].=C2=A0

The overhead of header protection (relative to all the cr= ypto cost) on the send side is now between 0.3% to 1% when using packets si= zed around 1400 bytes to 1500 bytes. The ratio against the total transport = cost is going to be less than that, and in practice, I think that they are = negligible. The story might be a bit different on the receive-side due to t= he dependency between header protection and AEAD, though I am sure there wo= uld also be ways to optimize.

In terms of CPU cycl= es spent by the QUIC stack, the overhead compared to TLS-over-TCP came down= to 5%. quicly can now sustain the send rate of 5Gbps while spending about = 30% of CPU cycles of one modern Intel CPU core [2].

These numbers are observations from one TLS / QUIC stack doing micro-benc= hmarks, though I tend to think that sharing performance numbers lead to peo= ple having better expectations regarding the CPU cost of QUIC. Some of us w= ere also concerned about the cost of header protection (a.k.a. packet numbe= r encryption) [4]. Hence posting here.

[1] https://gi= thub.com/h2o/quicly
=

--
Kazuho Oku
--00000000000056d08905a85f35d7--