Re: AEAD and header encryption overhead in QUIC

Matt Joras <> Thu, 18 June 2020 17:21 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 9E7843A0CE1 for <>; Thu, 18 Jun 2020 10:21:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id vMPfIgtXYKtw for <>; Thu, 18 Jun 2020 10:21:24 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::a2e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id C776E3A0CA7 for <>; Thu, 18 Jun 2020 10:21:23 -0700 (PDT)
Received: by with SMTP id q69so1619512vkq.10 for <>; Thu, 18 Jun 2020 10:21:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=eifcHAi/b6anFfzog/tzOoN8zc2F9SgD3JnvDRk6/NM=; b=UMs0+EkZii4y16k6vYl9lBdBDLKmGSnk4cR3JTVdSh83ugeABj0iWLGGcyVyAxrhsh rlBDqk5PNukWBddaU4T/ptRcxO0ba/EZhuzh+p3zgibOdCubzCdpoy4oOiWgbVmIC0Uf raOizABY42/H/aMcM7XnIVdnn6LuBiu2Q8ASM4Lpdgo4mwN08jahGZ+ysTVHR7jSmR2Y cyp7fIiEiKaTXuy8vtcyVImNd0M3I4N3Fn3sel42j1qJmHoUhUA33g/gl90KgG/Ga9in oYJST552Vtuc/odROP+Sx0AcsV4tl76gsHxZdfGER6EW4wIAPMHkqdIdO7ywQ3pFdszM MziA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=eifcHAi/b6anFfzog/tzOoN8zc2F9SgD3JnvDRk6/NM=; b=TOFgKxCUIOqxEfz1u6bhhOPn63pIfvqpiWQRc4xrhMVXCTyoyMXpjwNHYchAtuDMmn YCbPDq8jhNY8hGIjIzl9W0EU4t8TmhsWhfvxEunxttPQLqqnpJktp2sXukzK9DCUEmq9 +C+vopVamcI8s+k6lfKWKWebptvTQcyFluowTDV8FoqxHlWf4eRlA2sTnLV0sEJ2Rmw0 DqlQYAgTl8ldqUPqHjZCaBGRig6pQjDXUF+tuoFMm8Hwp79faJHx89XN0KWvQOj6cynb /spNcOaXUTBoFlPGsTzJx52igtxaXfoWKMJopQyWDNbpvBYZUOwDiHsLmZ9V95iEsH6Q KLDg==
X-Gm-Message-State: AOAM5331Gz2wY8BuyH3OsbcYerFPIqnPDj230w+gT4DFgDTJ77g8kTya saGktQZAhCuaSjZF/Sd+IQbsTHZN7aT7NMhQ8OE=
X-Google-Smtp-Source: ABdhPJx/XygiAPNA3v8PfqlKZ5mEEcYb0GezgRYp7rFJcjyeY1dPc+vk3+eu2BurlML5kz/AvysNcZsc6SIOQwKyPEw=
X-Received: by 2002:a1f:9511:: with SMTP id x17mr4391054vkd.101.1592500882717; Thu, 18 Jun 2020 10:21:22 -0700 (PDT)
MIME-Version: 1.0
References: <>
In-Reply-To: <>
From: Matt Joras <>
Date: Thu, 18 Jun 2020 10:21:11 -0700
Message-ID: <>
Subject: Re: AEAD and header encryption overhead in QUIC
To: Kazuho Oku <>
Content-Type: multipart/alternative; boundary="0000000000006b402405a85f01b4"
Archived-At: <>
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 18 Jun 2020 17:21:26 -0000

Wow, really interesting results, thanks for sharing!

I was curious, since it's not mentioned on the PRs that I saw, were these
tests done with any adjustments to the ACK frequency on the client? Or is
this with the default ACK policy of ACKing every other?

Matt Joras

On Thu, Jun 18, 2020 at 1:24 AM Kazuho Oku <> wrote:

> Hello folks,
> TL;DR: When existing crypto libraries are used, the AEAD cost of QUIC can
> become somewhat greater than that of TLS over TCP, due to those crypto
> libraries not optimized for handling short AEAD blocks. Crypto libraries
> optimized for short AEAD blocks can tighten the gap. Also, crypto libraries
> optimized for QUIC can combine AEAD operation and header protection vector
> generation, effectively eliminating the cost of header protection.
> A bit longer story:
> Recently, when testing quicly [1] using a NIC with *hardware support* for
> UDP GSO offload, we noticed that it uses +30% CPU cycles compared to TLS
> over TCP. This was due to our underlying crypto not being optimized for
> short AEAD blocks, and due to the call overhead of doing header protection
> one packet at a time. [2]
> We went on to implement our own AES-GCM library optimized for short AEAD
> blocks (i.e. packet-sized), that also generates the header protection
> vector while doing AEAD computation. By switching to the new crypto
> library, the performance drawback of the crypto itself compared to large
> AEAD blocks when down from 30% to 10% [3].
> The overhead of header protection (relative to all the crypto cost) on the
> send side is now between 0.3% to 1% when using packets sized around 1400
> bytes to 1500 bytes. The ratio against the total transport cost is going to
> be less than that, and in practice, I think that they are negligible. The
> story might be a bit different on the receive-side due to the dependency
> between header protection and AEAD, though I am sure there would also be
> ways to optimize.
> In terms of CPU cycles spent by the QUIC stack, the overhead compared to
> TLS-over-TCP came down to 5%. quicly can now sustain the send rate of 5Gbps
> while spending about 30% of CPU cycles of one modern Intel CPU core [2].
> These numbers are observations from one TLS / QUIC stack doing
> micro-benchmarks, though I tend to think that sharing performance numbers
> lead to people having better expectations regarding the CPU cost of QUIC.
> Some of us were also concerned about the cost of header protection (a.k.a.
> packet number encryption) [4]. Hence posting here.
> [1]
> [2]
> [3]
> [4]
> --
> Kazuho Oku