AEAD and header encryption overhead in QUIC

[Kazuho Oku <[e-mail]>]

Hello folks,

TL;DR: When existing crypto libraries are used, the AEAD cost of QUIC can
become somewhat greater than that of TLS over TCP, due to those crypto
libraries not optimized for handling short AEAD blocks. Crypto libraries
optimized for short AEAD blocks can tighten the gap. Also, crypto libraries
optimized for QUIC can combine AEAD operation and header protection vector
generation, effectively eliminating the cost of header protection.

A bit longer story:

Recently, when testing quicly [1] using a NIC with *hardware support* for
UDP GSO offload, we noticed that it uses +30% CPU cycles compared to TLS
over TCP. This was due to our underlying crypto not being optimized for
short AEAD blocks, and due to the call overhead of doing header protection
one packet at a time. [2]

We went on to implement our own AES-GCM library optimized for short AEAD
blocks (i.e. packet-sized), that also generates the header protection
vector while doing AEAD computation. By switching to the new crypto
library, the performance drawback of the crypto itself compared to large
AEAD blocks when down from 30% to 10% [3].

The overhead of header protection (relative to all the crypto cost) on the
send side is now between 0.3% to 1% when using packets sized around 1400
bytes to 1500 bytes. The ratio against the total transport cost is going to
be less than that, and in practice, I think that they are negligible. The
story might be a bit different on the receive-side due to the dependency
between header protection and AEAD, though I am sure there would also be
ways to optimize.

In terms of CPU cycles spent by the QUIC stack, the overhead compared to
TLS-over-TCP came down to 5%. quicly can now sustain the send rate of 5Gbps
while spending about 30% of CPU cycles of one modern Intel CPU core [2].

These numbers are observations from one TLS / QUIC stack doing
micro-benchmarks, though I tend to think that sharing performance numbers
lead to people having better expectations regarding the CPU cost of QUIC.
Some of us were also concerned about the cost of header protection (a.k.a.
packet number encryption) [4]. Hence posting here.

[1] https://github.com/h2o/quicly
[2] https://github.com/h2o/quicly/pull/359
[3] https://github.com/h2o/picotls/pull/310
[4] https://mailarchive.ietf.org/arch/msg/quic/Wy8JXwKyeYgWgRWizWDbyOLg0xc/

-- 
Kazuho Oku