Re: Packet number encryption

[Kazuho Oku <[e-mail]>]

Victor, Mikkel, thank you for the estimations.

To me it seems that the estimated overhead of 0.1% seems like a
natural number that we would see on production environment, where the
average packet size is not small.

I also agree that it would be worthwhile to look the case where small
packets are exchanged. The numbers deducted from Solarflare report is
interesting, however I am not sure if that reflects the ordinary use
of a protocol; my understanding is that HFT is exceptional.

One benchmark that might give us a more meaningful number is the DNS benchmark.

https://www.knot-dns.cz/benchmark/ lists a benchmark of several
authoritative DNS servers. The TLD benchmark and the Hosting (10k)
benchmark gives us the rough estimation on how much PPS we can achieve
when exchanging small packets.

The numbers we see on the benchmark is roughly 2M RPS (4M pps) on a 16
core (8x2) Intel Xeon processer running with HP enabled, when the
fastest DNS server is being used.

Running `openssl speed -evp` on a similar CPU gives me the following result:

    $ /usr/local/openssl-1.1.0/bin/openssl speed -evp aes128
    Doing aes-128-cbc for 3s on 16 size blocks: 133387877 aes-128-cbc's in 3.00s

This shows that 40M AES block operations can be run on a single core,
per second, which in turn means that a server with 16 cores can
perform 640M AES block operations per second (or 1,280M if HT is
enabled and if AES is not the bottleneck).

To summarize, if DNS had packet encryption, the overhead would be
somewhere from 0.3% to 0.6%.

Considering that, I would anticipate that the overhead of packet
number encryption will be neglible even for short packet workloads.


2018-02-12 16:19 GMT+09:00 Victor Vasiliev <[e-mail]>:
> I have no idea how you get that number (by which I mean, I have a lot of
> guesses, but none of which are solid or useful here).  I looked at the
> profiles of various workloads we're running in production, and I estimate
> that none of them would be impacted by PN encryption by worse than 0.1%.
>
> On Sat, Feb 10, 2018 at 12:12 PM, Praveen Balasubramanian
> <[e-mail]> wrote:
>>
>> Makes sense. The 1%+ overhead I had quoted was in comparison to full
>> protocol processing all the way from app to NIC. This is with an unoptimized
>> early QUIC implementation and not fully optimized UDP/IP stack (we have
>> focused much much more on optimizing TCP in the past). After software stack
>> optimizations, the crypto share of the cost will only keep going up for
>> which we have no technique other than to offload when such support is
>> available in hardware and offload is way more challenging when workload is
>> hosted in VMs and containers.
>>
>>
>>
>> From: Mikkel Fahnøe Jørgensen [mailto:[e-mail]]
>> Sent: Saturday, February 10, 2018 3:36 AM
>> To: Victor Vasiliev <[e-mail]>
>> Cc: Praveen Balasubramanian <[e-mail]>; [e-mail]; Marten
>> Seemann <[e-mail]>; huitema <[e-mail]>
>> Subject: Re: Packet number encryption
>>
>>
>>
>> To put numbers into perspective using Intel 2015 data
>>
>>
>>
>>
>> https://software.intel.com/en-us/articles/improving-openssl-performance#_Toc416943490
>>
>>
>>
>> A 64 byte message in AES-GCM AEAD in HW would use 1.03 cycles per byte or
>> 66 cycles total, or 22ns on a 3GHz core.
>>
>>
>>
>> For packet numbers we use the CBC encrypt numbers because here AES cannot
>> exploit block parallelism.
>>
>> Here we see 4.44 cycles/byte in HW or 71 cycles per block. With a 3GHz
>> setup that would amount to about 24ns overhead for packet encryption.
>>
>>
>>
>> Clearly it makes no sense that AES-GCM is faster than a single AES block
>> encryption, but these are only approximate numbers and CBC mode might have a
>> little overhead, so we clamp packet numbers to 22ns.
>>
>>
>>
>> Taking the 98ns overhead by the Solarflare report we get a total
>> (simplified) processing time is 98ns non-crypto, 22ns for packet number, and
>> 22ns for AEAD totalling 142ns. So the packet number encryption overhead
>> would be 22/(98+22)*100% = 18%. The numbers ignore other QUIC bookkeeping,
>> but that can be done in other cores or outside the latency critical window.
>>
>>
>>
>> This does not take into account that AEAD operation may operate less than
>> optimal because the packet number must be extracted first. On the other
>> hand, it is also not a disastrous overhead if no good alternative can be
>> found.
>>
>>
>>
>> Earlier AES-NI I’ve seen from Intel doc suggests around 100cycles in HW
>> for a single AES-128 block which would be 33ns per packet number in the
>> above example.
>>
>>
>>
>> On 10 February 2018 at 06.18.25, Mikkel Fahnøe Jørgensen
>> ([e-mail]) wrote:
>>
>> 98ns for 68 byte messages
>
>



-- 
Kazuho Oku