RE: Packet Number Encryption Performance

[Nick Banks <[e-mail]>]

I just want to add, my implementation already uses ECB from bcrypt (and I do the XOR) already. Bcrypt doesn’t expose CTR mode directly.

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

________________________________
From: Praveen Balasubramanian
Sent: Friday, June 22, 2018 9:26:44 AM
To: Ian Swett; Kazuho Oku
Cc: Nick Banks; IETF QUIC WG
Subject: RE: Packet Number Encryption Performance

Ian, do you expect that fraction of overall cost to hold once the UDP stack is optimized? Is your measurement on top of the recent kernel improvements? I expect crypto fraction of overall cost to keep increasing over time as the network stack bottlenecks are eliminated.

Kazuho, should the draft describe the optimizations you are making? Or are these are too OpenSSL specific?

From: QUIC [mailto:[e-mail]] On Behalf Of Ian Swett
Sent: Friday, June 22, 2018 4:45 AM
To: Kazuho Oku <[e-mail]>
Cc: Nick Banks <[e-mail]>; IETF QUIC WG <[e-mail]>
Subject: Re: Packet Number Encryption Performance

Thanks for digging into the details of this, Kazuho.  <4% increase in crypto cost is a bit more than I originally expected(~2%), but crypto is less than 10% of my CPU usage, so it's still less than 0.5% total, which is acceptable to me.

On Fri, Jun 22, 2018 at 2:45 AM Kazuho Oku <[e-mail]<mailto:[e-mail]>> wrote:


2018-06-22 12:22 GMT+09:00 Kazuho Oku <[e-mail]<mailto:[e-mail]>>:


2018-06-22 11:54 GMT+09:00 Nick Banks <[e-mail]<mailto:[e-mail]>>:
Hi Kazuho,

Thanks for sharing your numbers as well! I'm bit confused where you say you can reduce the 10% overhead to 2% to 4%. How do you plan on doing that?

As stated in my previous mail, the 10% of overhead consists of three parts, each consuming comparable number of CPU cycles. The two among the three is related to the abstraction layer and how CTR is implemented, while the other one is the core AES-ECB operation cost.

It should be able to remove the costly abstraction layer.

It should also be possible to remove the overhead of CTR, since in PNE, we need to XOR at most 4 octets (applying XOR is the only difference between CTR and ECB). That cost should be something that should be possible to be nullified.

Considering these aspects, and by looking at the numbers on the OpenSSL source code (as well as considering the overhead of GCM), my expectation goes to 2% to 4%.

Just did some experiments and it seems that the expectation was correct.

The benchmarks tell me that the overhead goes down from 10.0% to 3.8%, by doing the following:

* remove the overhead of CTR abstraction (i.e. use the ECB backend and do XOR by ourselves)
* remove the overhead of the abstraction layer (i.e. call the method returned by EVP_CIPHER_meth_get_do_cipher instead of calling EVP_EncryptUpdate)

Of course the changes are specific to OpenSSL, but I would expect that you can expect similar numbers assuming that you have access to an optimized AES implementation.



Sent from my Windows 10 phone
[HxS - 15254 - 16.0.10228.20075]

________________________________
From: Kazuho Oku <[e-mail]<mailto:[e-mail]>>
Sent: Thursday, June 21, 2018 7:21:17 PM
To: Nick Banks
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: Packet Number Encryption Performance

Hi Nick,

Thank you for bringing the numbers to the list.

I have just run a small benchmark using Quicly, and I see comparable numbers.

To be precise, I see 10.0% increase of CPU cycles when encrypting a Initial packet of 1,280 octets. I expect that we will see similar numbers on other QUIC stacks that also use picotls (with OpenSSL as a backend). Note that the number is only comparing the cost of encryption, the overhead ratio will be much smaller if we look at the total number of CPU cycles spent by a QUIC stack as a whole.

Looking at the profile, the overhead consists of three operations that each consumes comparable CPU cycles: core AES operation (using AES-NI), CTR operation overhead, CTR initialization. Note that picotls at the moment provides access to CTR crypto beneath the AEAD interface, which is to be used by the QUIC stacks.

I would assume that we can cut down the overhead to somewhere between 2% to 4%, but it might be hard to go down to somewhere near 1%, because we cannot parallelize the AES operation of PNE with that of AEAD (see https://github.com/openssl/openssl/blob/OpenSSL_1_1_0h/crypto/aes/asm/aesni-x86_64.pl#L24-L39<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenssl%2Fopenssl%2Fblob%2FOpenSSL_1_1_0h%2Fcrypto%2Faes%2Fasm%2Faesni-x86_64.pl%23L24-L39&data=02%7C01%7Cnibanks%40microsoft.com%7C11d55f17333e4a795d7008d5d7e6d93c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636652308843994134&sdata=kqMz4SsN%2F2ErGK06Qz8Z0vUzpl4MiipnNE2wAMUb46c%3D&reserved=0> about the impact of parallelization).

I do not think that 2% to 4% of additional overhead to the crypto is an issue for QUIC/HTTP, but current overhead of 10% is something that we might want to decrease. I am glad to be able to learn that now.


2018-06-22 5:48 GMT+09:00 Nick Banks <nibanks=[e-mail]<mailto:nibanks=[e-mail]>>:
Hello QUIC WG,

I recently implemented PNE for WinQuic (using bcrypt APIs) and I decided to get some performance numbers to see what the overhead of PNE was. I figured the rest of the WG might be interested.

My test just encrypts the same buffer (size dependent on the test case) 10,000,000 times and measured the time it took. The test then did the same thing, but also encrypted the packet number as well. I ran all that 10 times in total. I then collected the best times for each category to produce the following graphs and tables (full excel doc attached):

[cid:image003.png@01D40966.7655B6B0]


Time (ms)

Rate (Mbps)

Bytes

NO PNE

PNE

PNE Overhead

No PNE

PNE

4

2284.671

3027.657

33%

140.064

105.692

16

2102.402

2828.204

35%

608.827

452.584

64

2198.883

2907.577

32%

2328.45

1760.92

256

2758.3

3490.28

27%

7424.86

5867.72

600

4669.283

5424.539

16%

10280

8848.68

1000

6130.139

6907.805

13%

13050.3

11581.1

1200

6458.679

7229.672

12%

14863.7

13278.6

1450

7876.312

8670.16

10%

14727.7

13379.2


I used a server grade lab machine I had at my disposal, running the latest Windows 10 Server DataCenter build. Again, these numbers are for crypto only. No QUIC or UDP is included.

Thanks,
- Nick




--
Kazuho Oku



--
Kazuho Oku



--
Kazuho Oku