Re: Impact of hardware offloads on network stack performance

[Mikkel Fahnøe Jørgensen <[e-mail]>]

Yes, it is touching the memory that is the big problem.

24ns is measurable time in my world where a 500 byte structured message is
easily handled, but in reality it is the same latency as a 7 meter longer
ethernet cable due to c (the speed of light).

CPU cores work by cache coherency and prefetching. As long as data is only
written once, data can funnel into the different CPU cache systems but if
data is later touched the message bus between CPU’s must invalidate a lot
of assumptions and start over.

In the current form of PR 1079, the packet number is encrypted by writing
over data in the buffer, so in order to decrypt it, or just to verify it,
it is necessary to either copy the buffer - which means that all other
interested CPU’s must fetch new data, or the data is modified meaning cache
invalidation.

Fortunately this part of the problem is not so hard to solve: remove the
packet number from the AEAD protected data - there is no benefit in having
it there. Store and encrypt it separately. This is a problem because an AES
block is 16 bytes and smaller encryption schemes are hard to make secure.
By overwriting packet data PR 1079 places some of block size over existing
non-packet number data which saves space but hurts cache.

My unverified assumption is that this can be improved: the packet number
can be encrypted indirectly by first encrypting arbitrary content (say the
tag) of the existing already encrypted packet. The encrypted data is NOT
stored  but xor’ed with the packet number. The packet number is now
appended after the tag, or anywhere else, just not inside AEAD data.

So how to decrypt? When the packet is received you encrypt the same data as
before and get 16 bytes out. You locate the encrypted packet number and xor
to get the clear text of the packet number.

Now you can start to decrypt and/or verify the AEAD packet content because
the decrypted packet number is the IV used in that crypto process. Notice
that in this process we only operate with a 16-byte temporary block that we
do NOT store in he packet buffer. We get a decoded packet number that we do
NOT store in the packet buffer. We can store it anywhere else, as long as
it is in a memory area with its own cache line boundary.

This is still not ideal because the latency is not only the 24ns decryption
but also writing the decrypted packet number somewhere and communicate it
to a process that operate on packet data. For example, if we have two cores
- one for verification, and one for decryption, then one must necessarily
send data to the other, or they most both do the same packet number
decryption (which probably is the better idea, but now it costs 48ns total).



On 6 April 2018 at 22.55.33, Ian Swett ([e-mail]) wrote:

PNE doesn't preclude UDP LRO or pacing offload.

However, in a highly optimized stack(ie: The Disk|Crypt|Net paper from
SIGCOMM 2017) not touching(ie: bringing into memory or cache) the content
to be sent until the last possible moment is critical.  PNE likely means
touching that content much earlier.

But personally I consider the actual crypto we're talking about(2 AES
instructions?) adding basically free.

On Fri, Apr 6, 2018 at 4:37 PM Mike Bishop <[e-mail]> wrote:

> Thanks for the clarification.  I guess what I’m really getting down to is
> how do we quantify the actual *cost* of PNE?  Are we substantially
> increasing the crypto costs?  If Ian is saying that crypto is comparatively
> cheap and the cost is that it’s harder to offload something that’s
> comparatively cheap, what have we lost?  I’d think we want to offload the
> most intensive piece we can, and it seems like we’re talking about crypto
> offloads....  Or are we saying instead that the crypto makes it harder to
> offload other things in the future, like a QUIC equivalent to LRO/LSO?
>
>
>
> *From:* Mikkel Fahnøe Jørgensen [mailto:[e-mail]]
> *Sent:* Thursday, April 5, 2018 9:45 PM
> *To:* Ian Swett <[e-mail]>; Mike Bishop <[e-mail]>
> *Cc:* Praveen Balasubramanian <[e-mail]>; IETF QUIC WG <
> [e-mail]>
> *Subject:* Re: Impact of hardware offloads on network stack performance
>
>
>
> To be clear - I don’t think crypto is overshadowing other issues as Mike
> read my post. It certainly comes at a cost, but either multiple cores or
> co-processors will deal with this. 1000ns is 10Gbps crypto speed on one
> core and it is higly parallelisable and cache friendly.
>
>
>
> But if you have to drip your packets through a traditional send interface
> that is copying, buffering or blocking, and certainly sync’ing with the
> kernel - it is going to be tough.
>
>
>
> For receive you risk high latency or top much scheduling in receive
> buffers.
>
>
> ------------------------------
>
> *From:* Ian Swett <[e-mail]>
> *Sent:* Friday, April 6, 2018 4:06:01 AM
> *To:* Mike Bishop
> *Cc:* Mikkel Fahnøe Jørgensen; Praveen Balasubramanian; IETF QUIC WG
> *Subject:* Re: Impact of hardware offloads on network stack performance
>
>
>
>
>
> On Thu, Apr 5, 2018 at 5:57 PM Mike Bishop <[e-mail]> wrote:
>
> That’s interesting data, Praveen – thanks.  There is one caveat there,
> which is that (IIUC, on current hardware) you can’t do packet pacing with
> LSO, and so my understanding is that even for TCP many providers disable
> this in pursuit of better egress management.  So what we’re really saying
> is that:
>
>    - On current hardware and OS setups, UDP runs less than half as much
>    throughput as TCP; that needs some optimization, as we’ve already discussed
>    - TCP will dramatically gain performance once hardware offload catches
>    up and allows *paced* LSO 😊
>
>
>
> However, as Mikkel points out, the crypto costs are likely to overwhelm
> the OS throughput / scheduling issues in real deployments.  So I think the
> other relevant piece to understanding the cost here is this:
>
>
>
> I have a few cases(both client and server) where my UDP send costs are
> more than 30%(in some cases 50%) of CPU consumption, and crypto is less
> than 10%.  So currently, I assume crypto is cheap(especially AES-GCM when
> hardware acceleration is available) and egress is expensive.  UDP ingres is
> not that expensive, but could use a bit of optimization as well.
>
>
>
> The only 'fancy' thing our current code is doing for crypto is encrypting
> in place, which was a ~2% win.  Nice, but not transformative.  See
> EncryptInPlace
> <https://cs.chromium.org/chromium/src/net/quic/core/quic_framer.cc?sq=package:chromium&l=1893>
>
>
>
> I haven't done much work benchmarking Windows, so possibly the Windows UDP
> stack is really fast, and so crypto seems really slow in comparison?
>
>
>
>
>    - What is the throughput and relative cost of TLS/TCP versus QUIC
>    (i.e. how much are the smaller units of encryption hurting us versus, say,
>    16KB TLS records)?
>
>
>    - TLS implementations already vary here:  Some implementations choose
>    large record sizes, some vary record sizes to reduce delay / HoLB, so this
>    probably isn’t a single number.
>
>
>    - How much additional load does PNE add to this difference?
>    - To what extent would PNE make future crypto offloads *impossible*
>    versus *requires more R&D to develop*?
>
>
>
> *From:* QUIC [mailto:[e-mail]] *On Behalf Of* Mikkel Fahnøe
> Jørgensen
> *Sent:* Wednesday, April 4, 2018 1:34 PM
> *To:* Praveen Balasubramanian <pravb=[e-mail]>;
> IETF QUIC WG <[e-mail]>
> *Subject:* Re: Impact of hardware offloads on network stack performance
>
>
>
> Thanks for sharing these numbers.
>
>
>
> My guess is that these offloads deal with kernel scheduling, memory cache
> issues, and interrupt scheduling.
>
>
>
> It probably has very little to do with crypto, TCP headers, and any other
> cpu sensitive processing.
>
>
>
> This is where netmap enters: you can transparently feed the data as it
> becomes available with very little sync work and you can also efficiently
> pipeline so you pass data to decryptor, then to app, with as little bus
> traffic as possible. No need to copy data or synchronize kernel space.
>
>
>
> For 1K packets you would use about 1000ns (3 cycles/byte) on crypto and
> this would happen in L1 cache. It would of course consume a core which a
> crypto offload would not, but that can be debated because with 18 core
> systems, your problem is memory and network more than CPU.
>
>
>
> My concern with PNE is for small packets and low latency where
>
> 1) an estimated 24ns for PN encryption and decryption becomes measurable
> if your network is fast enough.
>
> 2) any damage to the packet buffer cause all sorts of memory and cpu bus
> traffic issues.
>
>
>
> 1) is annoying. 2) is bad, completely avoidable but not as PR 1079 is
> formulated currently.
>
>
>
> 2) is also likely bad for hardware offload units as well.
>
>
>
> As to SRV-IO - I’m not familiar with it, but obviously there is some IO
> abstraction layer - the question is how you make it accessible to apps as
> opposed to device drivers that does nor work with you QUIC custom stack,
> and netmap is one option here.
>
>
>
> Kind Regards,
>
> Mikkel Fahnøe Jørgensen
>
>
>
> On 4 April 2018 at 22.15.52, Praveen Balasubramanian (
> pravb=[e-mail]) wrote:
>
> Some comparative numbers from an out of box default settings Windows
> Server 2016 (released version) for a single connection with a
> microbenchmark tool:
>
>
>
> *Offloads enabled*
>
> *TCP gbps*
>
> *UDP gbps*
>
> *LSO + LRO + checksum*
>
> 24
>
> 3.6
>
> *Checksum only*
>
> 7.6
>
> 3.6
>
> *None*
>
> 5.6
>
> 2.3
>
>
>
> This is for a fully bottlenecked CPU core -- *if you run lower data rates
> there is still a significant difference in Cycles/byte cost*. Same
> increased CPU cost applies for client systems going over high data rate
> Wifi and cellular.
>
>
>
> This is without any crypto. Once you add crypto the numbers become much
> worse *with crypto cost becoming dominant*. Adding another crypto step
> further exacerbates the problem. Hence crypto offload gains in importance
> followed by these batch offloads.
>
>
>
> If folks need any more numbers I’d be happy to provide them.
>
>
>
> Thanks
>
>
>
>