RE: Impact of hardware offloads on network stack performance

["Deval, Manasi" <[e-mail]>]

The current socket scheme uses the kernel stack to implement the Ethernet – IP – UDP stack processing for QUIC packets. The AF_XDP provides a raw frame so one would need a policy on where to run the protocol stack processing.

Another comment on offloads -
Thanks for the insightful point of how crypto offload enables other offloads like segmentation. A segmentation offload for QUIC would need to increment packet numbers for each of the individual packets put on the wire. The transmitted packet numbers for a transmit scatter gather list would be deterministic. Therefore, this should not adversely impact any retransmits.

Thanks,
Manasi


From: craigt [mailto:[e-mail]]
Sent: Thursday, May 10, 2018 5:50 AM
To: Roberto Peon <[e-mail]>
Cc: Watson Ladd <[e-mail]>; Mike Bishop <[e-mail]>; Praveen Balasubramanian <[e-mail]>; Lubashev, Igor <[e-mail]>; Ian Swett <ianswett=[e-mail]>; Mikkel Fahnøe Jørgensen <[e-mail]>; IETF QUIC WG <[e-mail]>; Deval, Manasi <[e-mail]>
Subject: Re: Impact of hardware offloads on network stack performance

Has anyone experimented with the zerocopy socket interfaces for exactly this use case?
  https://netdevconf.org/2.1/papers/netdev.pdf

It would seem that is a more natural fit to the assertions made by several of the authors surrounding packetisation.
I'm unaware if Windows/BSD have similar interfaces/implementations.

c


On 30 April 2018 at 21:37, Roberto Peon <[e-mail]<mailto:[e-mail]>> wrote:
QUIC (mostly) doesn’t do that (rexmit a packet verbatim)—you’d need a new packet# for the rexmitted packet.
-=R

From: QUIC <[e-mail]<mailto:[e-mail]>> on behalf of Watson Ladd <[e-mail]<mailto:[e-mail]>>
Date: Monday, April 23, 2018 at 1:56 PM
To: Mike Bishop <[e-mail]<mailto:[e-mail]>>
Cc: Praveen Balasubramanian <[e-mail]<mailto:[e-mail]>>, "Lubashev, Igor" <[e-mail]<mailto:[e-mail]>>, Ian Swett <ianswett=[e-mail]<mailto:[e-mail]>>, Mikkel Fahnøe Jørgensen <[e-mail]<mailto:[e-mail]>>, IETF QUIC WG <[e-mail]<mailto:[e-mail]>>, "Deval, Manasi" <[e-mail]<mailto:[e-mail]>>
Subject: Re: Impact of hardware offloads on network stack performance



On Mon, Apr 23, 2018 at 1:06 PM, Mike Bishop <[e-mail]<mailto:[e-mail]>> wrote:
I agree – the point of offloading the crypto is not only to spare that cost on the CPU, though that can be useful, it’s to enable further offloads that can’t be done on the encrypted packet.

LRO that could simply group packets by CID and deliver a single batch of packets from a single CID in a single invocation seems like a great first step, and doesn’t even require crypto offload.  Once you have crypto offload, of course those packets per CID could be delivered already decrypted.  I’d want to see data on whether there’s value in making the received data look like one huge packet when there’s not a single bytestream, but that’s not something we need to specify here.

LSO is more interesting, and the challenge there is that the transport stack still needs to know what data was in which packet so that it can do loss recovery properly.  QUIC explicitly disallows fragmentation of the UDP datagram, for the “painful and potentially non-deterministic” reasons you referenced.

Forgive my complete ignorance, but doesn't it just need to know what packet it sent?

More explicitly the sender would maintain a queue of sendable packets, and mark which have been sent. Acknowledged packets are removed from the queue. In event of loss, they simply retransmit the same packet. It's up to higher layers to pick the right size of packet and take data from the individual QUIC streams. If you wanted to expose the stream structure, that sounds a lot more complicated.

Sincerely,
Watson