Re: UDP source ports for HTTP/3 and QUIC

[Willy Tarreau <[e-mail]>]

On Fri, Jul 16, 2021 at 06:34:32AM +0000, Poul-Henning Kamp wrote:
> --------
> Willy Tarreau writes:
> 
> > Stefan made a good point about the problem that might result, with
> > inbound load balancing between multiple listeners (typically what's
> > achieved by L3 switches doing L3+L4 hash between multiple servers,
> > and operating systems hashing the source+destination port to pick a
> > different listening socket). Thus a suggestion might be to possibly
> > save resources by using a small amount of sockets, with "small" left
> > to the appreciation of the implementation.
> 
> We should run the question "few or many UDP ports?" past some some
> friendly 100G and 400G device driver maintainers.
> 
> The NICs I have been working with for the ESO ELT project all
> included UDP ports in the hash they used to decide which CPU core
> to deliver packets/interrupts to and we had to spread across UDP
> ports to or all the traffic would hit one single core.
> 
> I dont know if QUIC has registered with the NIC designers and device
> drivers writers yet, but given the opacity of QUIC packets, it is
> very hard to see what else than the UDP port they can feed into
> their hash, so I expect the answer to be "many".

That's true but queues are not the only parameter, as using many
sockets also results in a massive cache miss ratio and many more
syscalls. Even with TCP it's quite visible that using a few hundred
connections tends to be incredibly more efficient at 100Gbps than
using tens of thousands.

Ideally we want all queues to be used with a good balance while
limiting the amount of operations to transport these data.

Probably that some wording evocating that possibility offered by
the protocol and a suggestion to take various factors into account
(L4 LB, NIC queues, listeners, syscalls etc) is sufficient to let
implementors ask the relevant people for advice in their context.

Willy