Re: [IPsec] Discussion of draft-pwouters-ipsecme-multi-sa-performance

[Steffen Klassert <[e-mail]>]

Hi Valery,

On Mon, Oct 17, 2022 at 05:10:32PM +0300, Valery Smyslov wrote:
> > >
> > > > I could guess that the fallback SA *does* require locks.
> > >
> > > It also seems to me. So I see no difference if the packet
> > > can be re-steered to a different CPU, in any case we'll have
> > > performance penalty.
> > 
> > The fallback SA needs locking, as it can be used from any cpu.
> > But with the current approch, this is the only one that needs
> > locking.
> 
> Then my next question is - how the sending side decides
> whether to one of use per-CPU SAs or the fallback SA?
> My guess that the packet is handled by some kernel thread
> (i.e. by some CPU), so once this CPU figures out that 
> it doesn't have an SA - I assume it uses the fallback SA then.
> Is it right?

Right.

> If so, then why it cannot hand over the packet
> to the CPU that for sure has the needed SA (this CPU can 
> be indicated in the stub SA entry)?

Moving in flight packets to a different cpu is always a pain.
>From an implementation point of view, that should be avoided
whenever possible. Aside from the overhead it creates, we
also have to care about corener cases. For example, how
do you make sure that the SA on this remote cpu is still
there when the packet arrives on it?

Also, where would that stub SA entry be located, in a percpu
or in a global SAD? If it is an a percpu SAD, then remote
cpus must update all stub SA entries when a percpu SA
comes up or goes down. So we would need to lock the
percpu SADs. If it is in a global SAD, then why not just
negotiate keymat and use it (as a fallback)?

Instead of having a stub SA entry, I could think of encoding
the information which cpu has a valid SA to the policy. The
policy is global anyway. But then we still need to re-steer
the packets, what I really dislike.

Is it just that you don't like that the fallback SA MUST (or
maybe SHOULD) be always up, or is it that you don't want to
negotiate this additional SA at all?

Another possibility would be to use the same keymat on all
percpu SAs, as it was proposed at the discussion we had
at our 'IPsec coffee hour' last time. In that case you
have a valid SA on all cpus with a single negotiation.
But hat would require a change to ESP what this proposal
don't need.

> In both cases some
> locking is required - does the latter case require much more locking?

The percpu SAs don't need locking as long as you can make sure that
it is never ever accessed by a remote cpu. To guarantee this, something
that does the 'dirt work' is needed. In our case that would be the
fallback SA.

Steffen