Re: [IPsec] Discussion of draft-pwouters-ipsecme-multi-sa-performance

[Valery Smyslov <[e-mail]>]

Hi Steffen,

> Hi Valery,
> 
> > Then my next question is - how the sending side decides
> > whether to one of use per-CPU SAs or the fallback SA?
> > My guess that the packet is handled by some kernel thread
> > (i.e. by some CPU), so once this CPU figures out that
> > it doesn't have an SA - I assume it uses the fallback SA then.
> > Is it right?
> 
> Right.
> 
> > If so, then why it cannot hand over the packet
> > to the CPU that for sure has the needed SA (this CPU can
> > be indicated in the stub SA entry)?
> 
> Moving in flight packets to a different cpu is always a pain.

I understand.

> From an implementation point of view, that should be avoided
> whenever possible. Aside from the overhead it creates, we
> also have to care about corener cases. For example, how
> do you make sure that the SA on this remote cpu is still
> there when the packet arrives on it?

Then it should be re-steered to yet another CPU.
Fortunately this should happen infrequently.

> Also, where would that stub SA entry be located, in a percpu
> or in a global SAD? If it is an a percpu SAD, then remote
> cpus must update all stub SA entries when a percpu SA
> comes up or goes down. So we would need to lock the
> percpu SADs. If it is in a global SAD, then why not just
> negotiate keymat and use it (as a fallback)?

In a per-CPU SAD. Updating is costly, I agree, but should happen 
relatively infrequently. There may also be some optimization.
For example, when new per-CPU SA appears, you don't have
to update all remote stubs (let them continue to point to another existing SA), 
you only need to do it if a per-CPU SA is deleted.

> Instead of having a stub SA entry, I could think of encoding
> the information which cpu has a valid SA to the policy. The
> policy is global anyway. But then we still need to re-steer
> the packets, what I really dislike.

How much costly is re-steering comparing to a locking 
needed for a global fallback SA?

> Is it just that you don't like that the fallback SA MUST (or
> maybe SHOULD) be always up, or is it that you don't want to
> negotiate this additional SA at all?

The former. I'd like to avoid SAs that need special treatment,
whenever possible.

> Another possibility would be to use the same keymat on all
> percpu SAs, as it was proposed at the discussion we had
> at our 'IPsec coffee hour' last time. In that case you
> have a valid SA on all cpus with a single negotiation.
> But hat would require a change to ESP what this proposal
> don't need.

This would bring other difficulties. For example, if 
the amount of data that can be safely processed with
a single key is limited (as it should be), then you
need to count the sum of the traffic on all such SAs,
that would require some kind of locking.
Or you should use significantly smaller value
to be sure that traffic never reaches the threshold.

> > In both cases some
> > locking is required - does the latter case require much more locking?
> 
> The percpu SAs don't need locking as long as you can make sure that
> it is never ever accessed by a remote cpu. To guarantee this, something
> that does the 'dirt work' is needed. In our case that would be the
> fallback SA.

Then how per-SAs are installed? Doesn't it require some locking?

Regards,
Valery.

> Steffen