[IPsec] NO_PROPOSAL_CHOSEN vs INVALID_SYNTAX

[Paul Wouters <[e-mail]>]

Hi,

We are having a discussion about which notify to return in certain
cases. The issue comes down to the names of the notifies and their
actual dictated use in the RFC that does not always intuitively
maps to the name.

NO_PROPOSAL_CHOSEN can be interpreted as "no proposal from the IKE/IPsec
proposal list matches due to all proposals having at least one mismatching
transform" versus "no matching ike connection for your IKE proposal"
where proposal refers to the entire IKE proposal, not the proposals
list with transforms.

INVALID_SYNTAX can be interpreted as "malformed packet" but the RFC text
uses this as the "if all other errors dont match, use this one" so you
can end up returning this even if there is no invalid syntax at all.

So if your IPsec gateway only has static IP based VPNs and an unknown IP
connects, some feel NO_PROPOSAL_CHOSEN conveys that, while technically,
even though there is no invalid syntax in that proposal, the RFC states
we should return INVALID_SYNTAX.

Similarly, if during IKE_AUTH you are finding out there is no IPsec
configuration that matches the incoming client, there is no "proposal
list" to compare, so while NO_PROPOSAL_CHOSEN feels a more natural
match, should we really return INVALID_SYNTAX despite there being no
syntax problem? That is what the RFC says.

I guess in the end, we are really missing a "CONNECTION_REJECTED"
notify that would cover all the things not covered in the more specific
notifies.

What do other implementations do? Should we clarify this anywhere?

libreswan was using NO_PROPOSAL_CHOSEN for most of these, but is now
slated to be more strict to the RFC and use INVALID_SYNTAX. (and
clearly, I'm not happy about it but it seems the RFC dictates this)

Paul