Re: [IPsec] Some thoughts regarging draft-hopps-ipsecme-iptfs-01

[Steffen Klassert <[e-mail]>]

On Mon, Dec 02, 2019 at 10:57:59AM -0500, Christian Hopps wrote:
> > On Dec 2, 2019, at 9:11 AM, Steffen Klassert <[e-mail]> wrote:
> > On Mon, Dec 02, 2019 at 06:22:26AM -0500, Christian Hopps wrote:
> >>> On Dec 2, 2019, at 3:01 AM, Steffen Klassert <[e-mail]> wrote:
> >>> On Thu, Nov 28, 2019 at 04:49:36PM +0300, Valery Smyslov wrote:
> >>> 
> >>>> 4. I'd like to see more text in the draft regarding reassembling of incoming packets.
> >>> 
> >>> Yes, I think some words on how to reassemble the fragments are really
> >>> needed.
> >>> 
> >>>>   It seems to me that it can be done pretty easy by linking the reassembly logic
> >>>>   with replay protection window.
> >>> 
> >>> While it looks like doing the reassembling based on ESP sequence numbers
> >>> might be an easy approach, it could be also dangerous.
> >>> 
> >>> Consider a system that encapsulates two flows on different cpus
> >>> with the same SA. This system can TX packets in the following
> >>> order:
> >>> 
> >>> TX cpu0 inner flow0 SA0:
> >>> 
> >>>     Offset: 0                               Offset: 100        
> >>>     [ ESP1  (1500) ]                        [ ESP3  (1500) ]  
> >>>     [--800--][--800-                        -][-----1400---] 
> >>> 
> >>> --------------------------------------------------------------------------------------
> >>> TX cpu1 inner flow1 SA0:
> >>>                         Offset: 0                                Offset: 100   
> >>>                         [ ESP2  (1500) ]                        [ ESP4  (1500) ]
> >>>                         [--800--][--800-                        -][----1400----]    
> >>> 
> >>> 
> >>> On the receive side, it is not that clear how to reassemble the fragments
> >>> from ESP3 and ESP4 into the fragments from ESP1 and ESP2. Maybe some
> >>> packet ID in the IP-TFS header could help to identify related fragments.
> >> 
> >> Indeed the code mustn't fragment this way. :)
> >> 
> >> We could add a bit of text that one should avoid this mistake.
> > 
> > I'm not so sure whether the receiver should rely on that
> > the sender did the fragmentation right. I think a packet
> > ID, like IP fragments have, would just solve the problem.
> > The implementation would not even need to care about this
> > multicore race then.
> 
> The receiver can do any number of wrong things with what it sends, but I'd normally call those bugs. :)

Yes, that's true. But if the protocol allows to do things wrong, it is
a bug in the protocol :)

Maybe you can just make it clear at the sender side by saying something
like 'Fragments must be sent ordered and ESP encapsulated with consecutive
sequence numbers.'

> 
> Technically though, attaching a packet ID to the fragments to allowing sending them in any order saves only a little on code complexity (i.e., not using an ordering queue) on the sender side;

I hoped to avoid such an ordering/serialization queue as I fear
this will become a bottleneck. With the current design, I don't
see how to do this without a queue. I know, it is an implementation
detail, but implementation matters too :)

> however, it seems to add a disproportionate amount of complexity to the receiver/reassembly (which could e.g., be aggregating VPN server).

Yes, if you know that the next fragment comes with the next ESP
packet, things are much easier. So we have the complexity either
at the sender or the receiver side, not so sure what performs better.

> Adding a packet ID also means that you can't just chain the inner traffic buffers together to form the IP-TFS payload as you must now insert an extra header between each of the inner packets, this is going to affect performance and memory use on whitebox/software based deployments as well as reduce available bandwidth on the tunnel.

Good point. You can always chain extra headers and inner packets with
a scatter-gather list, but it will have some performance impact.

> I think we should try and keep fragmentation and reassembly as simple as possible so that it is easy to implement and get right.

I absolutely agree here.

> Having coded this using just the ESP sequence numbers to correct-order the received packets, I can say it's an easy-to-moderate complex function that performs well, it took a few iterations on the code to get it right and well tested. Adding another level to this receive complexity should only be done if we absolutely have to. I don't think we are there yet, given that the use of a simple ordering queue on the sender side is all we are talking about.
> 
> Again it's easy to add some text to the document to highlight what needs to be paid attention to if one is using multiple cores to send on a single IPsec SA.

As said above, just make it clear how the sender must order the
fragments if you don't want a packet ID.

Steffen