Re: [Masque] Encrypting between client and proxy

[Eric Rosenberg <[e-mail]>]

The two main advantages from forwarding mode are avoiding cumulative MTU loss and making it easy for the proxy to skip parts of the send and receive path. If we can employ per-hop encryption for forwarded mode packets in a way that doesn’t suffer from cumulative MTU loss, that sounds like a pretty compelling way to solve the “I want to chain many proxies” problem while removing the “observers on both sides of the proxy” caveat that’s included in the current draft’s security considerations.

Assuming encryption can be done in a way that doesn’t incur cumulative MTU loss, it’s worth considering how it affects the efficiency of implementations. Taking our current implementation as an example, we’ve opted to perform all forwarding at Linux’s XDP hook with an eBPF program. This allows us to run a custom program for each packet we receive. In this program, we only actually need to process up to the QUIC packet header in order to perform a rewrite and immediately send the packet out. How this program is actually executed is dependent on the network card and driver, but it can avoid copies, be executed before the Linux networking stack, and even be completely offloaded to the NIC. The numbers presented were not from full NIC offload, but rather “Native XDP” where the program runs on the main CPU, but may take advantage of direct memory access, avoid copies, etc.). Continuing with our implementation as an example, introducing encryption would likely require APIs (bpf helper functions) that aren’t available from mainline Linux kernels. We would likely have to patch our kernel to expose cryptographic operations to eBPF programs or we’d have to bring our forwarding implementation into userspace - both of which seem possible, but not something we’ve explored and it’s hard to say what the impact on performance/efficiency would be without testing. When considering full NIC offload, I wonder if network cards are less likely to expose cryptographic operations than they are to expose the very basic packet inspection and byte replacement for header rewriting required by the non-encrypting approach.

Thanks,
Eric

> On Nov 16, 2022, at 2:18 AM, Alex Chernyakhovsky <achernya=[e-mail]> wrote:
> 
> 
> 
> On Tue, Nov 15, 2022 at 8:57 PM Ben Schwartz <bemasc=[e-mail] <mailto:[e-mail]>> wrote:
> On Tue, Nov 15, 2022 at 6:44 PM Martin Thomson <[e-mail] <mailto:[e-mail]>> wrote:
> ...
> The protected payload is high entropy, being comprised of ciphertext and is anywhere from 20 to 65506 bytes.  This payload can be split into two, with a 12 byte sample being used as a nonce for AES-CTR.  The remainder of the payload, plus part of the first byte, could then be protecting using AES-CTR.  The remaining 8+ bytes could again be sampled again as a nonce to protect the first sample.
> 
> The thing you are looking for is called a Pseudo-Random Permutation [1].  I would encourage you to use HCTR2 [2].  (I learned a bit about these while working on [3].)
> 
> [1] https://en.wikipedia.org/wiki/Pseudorandom_permutation <https://en.wikipedia.org/wiki/Pseudorandom_permutation>
> [2] https://github.com/google/hctr2 <https://github.com/google/hctr2>
> [3] https://datatracker.ietf.org/doc/draft-cpbs-pseudorandom-ctls/ <https://datatracker.ietf.org/doc/draft-cpbs-pseudorandom-ctls/>
> 
> ...
> [1] A few people have noted that timing tends to be a dead giveaway here, as does packet size.  I have some ideas about how that might be managed, but let's start with the basics.
> I think enough of our threat model has to consider side-channels like packet size as out of scope (although I believe we do have the opportunity to add chaff to less-than-MTU packets); we can certainly do something about timing analysis. I would expect most interesting deployments of such a proxy to be very high throughput which would mean some amount of artificial queuing and mixing would go a long way towards increasing the search space for correlation.
> 
> Personally, I think this is probably fatal, so the encryption is not really buying you anything.   However, I would still support adding encryption here, at least optionally, for the sake of making the attacker's job a bit harder.  It's not the easiest thing to specify in a threat model, but there is a real difference between "instant undeniable confirmation" and "strong statistical inference" for an attacker.
> I am similarly in favor of trying to protect the packet further, but my understanding from the presentation at the session was that a lot of the CPU wins came from NIC offloads. If we end up hitting the host CPU to do the decryption, we should compare the implementation to the full CONNECT-UDP tunneling mode and see if it's worthwhile. 
> 
> ...
> -- 
> Masque mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/masque <https://www.ietf.org/mailman/listinfo/masque>
> -- 
> Masque mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/masque