RE: Never fragment: getting PMTU info transmitted reliably

["Lubashev, Igor" <[e-mail]>]

On Thursday, January 17, 2019 10:12 AM Tom Herbert <[e-mail]> wrote:

> On Wed, Jan 16, 2019, 9:48 PM Lubashev, Igor <mailto:[e-mail] wrote:
>> There is a lot of complexity in injecting entropy into IP addresses. As
>> long as using a 4-tupple for TCP and UDP traffic, 2-tupple+SPI for IPSec,
>> 2-tupple+Key for GRE, etc. "just works", there is little incentive to
>> deploy complex solutions that involve DNS, neighbour caches, etc. The hope
>> for new protocols on top of ipv6 is the flow label, because it is so simple
>> to use right by everyone: the sender, the receiver, and the middle boxes.
> 
> Igor,
> 
> I'm not sure I'd say that it "just works". Take a look at Maglev load
> balancer
> (https://urldefense.proofpoint.com/v2/url?u=https-3A__static.googleusercontent.com_media_research.google.com_en__pubs_archive_44824.pdf&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=Ov_Q64EPg0L4YTIJM46mUQ1XCRAUa2erO899x4BVQK8&s=HmQ5z09TFfKB0FmiHCgAgj6I_xnkpP6de0-529U28kw&e=)
> and all the complexity needed to ensure consistent routing to backends, and
> even with all that it still will drop some number of connections when
> backends change.

I am familiar with Maglev.  In fact, I run a load balancer system whose backend pool changes every few seconds.  To preserve connections, you either needs to do some clever tricks and/or keep per-connection state (our load balancer for TCP is mostly stateless).  It would not matter what part of the packet contains your load balancing connection key: IP, ports, QUIC CID, IPSec SPI, etc.  You either find a way for the server to pass a token to the client to include in each of their packets or you keep a per-connection state on the load balancer.

Moreover, I would argue that any method that has the client select a load balancing connection key (an ephemeral src port or less significant bits of IP address) is going to require load balancer to keep per-connection state or drop some connections on a backend pool change.  This is exactly why IETF QUIC has packets from clients feature a server-supplied Connection ID (after handshake completion).


> Using flow label or transport layer information is only best effort for
> persistence. Intermediate devices are allowed to change devices flow labels,
> NAT evictions of UDP may change the client side ports and address
> mid-flow.

You can also add IP fragmentation to the list.  NAT rebinding is a potential problem for UDP-based protocols, indeed (hence QUIC is trying to mitigate it).

I do not think intermediary devices are allowed to change flow labels, though, since RFC 6437 forbids this: "Once set to a non-zero value, the Flow Label is expected to be delivered unchanged to the destination node(s).  A forwarding node MUST either leave a non-zero flow label value unchanged or change it only for compelling operational security reasons as described in Section 6.1." (Off topic: "Section 6.1" makes me amused that it supposes that networks worried about covert exfiltration of information by the sender will allow packet transmissions by that sender sans the flow label.)


> QUIC is interesting since it doesn't use the IP addresses or ports
> for identifying connections.

QUIC allows either endpoint to provide a 0-length connection ID, effectively using IPs and ports for connection identification.  I believe Chromium does (did?) just that.  Moreover, a change in a 4-tuple during QUC handshake is likely to kill that handshake.


> That conceptually allows a client to purposely
> randomize both the source address and the port on every packet

That would not work too well for QUIC (at least the current version), since QUIC requires path validation (QUIC does not want to be a vector for reflection attacks).  The exact details of connection migration and validation are not completely settled, yet, but what is certain is that while frequent connection migration should work, it would need to be much less frequent than every packet.

- Igor