Re: [v6ops] IPv6 MTU Flow-label.... (related to draft-v6ops-pmtud-ecmp-problem-01)

[Mark ZZZ Smith <[e-mail]>]

>________________________________
> From: Jeroen Massar <[e-mail]>
>To: Brian E Carpenter <[e-mail]> 
>Cc: IPv6 Operations <[e-mail]>; 6man <[e-mail]> 
>Sent: Tuesday, 11 November 2014, 6:02
>Subject: Re: [v6ops] IPv6 MTU Flow-label.... (related to draft-v6ops-pmtud-ecmp-problem-01)
> 
>
>On 2014-11-10 19:57, Brian E Carpenter wrote:
>> Er, no, you really can't use the flow label like that. RFC6437.
>
>Hence, redefining the bits :) Few implementations use them anyway, thus
>we still have time to actually make good use of them.
>

So the "flow label" then field name wouldn't then be accurate, because an MTU value doesn't identify or contribute to the identification or labelling of a flow very well. There's a lot of troubleshooting and other value in "truth in advertising", or less cliché'd "accuracy in naming". Encoding non-flow labelling MTU information in that existing flow labelling field would therefore be a hack (MTU might be an attribute of a flow, but it isn't very specific or unique to a flow) for this one special case of stateless ECMP based load balancing. It wouldn't be useful in most or perhaps all other IP use cases. (So if this idea gained more traction, I'd start lobbying for the field to be renamed.)

It seems to me that the fundamental issue is that given how the IP protocols work (as this problem also occurs in IPv4), stateless ECMP isn't a very effective method of performing load balancing. While probably chosen for pure performance/throughput, it isn't distributing load to the LB hosts based on their current load (so it isn't actually 'balancing'), and because it isn't maintaining per-host state, isn't able to perform per-host specific processing/forwarding, which is what is needed for PMTUD to function in this scenario.

This/these methods of load balancing have generally annoyed me for a while, primarily because I have also had to troubleshoot or deal with these sorts of issues in the past. I think the fundamental issue is that to the network, the unicast LB address is looks like a single unicast destination/host, where as in reality is being shared by a number of hosts. While that might sound like the anycast problem, I think the difference is that even though with anycast multiple addresses are shared between a set of hosts, for any particular source host there is no variation in which of the anycast hosts is used over time, unless it fails (i.e., the routing system would normally consistently pick one of the anycast destinations for a particular source, as would IPv6's 'on-link' anycast method (see RFC7094 for more discussion)). In this LB case the destination LB host changes based on changes in TCP or UDP ports, which is occurring all of the time.

If high performance stateless and therefore load insensitive LB distribution is needed or wanted, then perhaps something more simple that doesn't use TCP or UDP port information might be better. For example, using SA+DA based forwarding in the LB, and then statically mapping portions of the SA address space to a particular LB host (e.g., 1/4 to LB host a, 1/4 to LB host b etc. for four LB hosts), with all LB hosts configured with the LB 'shared' unicast address (i.e., no NAT). If you want anything smarter than that, then I think the LB needs to become stateful, and start maintaining per-connection state.


>Note that I am specifically proposing setting the first four bits to 1
>aka 0xf so that the space can still be used for it's original intent too.
>
>> What you can do is use the flow label as intended, for ECMP or any
>> other kind of load balancing. RFC6438, RFC7098. They don't fix the
>> ICMPv6 problem though.
>
>It won't fix the 1-RTT issue that worries content providers either.
>
>That is the biggest problem for them: a ICMPv6 PTB is an extra round
>trip and they want speed. Hence including the MTU already in the packet.
>(though as noted in my reply, it won't work fully for async networks)
>
>> The key sentence in Joel's draft is
>> 
>>>    Because the PTB message is not identifiable as part of the
>>>    original flow by the packet header the results of the ICMPv6 ECMP
>>>    hash are unlikely to be hashed to the same nexthop as packets
>>>    matching TCP or UDP ECMP hash.
>> 
>> To fix that, I suspect that flow label reflection is needed.
>
>Won't work as that packet will not be matching the loadbalancers setup
>either.
>
>What would partially work is if the ICMPv6 PTB would copy the flow-label
>from the original flow that caused the PTB.
>
>But then still, large content providers will not be happy, as it does
>not satisfy the 1-RTT argument, which now causes them to just ignores
>PTBs and set the MSS to something tiny (thus in the end causing more
>packets, but at least all packets will go through).
>
>
>
>
>
>Greets,
>Jeroen
>
>_______________________________________________
>v6ops mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/v6ops
>
>
>
>