Re: [v6ops] Flow Label Load Balancing

Fernando Gont <> Fri, 27 November 2020 20:42 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 3E5D83A0E87; Fri, 27 Nov 2020 12:42:03 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id BC9DmJ34equ1; Fri, 27 Nov 2020 12:42:00 -0800 (PST)
Received: from ( [IPv6:2001:67c:27e4::14]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 955643A0E84; Fri, 27 Nov 2020 12:41:57 -0800 (PST)
Received: from [IPv6:2800:810:464:8164:d564:510f:6fb3:4524] (unknown [IPv6:2800:810:464:8164:d564:510f:6fb3:4524]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPSA id 3C33C283BC9; Fri, 27 Nov 2020 20:41:52 +0000 (UTC)
To: Tom Herbert <>
Cc: Alexander Azimov <>, tcpm <>, IPv6 Operations <>
References: <> <> <> <> <>
From: Fernando Gont <>
Message-ID: <>
Date: Fri, 27 Nov 2020 17:41:40 -0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: <>
Subject: Re: [v6ops] Flow Label Load Balancing
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: v6ops discussion list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 27 Nov 2020 20:42:03 -0000

Hello, Tom,

On 25/11/20 16:35, Tom Herbert wrote:
> Hi Fernando, comments in line...
> On Wed, Nov 25, 2020 at 12:13 AM Fernando Gont <> wrote:
>> Hi, Tom,
>> On 24/11/20 16:43, Tom Herbert wrote:
>> [....]
>>> Modulating the flow label is a means to affect the routing of packets
>>> through the network that uses flow labels as input to the ECMP hash.
>> What's the point?
>> 1) You cannot tell *if* the FL is being used.
> Generally true, but in a limited domain this information could be
> discerned. I'd note that it's also generally true that we don't know
> if there is a load balancer or stateful firewall in the path that
> requires consistent routing, but in a limited domain we could know
> that also.

Exactly. So my take is that the drawbacks for the general case outweigh 
the benefits of the specific case.

>> 2) Changing the FL does not necessarily mean that packets will employ a
>> different link.
> It's an opportunistic mechanism. If a connection is failing and we get
> a better path that fixes it by simply changing the flow label then
> what's the harm?

Complexity, and the possible negative implications for the general case.

That said, why don't you fix the problem where you should (i.e., routing)?

>> 3) If the network is failing, shouldn't you handle this via routing?
> Sure, but then that requires an out of band feedback loop from a TCP
> implementation to the network infrastructure to indicate there is a
> problem and then the network needs to respond.

Not really. TCP does not deal with paths (not should it). If the network 
detects a link is failing, it should route around the problem.

> That's significant
> infrastructure and higher reaction time than doing something in TCP
> and IP. Think of modulating the flow label is an inexpensive form of
> source routing within a limited domain that doesn't need any
> infrastructure or heavyweight protocols or something like segment
> routing.


* It breaks load balancing
* It depends on the source doing something -- but you don't necessarily 
control the source -- and you cannot tell if the source implements it
* It doesn't solve the problem for nodes that do not behave as expected

My take is that if you really care about this, you're already (or 
should!) be solving the problem in a different (and more general) way.

>>> The basic idea is that the flow label associated with a connection is
>>> randomly changed when the stack observes that the connection is
>>> failing (e.g. and an RTO). There is nothing in the specs that prevents
>>> this since the source is at liberty to set the flow label as it sees
>>> fit.
>> The FL is expected to remain constant for the life of a flow. A
>> retransmitted packet is part of the same flow as the
>> originally-transmitted packet. So this seems to be contradicting the
>> very specification of the FL.
>> For instance, If a RTO for a flow causes the FL to change, then one may
>> possibly argue that the FL is not naming/labeling what is said/expected
>> to be anming/labeling.
> Specifically, RFC6437 states:
> "It is therefore RECOMMENDED that source hosts support the flow label
> by setting the flow label field for all packets of a given flow to the
> same value chosen from an approximation to a discrete uniform
> distribution."
> So that is clearly a just recommendation, and not a requirement (and
> definitely not a MUST). Furthermore, RFC6437 states:
> "A forwarding node MUST either leave a non-zero flow label value
> unchanged or change it only for compelling operational security
> reasons as described in Section 6.1."

My concern is that we may leave the FL unusable.

> So there's no guarantee in the protocol specs that flow labels are
> consistent for the life of the connection, which means that the
> network cannot assume that and thus it would be incorrect if the
> network tried to enforce flow label consistency as a protocol
> requirement.

You're probably right in that respect. And, in retrospective, I think it 
would have been better if we had specified the FL to be unmutable.

> As I said, it is prudent to try to be consistent with
> flow labels and the default behavior in Linux should be changed,

Sorry: is the current Linux behavior to modulate the FL?

> however I do not believe there's a valid claim of non-conformance that
> motivates removal of the feature that is already deployed.

I personally consider the changing the FL for a flow to be a bug. That 
said, if the default setting is the correct/compliant behavior, at the 
end of the day we're all adults, and whoever overrides the default 
behavior is supposed to know what he/she does -- and if they don't, they 
get what they deserve  ;-)

So, in that light, I wouldn't push to remove the optional behavior, 
*provided* the default behavior is correct.

Fernando Gont
e-mail: ||
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1