Re: [v6ops] [tcpm] Flow Label Load Balancing

Brian E Carpenter <> Thu, 03 December 2020 20:17 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 81E4E3A0C5E; Thu, 3 Dec 2020 12:17:59 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id rEsAJ2NhXSdG; Thu, 3 Dec 2020 12:17:58 -0800 (PST)
Received: from ( [IPv6:2607:f8b0:4864:20::42a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id CF1823A0C64; Thu, 3 Dec 2020 12:17:57 -0800 (PST)
Received: by with SMTP id w187so2066642pfd.5; Thu, 03 Dec 2020 12:17:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=kEDGKP4sfCroZwuQ1m6T+B62pz/0WYqGJkbq/af38D0=; b=BM0srVnAxB6OzD5rI1HvPKyqbaKO7YVJvj/MlIq36GHUhdhh9Eg3mU23RIF5i32CBH Pd4a9Grla+Ko28SbsbqAyTZi4awDOSP0P9VJRceIrGcLeTwPrDFQ7cDbDxGdCNmPgSFr IWL+tmDatE6XnDoqV3U6ciloJW6VdF7yFV86twdR4cFZ71EyfLb4/3uoXjRi2zRFfo9s FWZtFFLGIV1iuih1DTBMHXZBtSZG7isClRIER9iGRJHzsmrmOQDPdT0xPXeSisynHcWI tqfVGTiN0PH1zWMyH2TcRYi7iY6ENerTmoY1kpcZ3EWw3UfJ2BsCdlYu9uhjUktXb4HZ iSIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=kEDGKP4sfCroZwuQ1m6T+B62pz/0WYqGJkbq/af38D0=; b=FNS0mcY5lTa0CL31+pzwv11Y/24t/kXBaitLxufNP3lQDZg7UNc5ipV4GC7xfC4xNI oupBGp5CKvhVLHdYNa60c1TuOwn7fTxp1kJA5Fq+6HIOVR63/iXH6Yh1lwHFMc/vNueW 8tvJiCYvVUWVZqWgGgkLyQYfYkn8f8ge7e8ehA2aryqvr28scp0e3A9VPpfIB5/DkniR yU+rG2dCyUNEO+zkukv0biX0rWTTflxo6KtU2sBhELUVl8UpjldaPZC0/z6uvUbeCHYa jazCnL/L+ClK4KbWUWvhFdNQBqyfoaVDP4rgHq4sQ3AGrLIbc0/CdcFwDW8oY9Y9W5kE SJbg==
X-Gm-Message-State: AOAM533JwTgoz1Utuhr5ZSkMTONXPEBLRw/W/e8nU/4sKX+r6iBR4ch2 wd9RZ0CF/wy+FtYz5gdoaZ0=
X-Google-Smtp-Source: ABdhPJyT7qAWiTzVh7vy6L7onRvcGwGPiQC1thZmWfkJ43/b9jp6HKwf4pS4WEGtXpp0RipntecwHQ==
X-Received: by 2002:a63:d90a:: with SMTP id r10mr4506831pgg.336.1607026677263; Thu, 03 Dec 2020 12:17:57 -0800 (PST)
Received: from [] ([]) by with ESMTPSA id a16sm2647308pfl.125.2020. (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Dec 2020 12:17:56 -0800 (PST)
To: Tom Herbert <>, Erik Kline <>
Cc: Fernando Gont <>, IPv6 Operations <>, tcpm <>, Fernando Gont <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
From: Brian E Carpenter <>
Message-ID: <>
Date: Fri, 4 Dec 2020 09:17:52 +1300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: <>
Subject: Re: [v6ops] [tcpm] Flow Label Load Balancing
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: v6ops discussion list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 03 Dec 2020 20:18:04 -0000

On 04-Dec-20 06:30, Tom Herbert wrote:
> On Thu, Dec 3, 2020 at 8:43 AM Erik Kline <> wrote:
>>>> Please include requirements for both hosts and network nodes. If there
>>>> is going to the be a requirement that hosts MUST make a 1:1 mapping to
>>>> a transport connection and the flow label MUST be consistent for the
>>>> life of connection, then I would expect there to also be a requirement
>>>> that the flow label MUST be immutable by all devices in the path-- in
>>>> particular the requirement of RFC6437 would need to be updated: "A
>>>> forwarding node MUST either leave a non-zero flow label value
>>>> unchanged or change it only for compelling operational security
>>>> reasons..."-- if hosts are not allowed to change a field they are
>>>> primarily responsible for setting, then the network should not be
>>>> allowed to modify it either.
>>> I don't recall _(of the top of my head) why we ended up making the field
>>> mutable ("SHOULD NOT  be modified" rather than "MUST NOT.."). IIRC, it
>>> had to do with allowing middle-boxes to mitigate FL-based covert
>>> channels.  If that's the case (Brian CC'ed) then, in retrospective, that
>>> seems to have been a bad idea.  -- I was part of the discussion, so I
>>> take my share of blame. :-)
>> My understanding/recollection is that since there were never any
>> restrictions on modification of the flow label from the very beginning
>> it wasn't super meaningful to try to close the barn door after all
>> these years.
> Erik,
> There are two apropos statements in RFC6437:
> "There is no way to verify whether a flow label has been modified en
> route or whether it belongs to a uniform distribution.  Therefore, no
> Internet-wide mechanism can depend mathematically on unmodified and
> uniformly distributed flow labels; they have a "best effort" quality."

Yes, a MUST NOT is futile because it cannot be enforced.

Also, the firewall folk  seemed very insistent that they would change
the flow label (or zero it) because paranoia. So specifying that if they
changed it, they should do it consistently, seemed better than a MUST NOT
which they would definitely ignore.

> "From an upper-layer viewpoint, a flow could consist of all packets in
> one direction of a specific transport connection or media stream.
> However, a flow is not necessarily 1:1 mapped to a transport
> connection."
> So building a load balancer, or any other intermediate network device,
> that *depends* on the flow label to be consistent for the life of a
> transport connection and 1:1 mapped to a transport connection is
> contrary to this guidance. The ship has already sailed on this in
> deployment, and even if the protocol requirements were to be
> sufficiently tightened that would take years to reach full effect and
> that would preclude other use cases where modifying the flow label is
> useful. If a stable 1:1 mapped identifier for transport ports is
> needed in the network layer, it might make sense to create a new HBH
> option for that. Personally, if I were building a load balancer I'd
> use neither the flow label nor transport ports for entropy, I'd look
> at using the destination address since 128 bits allows plenty of bits
> of entropy, it's consistent per transport flow and always set in an
> IPv6 packet.

But that doesn't help with server load balancing. For ECMP/LAG in a
transit network, the DA alone doesn't help either, if a particular 
destination is prone to very heavy flows for some reason. And of course
the ports don't help if they're encrypted.

There's no simple answer here.