Re: [tcpm] [v6ops] Flow Label Load Balancing
Tom Herbert <tom@herbertland.com> Wed, 25 November 2020 19:35 UTC
Return-Path: <tom@herbertland.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 663E43A1BDF for <tcpm@ietfa.amsl.com>; Wed, 25 Nov 2020 11:35:22 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=herbertland-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FH3W9f9aklsB for <tcpm@ietfa.amsl.com>; Wed, 25 Nov 2020 11:35:21 -0800 (PST)
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BE6353A1BE0 for <tcpm@ietf.org>; Wed, 25 Nov 2020 11:35:20 -0800 (PST)
Received: by mail-ed1-x52e.google.com with SMTP id r22so3788050edw.6 for <tcpm@ietf.org>; Wed, 25 Nov 2020 11:35:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=herbertland-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=iKRu8L6k2Wzc+EUc36nWFqtRrs07O1b+4cGtQZ2Jces=; b=1CvkNK/u5S1DLQfhdC2b/IBPyBXy52CQhbS5AFdaB3PGbNzGHFDnhEj1V95zZXxeFO ClJo5TU6yt5UiQSGBJ0xT4eEp4bMOClopYjjdLfXMdk+eOflH9Wj36QuSGb9L0l4swby rGf84txa/rjncjzUrbE9pdBJdEfWcBLndyjMKtQs9h02kQwwkbjwnU7PmB47Zq6FSzDx hB+B3cKSG+d50iHAPGMwzWfXZSOvlzjgdwcdr225s0xWZs8kPnBQ4sh+HVUTlr3fnmHW ee7hTPzwGCOb7LdvtLv81ltmKtCpmWgETwc/3xMc91NAZLpq8A5KEweGTNLoh9LJW4Tj reIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=iKRu8L6k2Wzc+EUc36nWFqtRrs07O1b+4cGtQZ2Jces=; b=GKKO9Gd33Kh2A/jpVtdoA3/27UWDLuY0qn8pnXANVa9oI+Kn6+ntjxvlDp+BMuOZMn gr1rek9U+ODGt4gImhM7DCwaZz3LF6zvleZf7BAjPabmNEuEabC3NTRA4Gg60bJErynm Oma1c98DnKKJ3feUi4/pSi83fyLEFM1ae534coWMZlVMddzzWWvSD0NtPbr/AXdriXIC Bx27qX4iBUPGejkFDoRh2j7cNmG1F4Mz4OTRAq46imfv+y2ZNXdBqIjz3cUGngt0NBNr +1HEt4qj3O34PYsIANT665w7HFoFIWyaVUdcbGFDxZGvvPaMdCa0V5SuQr/BvrTjC2Gm WY4g==
X-Gm-Message-State: AOAM533KKtHjYRMI27jMHY8ewqnMUlcZLK7cMsidfQj4BnF/vxph9gvu mPqYhM5MNmkbz/pQg4D/ZSQHA8qEsK2edsjOejLNYA==
X-Google-Smtp-Source: ABdhPJyZWH9g/uec0zdhtlFbmrH5jXXcQo/IL4Ps6aWcgsrIkDSWeqa0u+giKJ4nEDMo7jW1qe0ATkp6XJTJouXGM5Y=
X-Received: by 2002:a05:6402:114b:: with SMTP id g11mr5111616edw.228.1606332918541; Wed, 25 Nov 2020 11:35:18 -0800 (PST)
MIME-Version: 1.0
References: <CAEGSd=DY8t8Skor+b6LSopzecoUUzUZhti9s0kdooLZGxPEt+w@mail.gmail.com> <d29042a7-742b-a445-cf60-2773e5515ae5@gont.com.ar> <CALx6S37+1duoNGR3dZWesHsZvx15kX9wCWufPMh=esvMaSMF_g@mail.gmail.com> <63e7aad3-7094-7492-dbe4-3eefb5236de3@gont.com.ar>
In-Reply-To: <63e7aad3-7094-7492-dbe4-3eefb5236de3@gont.com.ar>
From: Tom Herbert <tom@herbertland.com>
Date: Wed, 25 Nov 2020 12:35:07 -0700
Message-ID: <CALx6S37t4jump6S-R5_xdo5DF+RnHtT4rU5-RuiC-2GQ0PXxkQ@mail.gmail.com>
To: Fernando Gont <fernando@gont.com.ar>
Cc: Alexander Azimov <a.e.azimov@gmail.com>, tcpm <tcpm@ietf.org>, IPv6 Operations <v6ops@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/NHMtd_QnjoG_JeZD9SofrBPVdKs>
Subject: Re: [tcpm] [v6ops] Flow Label Load Balancing
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 25 Nov 2020 19:35:23 -0000
Hi Fernando, comments in line... On Wed, Nov 25, 2020 at 12:13 AM Fernando Gont <fernando@gont.com.ar> wrote: > > Hi, Tom, > > On 24/11/20 16:43, Tom Herbert wrote: > [....] > > Modulating the flow label is a means to affect the routing of packets > > through the network that uses flow labels as input to the ECMP hash. > > What's the point? > > 1) You cannot tell *if* the FL is being used. > Generally true, but in a limited domain this information could be discerned. I'd note that it's also generally true that we don't know if there is a load balancer or stateful firewall in the path that requires consistent routing, but in a limited domain we could know that also. > 2) Changing the FL does not necessarily mean that packets will employ a > different link. It's an opportunistic mechanism. If a connection is failing and we get a better path that fixes it by simply changing the flow label then what's the harm? > > 3) If the network is failing, shouldn't you handle this via routing? > Sure, but then that requires an out of band feedback loop from a TCP implementation to the network infrastructure to indicate there is a problem and then the network needs to respond. That's significant infrastructure and higher reaction time than doing something in TCP and IP. Think of modulating the flow label is an inexpensive form of source routing within a limited domain that doesn't need any infrastructure or heavyweight protocols or something like segment routing. > > > > The basic idea is that the flow label associated with a connection is > > randomly changed when the stack observes that the connection is > > failing (e.g. and an RTO). There is nothing in the specs that prevents > > this since the source is at liberty to set the flow label as it sees > > fit. > > The FL is expected to remain constant for the life of a flow. A > retransmitted packet is part of the same flow as the > originally-transmitted packet. So this seems to be contradicting the > very specification of the FL. > > For instance, If a RTO for a flow causes the FL to change, then one may > possibly argue that the FL is not naming/labeling what is said/expected > to be anming/labeling. Specifically, RFC6437 states: "It is therefore RECOMMENDED that source hosts support the flow label by setting the flow label field for all packets of a given flow to the same value chosen from an approximation to a discrete uniform distribution." So that is clearly a just recommendation, and not a requirement (and definitely not a MUST). Furthermore, RFC6437 states: "A forwarding node MUST either leave a non-zero flow label value unchanged or change it only for compelling operational security reasons as described in Section 6.1." So there's no guarantee in the protocol specs that flow labels are consistent for the life of the connection, which means that the network cannot assume that and thus it would be incorrect if the network tried to enforce flow label consistency as a protocol requirement. As I said, it is prudent to try to be consistent with flow labels and the default behavior in Linux should be changed, however I do not believe there's a valid claim of non-conformance that motivates removal of the feature that is already deployed. Tom > > > > > The feature is useful in large datacenter networks, like > > pparently Facebook where the patches originate, since information > > discerned by TCP can opportunistically be applied to route selection. > > The practical issue is that there are stateful devices like firewalls > > that require consistent routing in the network in which case changing > > the flow label can confuse them. As I mentioned, the original intent > > was that the flow label randomization feature should be opt-in instead > > of on by default. > > So... where is the "source" of the packet that would be "modulating" the FL? > > Thanks, > -- > Fernando Gont > e-mail: fernando@gont.com.ar || fgont@si6networks.com > PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1 > > >
- [tcpm] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Joel M. Halpern
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Joel M. Halpern
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Yuchung Cheng
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Mark Smith
- Re: [tcpm] [v6ops] Flow Label Load Balancing Joseph Touch
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Mark Smith
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Michael Tuexen
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Erik Kline
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont