Re: [tcpm] [v6ops] Flow Label Load Balancing
Tom Herbert <tom@herbertland.com> Thu, 03 December 2020 20:38 UTC
Return-Path: <tom@herbertland.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE6CC3A0C81 for <tcpm@ietfa.amsl.com>; Thu, 3 Dec 2020 12:38:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=herbertland-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bm5rmnUaUz8O for <tcpm@ietfa.amsl.com>; Thu, 3 Dec 2020 12:38:56 -0800 (PST)
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 91CC83A0C54 for <tcpm@ietf.org>; Thu, 3 Dec 2020 12:38:52 -0800 (PST)
Received: by mail-ej1-x630.google.com with SMTP id jx16so5433465ejb.10 for <tcpm@ietf.org>; Thu, 03 Dec 2020 12:38:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=herbertland-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NIrav4tdAcakDCg0n7gnAvqr1Mn5aNrsxiqiFjmNwOU=; b=Eldg9cCt3VsCZru0t/MYk6TnPGRoZ0z9YbeRjy+CxTrIV1HR1Cla/V4roks76YPScc cdUuEx3cS/LvcKftzwEG6xXLrUhZAPCVteNlHMgLtC2ZEK4muNLxO6FVkqGRWkzvVMyS 1GKH8wGFvRUbgZnZFZX58Y/PsUjvfyjwnmunF1u0pjPE50cnRWQ0EE1GwqXOgtJUaT46 4MsR7Hv6NXy5f+eUT2wn4NbO5iWZcAYb3VsPuBG3MIftTFr9KRTpdrrrM6ILGLVoJ9ow StxAGPGBYvolI5FeERFXYVZX3TN9Z11pawKEm2b1OZCTC4uzRz5udT3f8+u8WBhLYzFY MUSg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NIrav4tdAcakDCg0n7gnAvqr1Mn5aNrsxiqiFjmNwOU=; b=r3ovONGIncdWIaoXF+osgLhoA4ao9W/JMOuwANMHfSSajRLbF7Fi5zWAdy7PIdxd3P T905GH4GrUV4/Y0b0tfQ2sY7H5dLR6meMML/pEAJ3S6g0LEmMNvPJVdD4azHe96cp9Bj g2BVmt+Brjd1FwIEWoLbM5fEmWyOXJB96icEfhd01oD8mL1elmlg85MaGQjN2Fcb2LR6 MrN+1RUDaNx/1peGapxMWsDkqtepkxpM1h0vjHMPAu49s2rP+nb6mGDWZ+05l53Fz3nW L3pHznqvEBOeph+lWSv8r3s61CoQw3kMUOjzf0ZQe083/TAZoIaWtsrYw7hElVNBZ5oo eTrw==
X-Gm-Message-State: AOAM531N0C1Dt4B6pmcy3zwVqqaTnwDecf0T9t2NyXvQ4+0NOS0q1FIp bHgtDWgOH9709AI3fQuKXkZ0sWwD8jbBoy4ka6GqlQ==
X-Google-Smtp-Source: ABdhPJzZzjuN8kyMl/2KzkfB5VN9VLQdYIkL7Ss22z65BHIUJXYdebynZmKFfPdsHOskTBjN54JrM0Dd+JC55vvg1Ow=
X-Received: by 2002:a17:906:3294:: with SMTP id 20mr4211114ejw.239.1607027930742; Thu, 03 Dec 2020 12:38:50 -0800 (PST)
MIME-Version: 1.0
References: <CAEGSd=DY8t8Skor+b6LSopzecoUUzUZhti9s0kdooLZGxPEt+w@mail.gmail.com> <CALx6S37t4jump6S-R5_xdo5DF+RnHtT4rU5-RuiC-2GQ0PXxkQ@mail.gmail.com> <96b6d04b-e5bb-ba79-0281-e9599109be95@gont.com.ar> <CALx6S34uCrA1QdvLV8fpRKaJGLWMgtCmBCnrsBjU3TS+kXUs3Q@mail.gmail.com> <CAO42Z2xn_+7EVpjGyEU3aAdBmt1h=a4MPXFjjoTi_JeM2w9pkg@mail.gmail.com> <f66cbccd-55ed-375b-743b-7fc6c48a50c2@gont.com.ar> <CAO42Z2xqU4gs9iP=u_0Z16Qk+U24_YH0h5vTmJRJ5XZXZ0nweQ@mail.gmail.com> <0d38980a-f1c5-fac5-a9b1-0711d61353d1@gont.com.ar> <CAEGSd=A_e-db8m2VN+2wEuXj9e+GTq7brYfY_fwW7tysUr19Ng@mail.gmail.com> <CALx6S34xQF-PHQRqom_O8=amoRFmVrzHL-qh8765mtr1XnF2Wg@mail.gmail.com> <d59e1785-672d-8cc7-f844-51c64a440a57@gont.com.ar> <CALx6S373yLJwHigv4Yo-xdtRkZ9YsB0J9cwXqy0BWpwXSiHCPg@mail.gmail.com> <42b6e327-08b5-292b-be65-28f1a8508a69@si6networks.com> <CAMGpriUb3WQqFhtDULy=Avbf8dWh1LsO=LBfvGUf3ozBAg7myA@mail.gmail.com> <CALx6S373gk+GweWm=ph8fHgSk7rDPGQCXjdC++G6Jk2NGo1x-A@mail.gmail.com> <85dd3823-ff2d-94ac-3535-d3d042bdd338@gmail.com>
In-Reply-To: <85dd3823-ff2d-94ac-3535-d3d042bdd338@gmail.com>
From: Tom Herbert <tom@herbertland.com>
Date: Thu, 03 Dec 2020 13:38:39 -0700
Message-ID: <CALx6S35oJRRnqMNOqGMz1XfkR-f8Hn7RBcQwbEyEL2j_ueGjUg@mail.gmail.com>
To: Brian E Carpenter <brian.e.carpenter@gmail.com>
Cc: Erik Kline <ek.ietf@gmail.com>, Fernando Gont <fgont@si6networks.com>, IPv6 Operations <v6ops@ietf.org>, tcpm <tcpm@ietf.org>, Fernando Gont <fernando@gont.com.ar>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/6xrx95h7HgsgHJksCBtuEOE-CDE>
Subject: Re: [tcpm] [v6ops] Flow Label Load Balancing
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 03 Dec 2020 20:39:05 -0000
On Thu, Dec 3, 2020 at 1:17 PM Brian E Carpenter <brian.e.carpenter@gmail.com> wrote: > > On 04-Dec-20 06:30, Tom Herbert wrote: > > On Thu, Dec 3, 2020 at 8:43 AM Erik Kline <ek.ietf@gmail.com> wrote: > >> > >>>> Please include requirements for both hosts and network nodes. If there > >>>> is going to the be a requirement that hosts MUST make a 1:1 mapping to > >>>> a transport connection and the flow label MUST be consistent for the > >>>> life of connection, then I would expect there to also be a requirement > >>>> that the flow label MUST be immutable by all devices in the path-- in > >>>> particular the requirement of RFC6437 would need to be updated: "A > >>>> forwarding node MUST either leave a non-zero flow label value > >>>> unchanged or change it only for compelling operational security > >>>> reasons..."-- if hosts are not allowed to change a field they are > >>>> primarily responsible for setting, then the network should not be > >>>> allowed to modify it either. > >>> > >>> I don't recall _(of the top of my head) why we ended up making the field > >>> mutable ("SHOULD NOT be modified" rather than "MUST NOT.."). IIRC, it > >>> had to do with allowing middle-boxes to mitigate FL-based covert > >>> channels. If that's the case (Brian CC'ed) then, in retrospective, that > >>> seems to have been a bad idea. -- I was part of the discussion, so I > >>> take my share of blame. :-) > >> > >> My understanding/recollection is that since there were never any > >> restrictions on modification of the flow label from the very beginning > >> it wasn't super meaningful to try to close the barn door after all > >> these years. > > > > Erik, > > > > There are two apropos statements in RFC6437: > > > > "There is no way to verify whether a flow label has been modified en > > route or whether it belongs to a uniform distribution. Therefore, no > > Internet-wide mechanism can depend mathematically on unmodified and > > uniformly distributed flow labels; they have a "best effort" quality." > > Yes, a MUST NOT is futile because it cannot be enforced. > > Also, the firewall folk seemed very insistent that they would change > the flow label (or zero it) because paranoia. So specifying that if they > changed it, they should do it consistently, seemed better than a MUST NOT > which they would definitely ignore. > > > "From an upper-layer viewpoint, a flow could consist of all packets in > > one direction of a specific transport connection or media stream. > > However, a flow is not necessarily 1:1 mapped to a transport > > connection." > > > > So building a load balancer, or any other intermediate network device, > > that *depends* on the flow label to be consistent for the life of a > > transport connection and 1:1 mapped to a transport connection is > > contrary to this guidance. The ship has already sailed on this in > > deployment, and even if the protocol requirements were to be > > sufficiently tightened that would take years to reach full effect and > > that would preclude other use cases where modifying the flow label is > > useful. If a stable 1:1 mapped identifier for transport ports is > > needed in the network layer, it might make sense to create a new HBH > > option for that. Personally, if I were building a load balancer I'd > > use neither the flow label nor transport ports for entropy, I'd look > > at using the destination address since 128 bits allows plenty of bits > > of entropy, it's consistent per transport flow and always set in an > > IPv6 packet. > > But that doesn't help with server load balancing. For ECMP/LAG in a > transit network, the DA alone doesn't help either, if a particular > destination is prone to very heavy flows for some reason. Brian, Specifically, that is a problem for a host that uses a _single_ IPv6 address for all its connections. What I am suggesting is to give hosts many addresses, which we already do anyway like when mobile devices and data center servers are assigned /64s. If a host then randomly chooses an address from that block for each connection it creates thenl the address 2-tuple contains all the entropy needed for purposes of load balancing, ECMP, etc. is contained in the source address. Addresses are guaranteed to be present and plaintext in each packet so no DPI required, they're as stable as any flow identifying information in a packet can be and they offer much more entropy than sixteen bits of port number of twenty bit flow label. Since the block address assignments or already happening, this seems more to be a host implementation matter where we perform address selection for a new connection by random selection in the block. It's on my wish list of things to implement :-) Tom > And of course > the ports don't help if they're encrypted. > > There's no simple answer here. > > Brian
- [tcpm] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Joel M. Halpern
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Joel M. Halpern
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Yuchung Cheng
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Mark Smith
- Re: [tcpm] [v6ops] Flow Label Load Balancing Joseph Touch
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Mark Smith
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Alexander Azimov
- Re: [tcpm] [v6ops] Flow Label Load Balancing Michael Tuexen
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Erik Kline
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Brian E Carpenter
- Re: [tcpm] [v6ops] Flow Label Load Balancing Tom Herbert
- Re: [tcpm] [v6ops] Flow Label Load Balancing Fernando Gont