Re: [v6ops] Flow Label Load Balancing

Tom Herbert <tom@herbertland.com> Tue, 24 November 2020 19:44 UTC

Return-Path: <tom@herbertland.com>
X-Original-To: v6ops@ietfa.amsl.com
Delivered-To: v6ops@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 08E833A18BC for <v6ops@ietfa.amsl.com>; Tue, 24 Nov 2020 11:44:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=herbertland-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wBqG2dBBEPZm for <v6ops@ietfa.amsl.com>; Tue, 24 Nov 2020 11:44:06 -0800 (PST)
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 320D13A18BB for <v6ops@ietf.org>; Tue, 24 Nov 2020 11:44:06 -0800 (PST)
Received: by mail-ej1-x635.google.com with SMTP id mc24so9126494ejb.6 for <v6ops@ietf.org>; Tue, 24 Nov 2020 11:44:06 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=herbertland-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=FU2O2RNZOW//Q1QmbUgngJsTZOfiFc8CERjeQLE9RyM=; b=MuM2a2ACT0B2bBsONHR4Y3o1+oAtOwPb+z4PmSsGwxi6u3joaI1HmUOMSmDzAaC8ff CTTKWzxMxGFxR2AAFCPQYOr5iEllhF+MDy396vDZxevgBjsuMUyE2iAZsUBOoJfrvi0p U4iuqFBkvnoG+w0KpTfvL2BBZ+Njln4ss8DEJBIa8gSZ8myS8Q2m7HqMRO69upbk64em 0EqB/PV6bVRm5sgc59EYDld1xedD++td0r1sjydx/tE2JGtNCI2RHniak3muw8Rm6OSM oLE3chsp8+YWhVQZ26mZbW6mwb99JHzgNhtRtceXu4LL4nskp6dD8I4lIbKSWUsd8Uv9 Uyxg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FU2O2RNZOW//Q1QmbUgngJsTZOfiFc8CERjeQLE9RyM=; b=ckqwk0sMZcxUAAGmhw0EHrAzZ4ZlW4KOGlXL6STF2lzJphn7kvUb8cE1s0TLFWOWw7 +wvp7jVxvfrOR+F09EMaxXsY5ZfFiSSHBPBEJSg3tTAN0VN9bHN6TdiLYrk3bUYm3lY5 ioFhYSocdggRH2Ttl3vi4zKe7f9QCQko7fAm8bdnX6cJAPv+LwzJlJnkXJ3CVO32+qWG Wi+Xch5mrVvE9Bbgsq/tDg7G9aoeJSIE5BYGQROjwgUS6gCJmV2tGvBY9F7Fxoe2qsk9 iQbT0hXqB49JPXrh0FcBcHLdOwkHzzZ60rdUpeW6/6uZ80woIz5KncEm8JFaIDGIg2vd 6xDA==
X-Gm-Message-State: AOAM531HqCAAvnLC5yRMktddVBaWlQwoJhJ2Uofh55YkacpRswgUuVXr KD/itCRLYTJxGYgLW5io3vHuQfqrVpbsxUAatVQTJw==
X-Google-Smtp-Source: ABdhPJx20aiuCkrIyqkT15ZaWAeIEao5pLNrlTzgaz7F+bCqqZHC3tHUWhpJRQq1DRNlnXGgT4aFYBb8WVuoop/BAxU=
X-Received: by 2002:a17:906:e2da:: with SMTP id gr26mr5337739ejb.265.1606247044480; Tue, 24 Nov 2020 11:44:04 -0800 (PST)
MIME-Version: 1.0
References: <CAEGSd=DY8t8Skor+b6LSopzecoUUzUZhti9s0kdooLZGxPEt+w@mail.gmail.com> <d29042a7-742b-a445-cf60-2773e5515ae5@gont.com.ar>
In-Reply-To: <d29042a7-742b-a445-cf60-2773e5515ae5@gont.com.ar>
From: Tom Herbert <tom@herbertland.com>
Date: Tue, 24 Nov 2020 12:43:52 -0700
Message-ID: <CALx6S37+1duoNGR3dZWesHsZvx15kX9wCWufPMh=esvMaSMF_g@mail.gmail.com>
To: Fernando Gont <fernando@gont.com.ar>
Cc: Alexander Azimov <a.e.azimov@gmail.com>, tcpm <tcpm@ietf.org>, IPv6 Operations <v6ops@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/v6ops/JndNRtMM9J85SxxoVgdUHEqxRbo>
Subject: Re: [v6ops] Flow Label Load Balancing
X-BeenThere: v6ops@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: v6ops discussion list <v6ops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/v6ops>, <mailto:v6ops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/v6ops/>
List-Post: <mailto:v6ops@ietf.org>
List-Help: <mailto:v6ops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/v6ops>, <mailto:v6ops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Nov 2020 19:44:08 -0000

On Tue, Nov 24, 2020 at 12:07 PM Fernando Gont <fernando@gont.com.ar> wrote:
>
> On 19/11/20 07:48, Alexander Azimov wrote:
> > Dear colleagues,
> >
> > I have added in the cc both v6op and tcpm for a reason and let me
> > explain why.
> >
> > It's clear that we are moving forward with load balancing that uses flow
> > label (FL). And the pressure will increase with SRv6 adoption. But at
> > the moment wide adoption of FL-based load-balancing may create
> > significant issues for TCP Anycast services.
> >
> > RFC6437 suggests putting hash from 5-tuple into FL value. And as far as
> > I know, there is no document that updates this behavior. This
> > description is perfectly fine, but what is implemented in the Linux
> > kernel is different: FL is carrying hash from 5-tuple with an additional
> > seed, and this seed is randomly changed after each RTO/SYN_RTO event.
>
> Changing the FL upon RTO is a bug.
>
> I guess/assume that when you say SYN-RTO, you really mean "user
> timeout", rather than RTO. If you don't, then that's also a bug.
> If you do, I fail to understand what's the reason for wanting the FL to
> change in that case, because as a result of port randomization, it 0s
> unlikely that the same four-tuple is employed for the next connection retry.
>
>
> > Here are related patches:
> >
> >   * https://lore.kernel.org/netdev/alpine.DEB.2.02.1407012100290.20628@tomh.mtv.corp.google.com/
> >   * https://lore.kernel.org/netdev/1438124526-2129341-1-git-send-email-tom@herbertland.com/
> >   * https://lore.kernel.org/netdev/20160928020337.3057238-1-brakmo@fb.com/
> >
> >
> > This is a great thing by the way because in the data center
> > environment with multiple equal paths it gives a way to have
> > pseudo-multipath TCP which jumps between paths in case of an outage.
> > There might be interest to writedown an informational document for this.
>
> That's a bad idea, since specs-wise the Flow-Label is not guaranteed to
> remain unchanged from source to destination. If you want to ahve
> multiple paths, then you should implement that in routing.
>
>
>
> > I wonder what you think is a proper solution:
> >
> >   * Making FL related RTO change as knob instead of default behavior;
> >   * Adding negotiation behavior in TCP;
> >   * Something else?
>
> Just make the FL a function of the connection "identifier". And keeo it
> constant for the lifetime of that conenction.

Fernando,

Modulating the flow label is a means to affect the routing of packets
through the network that uses flow labels as input to the ECMP hash.
The basic idea is that the flow label associated with a connection is
randomly changed when the stack observes that the connection is
failing (e.g. and an RTO). There is nothing in the specs that prevents
this since the source is at liberty to set the flow label as it sees
fit. The feature is useful in large datacenter networks, like
pparently Facebook where the patches originate, since information
discerned by TCP can opportunistically be applied to route selection.
The practical issue is that there are stateful devices like firewalls
that require consistent routing in the network in which case changing
the flow label can confuse them. As I mentioned, the original intent
was that the flow label randomization feature should be opt-in instead
of on by default.

Tom
>
> Thanks,
> --
> Fernando Gont
> e-mail: fernando@gont.com.ar || fgont@si6networks.com
> PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
>
>
>
> _______________________________________________
> v6ops mailing list
> v6ops@ietf.org
> https://www.ietf.org/mailman/listinfo/v6ops