Re: [v6ops] Flow Label Load Balancing

Brian E Carpenter <brian.e.carpenter@gmail.com> Tue, 24 November 2020 19:45 UTC

Return-Path: <brian.e.carpenter@gmail.com>
X-Original-To: v6ops@ietfa.amsl.com
Delivered-To: v6ops@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C0AB93A18BF; Tue, 24 Nov 2020 11:45:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Level:
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1DIlQD3lTVTI; Tue, 24 Nov 2020 11:45:40 -0800 (PST)
Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 19B733A18BD; Tue, 24 Nov 2020 11:45:40 -0800 (PST)
Received: by mail-pg1-x533.google.com with SMTP id 81so138414pgf.0; Tue, 24 Nov 2020 11:45:40 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=l3QdHBNYMCLYS5OFoHm7WqZ7++WzPT2C5Rx7LvQUUW8=; b=fUeU8mppvgPYZNhUoXlm+/13drnVp2Uz0wzmjMDva3PcBGuNfBCbLNvxFqT6/YxU53 taU44jHVA3ZyZpC6D3LOcFPlXRJcPyztJoWbfP3FwfDqIEpb5cMTUumc3qLxTGJHKkZU dhE38FdGD30JlADytwQDHi9VbEhnahvJdPO/eaaK8H/1tRya8BXWAU8COhN4MJtODoZU vewS+IlLCk18vNI2D2LQyR96V6ImDg5PY3uGVfPFbo2fHkBEOri1gja55q6TqtbDSL0X /X9PpCYNyvMsYMy6x2pgKEYgfZBa/2yw6wtSMTnCVc2EGPfNP1JOJDDansFPkps04jhp trug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=l3QdHBNYMCLYS5OFoHm7WqZ7++WzPT2C5Rx7LvQUUW8=; b=fclLeMBErhrEyMMi7tIvnGUg4mERU0tsvmmWZvumsollFUIkvGX61X6OuDDTxKbdrp 1HtdAesJEPiPbasB+lLUo3qkEIyZX1CCeNG7ad7zCxmPhvVMhWmd64ow0kSoA6ZuuFBi 9XtvQMKMAkPgJVgd5vBbqBjXant5zmyd4dKjXniMEyMqun2JYDf3Pwdhd8FMRwRamRsB LHI8yqRoObfwo86GCCtCZwP5L8TdnriGdmAuLd0ejQF4ChIMgWvVhQ5XXhj0AjUyDErq zyjlMCiPVz44+30OoPuW8gF7oCrRQRXrw+qs+LGwzdDjx3egK4H4LxlGBslA7X64Otmh pjHQ==
X-Gm-Message-State: AOAM532mBOHkDmnravYhpn/TPBCuU0S/x0TQiLvLQ5gOvqwFhPcIgnSN gt6PddEB+XvdeJTqu8YRll0=
X-Google-Smtp-Source: ABdhPJx0ZmKNteCUBaHAg7f9GJobBYwVWBd1yWDfJknzzAaae67YUowh0xal3/HjhpB3r1vHP4Bm5A==
X-Received: by 2002:a17:90b:b15:: with SMTP id bf21mr186253pjb.21.1606247139592; Tue, 24 Nov 2020 11:45:39 -0800 (PST)
Received: from [192.168.178.20] ([151.210.131.28]) by smtp.gmail.com with ESMTPSA id z12sm15943744pfg.123.2020.11.24.11.45.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Nov 2020 11:45:38 -0800 (PST)
To: Alexander Azimov <a.e.azimov@gmail.com>, Fernando Gont <fernando@gont.com.ar>
Cc: IPv6 Operations <v6ops@ietf.org>, tcpm <tcpm@ietf.org>
References: <CAEGSd=DY8t8Skor+b6LSopzecoUUzUZhti9s0kdooLZGxPEt+w@mail.gmail.com> <d29042a7-742b-a445-cf60-2773e5515ae5@gont.com.ar> <CAEGSd=AB5DMopq5Hc0ydZwP+xQuwxNBHuFSpCPcZvnaZbJfRoQ@mail.gmail.com>
From: Brian E Carpenter <brian.e.carpenter@gmail.com>
Message-ID: <fc7693e8-a57b-004d-a019-159060c6feef@gmail.com>
Date: Wed, 25 Nov 2020 08:45:34 +1300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1
MIME-Version: 1.0
In-Reply-To: <CAEGSd=AB5DMopq5Hc0ydZwP+xQuwxNBHuFSpCPcZvnaZbJfRoQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/v6ops/_Kqnx6IWOga19iGfuis12e-83rQ>
Subject: Re: [v6ops] Flow Label Load Balancing
X-BeenThere: v6ops@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: v6ops discussion list <v6ops.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/v6ops>, <mailto:v6ops-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/v6ops/>
List-Post: <mailto:v6ops@ietf.org>
List-Help: <mailto:v6ops-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/v6ops>, <mailto:v6ops-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Nov 2020 19:45:42 -0000

On 25-Nov-20 08:28, Alexander Azimov wrote:
> Hi Fernando,
> 
> Stating that FL change during TCP session lifetime is a bug - is a bit harsh.

It's a bug. It definitely breaks server load balancing, which in practice is more important than en route load balancing, because it will probably switch your session to a different server. (OK, I know that there are mechanisms sometimes for rescuing a session that has unexpectedly moved from one server to another, but in general it's a fail.)

I assume you have read https://tools.ietf.org/html/rfc7098

Regards
    Brian

> 
> It is a fantastic idea to change the FL value if RTO or SYN_RTO happens in a controlled environment. 
> These are very specific TCP timeouts, that provide enough guarantee that there will be no out-of-order packets, though your packets will reach the destination even in case of an outage inside your network. Zero influence on your services in case of the network outage - doesn't sound like a bug for me.
> 
> The problem is with the current Linux (though I haven't checked other OSes) defaults. Specifically, with the default behavior after RTO event.
> 
> вт, 24 нояб. 2020 г. в 22:06, Fernando Gont <fernando@gont.com.ar <mailto:fernando@gont.com.ar>>:
> 
>     On 19/11/20 07:48, Alexander Azimov wrote:
>     > Dear colleagues,
>     >
>     > I have added in the cc both v6op and tcpm for a reason and let me
>     > explain why.
>     >
>     > It's clear that we are moving forward with load balancing that uses flow
>     > label (FL). And the pressure will increase with SRv6 adoption. But at
>     > the moment wide adoption of FL-based load-balancing may create
>     > significant issues for TCP Anycast services.
>     >
>     > RFC6437 suggests putting hash from 5-tuple into FL value. And as far as
>     > I know, there is no document that updates this behavior. This
>     > description is perfectly fine, but what is implemented in the Linux
>     > kernel is different: FL is carrying hash from 5-tuple with an additional
>     > seed, and this seed is randomly changed after each RTO/SYN_RTO event.
> 
>     Changing the FL upon RTO is a bug.
> 
>     I guess/assume that when you say SYN-RTO, you really mean "user
>     timeout", rather than RTO. If you don't, then that's also a bug.
>     If you do, I fail to understand what's the reason for wanting the FL to
>     change in that case, because as a result of port randomization, it 0s
>     unlikely that the same four-tuple is employed for the next connection retry.
> 
> 
>     > Here are related patches:
>     >
>     >   * https://lore.kernel.org/netdev/alpine.DEB.2.02.1407012100290.20628@tomh.mtv.corp.google.com/
>     >   * https://lore.kernel.org/netdev/1438124526-2129341-1-git-send-email-tom@herbertland.com/
>     >   * https://lore.kernel.org/netdev/20160928020337.3057238-1-brakmo@fb.com/
>     >
>     >
>     > This is a great thing by the way because in the data center
>     > environment with multiple equal paths it gives a way to have
>     > pseudo-multipath TCP which jumps between paths in case of an outage.
>     > There might be interest to writedown an informational document for this.
> 
>     That's a bad idea, since specs-wise the Flow-Label is not guaranteed to
>     remain unchanged from source to destination. If you want to ahve
>     multiple paths, then you should implement that in routing.
> 
> 
> 
>     > I wonder what you think is a proper solution:
>     >
>     >   * Making FL related RTO change as knob instead of default behavior;
>     >   * Adding negotiation behavior in TCP;
>     >   * Something else?
> 
>     Just make the FL a function of the connection "identifier". And keeo it
>     constant for the lifetime of that conenction.
> 
>     Thanks,
>     -- 
>     Fernando Gont
>     e-mail: fernando@gont.com.ar <mailto:fernando@gont.com.ar> || fgont@si6networks.com <mailto:fgont@si6networks.com>
>     PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
> 
> 
> 
> 
> 
> -- 
> Best regards,
> Alexander Azimov
> 
> _______________________________________________
> v6ops mailing list
> v6ops@ietf.org
> https://www.ietf.org/mailman/listinfo/v6ops
>