Re: [v6ops] Flow Label Load Balancing

[Tom Herbert <[e-mail]>]

On Fri, Nov 27, 2020 at 1:41 PM Fernando Gont <[e-mail]> wrote:
>
> Hello, Tom,
>
> On 25/11/20 16:35, Tom Herbert wrote:
> > Hi Fernando, comments in line...
> >
> > On Wed, Nov 25, 2020 at 12:13 AM Fernando Gont <[e-mail]> wrote:
> >>
> >> Hi, Tom,
> >>
> >> On 24/11/20 16:43, Tom Herbert wrote:
> >> [....]
> >>> Modulating the flow label is a means to affect the routing of packets
> >>> through the network that uses flow labels as input to the ECMP hash.
> >>
> >> What's the point?
> >>
> >> 1) You cannot tell *if* the FL is being used.
> >>
> > Generally true, but in a limited domain this information could be
> > discerned. I'd note that it's also generally true that we don't know
> > if there is a load balancer or stateful firewall in the path that
> > requires consistent routing, but in a limited domain we could know
> > that also.
>
> Exactly. So my take is that the drawbacks for the general case outweigh
> the benefits of the specific case.
>
>
>
> >> 2) Changing the FL does not necessarily mean that packets will employ a
> >> different link.
> >
> > It's an opportunistic mechanism. If a connection is failing and we get
> > a better path that fixes it by simply changing the flow label then
> > what's the harm?
>
> Complexity, and the possible negative implications for the general case.
>
> That said, why don't you fix the problem where you should (i.e., routing)?
>
>
>
> >> 3) If the network is failing, shouldn't you handle this via routing?
> >>
> > Sure, but then that requires an out of band feedback loop from a TCP
> > implementation to the network infrastructure to indicate there is a
> > problem and then the network needs to respond.
>
> Not really. TCP does not deal with paths (not should it). If the network
> detects a link is failing, it should route around the problem.
>

Fernando,

All we know in this case is that *TCP* has detected a connection is
failing and there may be a potential problem with a path, we don't
know that the network has detected any problems like some link is
failing or will ever detect any problem. So in such a scenario, what
recourse does the host have to try to salvage its connections? If the
answer is that the host should always patiently wait for the network
to figure out what is happening, then that's going to be a hard sell
to application and host stack developers (remember the network is a
black box from their perspective that they really don't trust). If the
answer is that the host is supposed to inform the network of issues
with it's connections and then host and network work together to solve
then that's great; but then what is the general protocol that has been
established to facilitate that and solved the problem of how hosts and
network work together in concert to solve user's problems?

Tom

>
>
> > That's significant
> > infrastructure and higher reaction time than doing something in TCP
> > and IP. Think of modulating the flow label is an inexpensive form of
> > source routing within a limited domain that doesn't need any
> > infrastructure or heavyweight protocols or something like segment
> > routing.
>
> Downsides:
>
> * It breaks load balancing
> * It depends on the source doing something -- but you don't necessarily
> control the source -- and you cannot tell if the source implements it
> * It doesn't solve the problem for nodes that do not behave as expected
>
> My take is that if you really care about this, you're already (or
> should!) be solving the problem in a different (and more general) way.
>
>
>
> >>> The basic idea is that the flow label associated with a connection is
> >>> randomly changed when the stack observes that the connection is
> >>> failing (e.g. and an RTO). There is nothing in the specs that prevents
> >>> this since the source is at liberty to set the flow label as it sees
> >>> fit.
> >>
> >> The FL is expected to remain constant for the life of a flow. A
> >> retransmitted packet is part of the same flow as the
> >> originally-transmitted packet. So this seems to be contradicting the
> >> very specification of the FL.
> >>
> >> For instance, If a RTO for a flow causes the FL to change, then one may
> >> possibly argue that the FL is not naming/labeling what is said/expected
> >> to be anming/labeling.
> >
> > Specifically, RFC6437 states:
> >
> > "It is therefore RECOMMENDED that source hosts support the flow label
> > by setting the flow label field for all packets of a given flow to the
> > same value chosen from an approximation to a discrete uniform
> > distribution."
> >
> > So that is clearly a just recommendation, and not a requirement (and
> > definitely not a MUST). Furthermore, RFC6437 states:
> >
> > "A forwarding node MUST either leave a non-zero flow label value
> > unchanged or change it only for compelling operational security
> > reasons as described in Section 6.1."
>
> My concern is that we may leave the FL unusable.
>
>
>
>
> > So there's no guarantee in the protocol specs that flow labels are
> > consistent for the life of the connection, which means that the
> > network cannot assume that and thus it would be incorrect if the
> > network tried to enforce flow label consistency as a protocol
> > requirement.
>
> You're probably right in that respect. And, in retrospective, I think it
> would have been better if we had specified the FL to be unmutable.
>
>
>
> > As I said, it is prudent to try to be consistent with
> > flow labels and the default behavior in Linux should be changed,
>
> Sorry: is the current Linux behavior to modulate the FL?
>
>
> > however I do not believe there's a valid claim of non-conformance that
> > motivates removal of the feature that is already deployed.
>
> I personally consider the changing the FL for a flow to be a bug. That
> said, if the default setting is the correct/compliant behavior, at the
> end of the day we're all adults, and whoever overrides the default
> behavior is supposed to know what he/she does -- and if they don't, they
> get what they deserve  ;-)
>
> So, in that light, I wouldn't push to remove the optional behavior,
> *provided* the default behavior is correct.
>
> Thanks,
> --
> Fernando Gont
> e-mail: [e-mail] || [e-mail]
> PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
>
>
>