Re: [v6ops] Flow Label Load Balancing

[Tom Herbert <[e-mail]>]

Joel, is there a normative definition of a flow?

On Wed, Nov 25, 2020, 1:27 PM Joel M. Halpern <[e-mail]> wrote:

> No, as Brian says, there are escape clauses in the flow definitions.
>
> But changing the flow label due to traffic problems does not correspond
> to packets being in actually different flows.
> If one were using UDP, and mixing loss sensitive packets with loss
> insensitive packets for a special application, sure, one could use two
> flow labels.  But that is not what you are describing.
>
> Yours,
> Joel
>
> On 11/25/2020 3:05 PM, Tom Herbert wrote:
> > Joel,
> >
> > Is there an RFC that clearly and unambiguously states that a host MUST
> > use the same flow label for the lifetime _and_ clearly defines exactly
> > what a flow is with respect to such a requirement (for instance, how
> > would you define a flow and enforce such a requirement in UDP? IPsec?
> > other encapsulations?). If there is such a requirement then we'll
> > change the code to be conformant.
> >
> > Tom
> >
> > On Wed, Nov 25, 2020 at 12:39 PM Joel M. Halpern <[e-mail]>
> wrote:
> >>
> >> This kind of thing is why, as I understand it, MPTCP has discovery
> >> mechanisms ot know if both sides use it, and can select alternative
> >> addresses for communication.
> >>
> >> Trying to guess flow labels that might avoid a problem because it might
> >> be an ECMP problem, is just flailing about.  Not a good design for
> >> operational protocols.
> >>
> >> And in general, designing protocols around "I know exactly what is going
> >> on"  (the requirement for what you describe that goes well beyond just
> >> "limited domains") is also a recipe for failure.
> >>
> >> The Flow Label RFCs are actually very explicit that a flow label is
> >> supposed to be stable for the life of the flow.  Otherwise, it isn't a
> >> flow label.
> >>
> >> Yours,
> >> Joel
> >>
> >> On 11/25/2020 2:35 PM, Tom Herbert wrote:
> >>> Hi Fernando, comments in line...
> >>>
> >>> On Wed, Nov 25, 2020 at 12:13 AM Fernando Gont <[e-mail]>
> wrote:
> >>>>
> >>>> Hi, Tom,
> >>>>
> >>>> On 24/11/20 16:43, Tom Herbert wrote:
> >>>> [....]
> >>>>> Modulating the flow label is a means to affect the routing of packets
> >>>>> through the network that uses flow labels as input to the ECMP hash.
> >>>>
> >>>> What's the point?
> >>>>
> >>>> 1) You cannot tell *if* the FL is being used.
> >>>>
> >>> Generally true, but in a limited domain this information could be
> >>> discerned. I'd note that it's also generally true that we don't know
> >>> if there is a load balancer or stateful firewall in the path that
> >>> requires consistent routing, but in a limited domain we could know
> >>> that also.
> >>>
> >>>> 2) Changing the FL does not necessarily mean that packets will employ
> a
> >>>> different link.
> >>>
> >>> It's an opportunistic mechanism. If a connection is failing and we get
> >>> a better path that fixes it by simply changing the flow label then
> >>> what's the harm?
> >>>
> >>>>
> >>>> 3) If the network is failing, shouldn't you handle this via routing?
> >>>>
> >>> Sure, but then that requires an out of band feedback loop from a TCP
> >>> implementation to the network infrastructure to indicate there is a
> >>> problem and then the network needs to respond. That's significant
> >>> infrastructure and higher reaction time than doing something in TCP
> >>> and IP. Think of modulating the flow label is an inexpensive form of
> >>> source routing within a limited domain that doesn't need any
> >>> infrastructure or heavyweight protocols or something like segment
> >>> routing.
> >>>
> >>>>
> >>>>
> >>>>> The basic idea is that the flow label associated with a connection is
> >>>>> randomly changed when the stack observes that the connection is
> >>>>> failing (e.g. and an RTO). There is nothing in the specs that
> prevents
> >>>>> this since the source is at liberty to set the flow label as it sees
> >>>>> fit.
> >>>>
> >>>> The FL is expected to remain constant for the life of a flow. A
> >>>> retransmitted packet is part of the same flow as the
> >>>> originally-transmitted packet. So this seems to be contradicting the
> >>>> very specification of the FL.
> >>>>
> >>>> For instance, If a RTO for a flow causes the FL to change, then one
> may
> >>>> possibly argue that the FL is not naming/labeling what is
> said/expected
> >>>> to be anming/labeling.
> >>>
> >>> Specifically, RFC6437 states:
> >>>
> >>> "It is therefore RECOMMENDED that source hosts support the flow label
> >>> by setting the flow label field for all packets of a given flow to the
> >>> same value chosen from an approximation to a discrete uniform
> >>> distribution."
> >>>
> >>> So that is clearly a just recommendation, and not a requirement (and
> >>> definitely not a MUST). Furthermore, RFC6437 states:
> >>>
> >>> "A forwarding node MUST either leave a non-zero flow label value
> >>> unchanged or change it only for compelling operational security
> >>> reasons as described in Section 6.1."
> >>>
> >>> So there's no guarantee in the protocol specs that flow labels are
> >>> consistent for the life of the connection, which means that the
> >>> network cannot assume that and thus it would be incorrect if the
> >>> network tried to enforce flow label consistency as a protocol
> >>> requirement. As I said, it is prudent to try to be consistent with
> >>> flow labels and the default behavior in Linux should be changed,
> >>> however I do not believe there's a valid claim of non-conformance that
> >>> motivates removal of the feature that is already deployed.
> >>>
> >>> Tom
> >>>
> >>>
> >>>
> >>>
> >>>>
> >>>>
> >>>>
> >>>>> The feature is useful in large datacenter networks, like
> >>>>> pparently Facebook where the patches originate, since information
> >>>>> discerned by TCP can opportunistically be applied to route selection.
> >>>>> The practical issue is that there are stateful devices like firewalls
> >>>>> that require consistent routing in the network in which case changing
> >>>>> the flow label can confuse them. As I mentioned, the original intent
> >>>>> was that the flow label randomization feature should be opt-in
> instead
> >>>>> of on by default.
> >>>>
> >>>> So... where is the "source" of the packet that would be "modulating"
> the FL?
> >>>>
> >>>> Thanks,
> >>>> --
> >>>> Fernando Gont
> >>>> e-mail: [e-mail] || [e-mail]
> >>>> PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
> >>>>
> >>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> v6ops mailing list
> >>> [e-mail]
> >>> https://www.ietf.org/mailman/listinfo/v6ops
> >>>
>