Re: Pseudorandom Flow Labels

Shane Amante <shane@castlepoint.net> Tue, 05 April 2011 21:21 UTC

Return-Path: <shane@castlepoint.net>
X-Original-To: ipv6@core3.amsl.com
Delivered-To: ipv6@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6CF7B28C0E2 for <ipv6@core3.amsl.com>; Tue, 5 Apr 2011 14:21:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.554
X-Spam-Level:
X-Spam-Status: No, score=-2.554 tagged_above=-999 required=5 tests=[AWL=0.045, BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FN92Ns6IkRfA for <ipv6@core3.amsl.com>; Tue, 5 Apr 2011 14:21:58 -0700 (PDT)
Received: from dog.tcb.net (dog.tcb.net [64.78.150.133]) by core3.amsl.com (Postfix) with ESMTP id 4B2813A67C1 for <ipv6@ietf.org>; Tue, 5 Apr 2011 14:21:58 -0700 (PDT)
Received: by dog.tcb.net (Postfix, from userid 0) id 83B1F268676; Tue, 5 Apr 2011 15:23:41 -0600 (MDT)
Received: from host2.tcb.net (64.78.235.218 [64.78.235.218]) (authenticated-user smtp) (TLSv1/SSLv3 AES128-SHA 128/128) by dog.tcb.net with SMTP; Tue, 05 Apr 2011 15:23:41 -0600 (MDT) (envelope-from shane@castlepoint.net)
X-Avenger: version=0.7.8; receiver=dog.tcb.net; client-ip=64.78.235.218; client-port=60710; data-bytes=0
Subject: Re: Pseudorandom Flow Labels
Mime-Version: 1.0 (Apple Message framework v1084)
Content-Type: text/plain; charset=us-ascii
From: Shane Amante <shane@castlepoint.net>
In-Reply-To: <201104051958.p35JwaJP019044@cichlid.raleigh.ibm.com>
Date: Tue, 5 Apr 2011 15:23:40 -0600
Content-Transfer-Encoding: quoted-printable
Message-Id: <92DAE8F3-471F-4331-8882-3788631959F5@castlepoint.net>
References: <BD901061-96AC-4915-B7CE-2BC1F70861A5@castlepoint.net> <201104051958.p35JwaJP019044@cichlid.raleigh.ibm.com>
To: Thomas Narten <narten@us.ibm.com>
X-Mailer: Apple Mail (2.1084)
Cc: 6man List <ipv6@ietf.org>
X-BeenThere: ipv6@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: "IPv6 Maintenance Working Group \(6man\)" <ipv6.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/ipv6>, <mailto:ipv6-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ipv6>
List-Post: <mailto:ipv6@ietf.org>
List-Help: <mailto:ipv6-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipv6>, <mailto:ipv6-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Apr 2011 21:21:59 -0000

Thomas,

On Apr 5, 2011, at 13:58 MDT, Thomas Narten wrote:
[--snip--]
> I take it as a given that doing ECMP on the src/dst address gets you
> 80% of what you need today. Adding in the Flow Label (if set) will
> take you much further. I am not convinced you need real "pseudo
> randomness" in the Flow Label to get the benefits being called for.

As I pointed out below, we already have pseudorandom distribution of flows today, given that most host implementations use RFC 6056 and (IPv4) load-balancing routers/switches use the 5-tuple as input-keys ... included in those input-keys is a "pseudorandom ephemeral port".  In short, if you remove the pseudo-random requirement from flow-labels, (and I can only load-balance on just the 3-tuple), I guarantee I will end up with less efficient load-balancing.

Second, the economic results of what you suggest would be bad.  IOW, operators need to pack all links, in a LAG and/or ECMP group, as _efficiently_ as (reasonably) possible, particularly over Inter-City and especially over Inter-Continental links where costs are substantial.  In summary, "good enough" is not adequate, hence the reqm't for pseudorandom-ness in flow-labels (originated by hosts!) to provide the a very, very high probability that flows will be "uniformly distributed".


>> 2) If we expect that if a intermediate router or switch is using
>> *just* the 3-tuple of {src_ip, dst_ip + flow_label} as input-keys to
>> compute a load-balancing hash algorithm, then the more random the
>> flow-label, the better load-distribution of /all/ traffic will be
>> across the LAG and/or ECMP paths.
> 
> Understood. But what you need for this is not (necessarily) pseudo
> randomness. Just sufficient variability (when combined with the
> src/dst) to get good distribution on the output side of the hash.

I'll state it a bit differently: I need flow-labels as close to the [pseudo-random] properties provided by RFC 6056.


>> a) As a network operator I have no control over the number of IP
>>     addresses used by content farms or residential networks.
>>     Furthermore, it's not clear that as more & more machines (and,
>>     smartphones!) ship that support multiple CPU's and/or multiple
>>     cores that each of those discrete computing units will (or,
>>     should?) receive their own IP address.  With IPv6 that's at
>>     least a possibly, but perhaps due to "ease of development",
>>     that won't happen often, or at all.  If we strongly recommend a
>>     pseudo-random flow-label and a device is only capable of
>>     load-balancing using a 3-tuple, then I've at least got a
>>     [relatively] unique flow-label in the packet header to provide
>>     load distribution.
> 
> I think we can assume that if we use both the src/dst, we will get a
> good degree of distribution in the values. Adding the Flow Label gives
> more. I am just not convinced that to get good distribution we need to
> *require* (or strongly suggest) psuedo randomness in the Flow
> Lable. We know that by simply incrementing the Flow Label by 1 for
> each flow, we get sufficient distribution. That is *way* easier to
> implement than something else.

Unfortunately, if:
a)  All hosts at a site start off selecting "1" for their flow-labels; or,
b)  Don't cycle through the full-range of flow-label values; or,
c)  a [large] set of hosts experience synchronization of flow-label values; or,
d)  Reset back to flow-label "1" after some short time period
... then you nullify the claim of having "sufficient distribution" of flow-labels and, consequently, flows.  Instead, if we tell implementers to select a pseudorandom value, using an O/S and/or API's random() function, we're much more likely to get "sufficient distribution".


>> b) The reason that we observe in today's production networks very
>>     good load-distribution over LAG and/or ECMP paths is most
>>     likely the result of, first, hosts selecting a pseudo-random
>>     value for their ephemeral ports (based on RFC 6056) and,
>>     second, the ability for intermediate routers/switches to use
>>     the traditional 5-tuple for input-keys for load-distribution on
>>     those paths.  Since we would like to move in a direction of
>>     LAG/ECMP load-balancing based on just the 3-tuple of {src_ip,
>>     dst_ip, flow_label}, then we should not take a step backwards
>>     from where we are today wrt pseudorandom-ness of individual
>>     flows.
> 
> This takes me back to an earlier point. Then suggest that if the port
> numbers are pseudo random, provide me with a simple algorithm for
> mapping that into a good Flow Label value.

draft-gont-6man-flowlabel-security-01?  Which is already referenced from draft-ietf-6man-flow-3697bis-02.


>> 3) Finally, if we expect that the flow-label may (or, hopefully
>> will) get used as a lightweight method of detecting and, possibly,
>> preventing 3rd-party DoS or traffic injection attacks, (i.e.:
>> draft-gont-6man-flowlabel-security), then it depends on generation
>> of pseudo-random values for flow-labels in order that off-path
>> attackers have a reasonably low chance of guessing a valid
>> flow-label.
> 
> I'm less convinced that there are real significant DOS attacks that
> using a pseudo-random flow label can address.
> 
> Now. What about TEPs? They have no way of knowing whether the port
> numbers being used provide proper randomness. That implies if they are
> going to generate pseudo-random Flow Labels, they have a *lot* more
> work to do. And to be sure that all packets from a given "flow" are
> given the same Flow Label, they may have to maintain state. i.e., so
> that subsequent packets from the same flow get assigned the same Flow
> Label value. I doubt you are suggesting that.

You are correct.


> But the current
> documents leave this all to the reader. Again, I think some concrete
> recommendations are in order. If you think TEPs will, in fact, just
> produce a Flow Label value based on whatever ports the packets being
> tunneled contain, then just say so and be done with it.

OK.  I think I understand where you are coming from with respect to draft-ietf-6man-flow-ecmp-01.  Hopefully the above comment(s) will allow us to craft text that will resolve your concerns with that draft once and for all.  However, that still leaves draft-ietf-6man-flow-3697bis-02 open for discussion.


> But don't
> require that they produce "pseudo random" values if in fact, we are
> pretty sure TEPs won't actually implement this.

Understood.

-shane