Re: ECMP and flow label [Re: 6man w.g. last call for <draft-ietf-6man-segment-routing-header-11.txt>]

"Joel M. Halpern" <jmh@joelhalpern.com> Fri, 30 March 2018 23:57 UTC

Return-Path: <jmh@joelhalpern.com>
X-Original-To: ipv6@ietfa.amsl.com
Delivered-To: ipv6@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B368D127444 for <ipv6@ietfa.amsl.com>; Fri, 30 Mar 2018 16:57:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level:
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=joelhalpern.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0exlUkyGUBRk for <ipv6@ietfa.amsl.com>; Fri, 30 Mar 2018 16:57:31 -0700 (PDT)
Received: from maila2.tigertech.net (maila2.tigertech.net [208.80.4.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EC4771273E2 for <ipv6@ietf.org>; Fri, 30 Mar 2018 16:57:30 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by maila2.tigertech.net (Postfix) with ESMTP id D4132240188; Fri, 30 Mar 2018 16:57:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelhalpern.com; s=2.tigertech; t=1522454250; bh=9A/xxedqBq6g16QC5Isf35H9TxMQF3Fi/uVburVzZkk=; h=Subject:To:References:From:Date:In-Reply-To:From; b=mNnXW+3V8ez0xCEglf4tPSifFMLf4zb9CtUIdJzUr3gnexkinvabXqZ9DWc+7rD56 CqtyCXWjzYFs/GELpnLhThgUwknI9nO7WuYVjIzdemZAt3bOFx0Y2GcjVMX/e+wWxp LEjesJd9nLThz0A5bkDydujdKKyggKtN7yG8AGh0=
X-Virus-Scanned: Debian amavisd-new at maila2.tigertech.net
Received: from Joels-MacBook-Pro.local (unknown [50.225.209.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by maila2.tigertech.net (Postfix) with ESMTPSA id 37AE924017F; Fri, 30 Mar 2018 16:57:30 -0700 (PDT)
Subject: Re: ECMP and flow label [Re: 6man w.g. last call for <draft-ietf-6man-segment-routing-header-11.txt>]
To: Brian E Carpenter <brian.e.carpenter@gmail.com>, ipv6@ietf.org
References: <20160428004904.25189.43047.idtracker@ietfa.amsl.com> <FB1C6E49-81F7-49DD-8E8B-2C0C4735071B@gmail.com> <523d27a3-285e-6bcf-2b07-2cd8d31b0915@joelhalpern.com> <CALx6S34s-XjgwYaiqFJVvPwmmUYD4qPaCv-Ku+h6q1A9aLz9hA@mail.gmail.com> <2a1a2cbe-772b-d13e-67ba-71106d40d61b@joelhalpern.com> <98bb6524-fc47-74f8-4b51-76581e43b825@joelhalpern.com> <3f900fd5-60aa-98e9-a357-f5ece5db1891@gmail.com>
From: "Joel M. Halpern" <jmh@joelhalpern.com>
Message-ID: <067d9434-ff2b-9f10-d8aa-34a28a21f660@joelhalpern.com>
Date: Fri, 30 Mar 2018 19:57:29 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <3f900fd5-60aa-98e9-a357-f5ece5db1891@gmail.com>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/ipv6/3kr33ZiYfA9F38zOaYYkiQePpXA>
X-BeenThere: ipv6@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "IPv6 Maintenance Working Group \(6man\)" <ipv6.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipv6>, <mailto:ipv6-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ipv6/>
List-Post: <mailto:ipv6@ietf.org>
List-Help: <mailto:ipv6-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipv6>, <mailto:ipv6-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 30 Mar 2018 23:57:34 -0000

As far as I can tell, it is not sufficient for nodes originating SRH to 
be compliant with 6437.  In order for this usage to work for SRH 
packets, all IP routers in the domain have to be using the flow label 
ECMP behavior.  (This is one of the insufficiently specified assumptions 
in the draft.)
Given that all routers are applying this behavior, it follows that this 
will be applied to traffic from all sources.

Thus, for example, hosts which send traffic with varying flow labels 
(which apparently exist, even if I have no clue why they do this) will 
have their traffic damaged.

Yours,
Joel

On 3/30/18 6:50 PM, Brian E Carpenter wrote:
> On 31/03/2018 05:38, Joel M. Halpern wrote:
>> I responded a bit too quickly to Tom's note.
>> I agree with the later parts of his note.  It is the ECMP behavior of
>> intermediate routers that cause me concern.
>> Whether his proposed fix is the right one or not is unclear, as the blog
>> post I referred to suggested more issues.  (And I apologize for being
>> unable to find a pointer to the blog post.)
> 
> OK, but if this is a real issue with our RFCs, and not an implementation
> defect, we need to fix it,independently of the current draft.
> 
> More below...
> 
>> Yours,
>> Joel
>>
>> On 3/30/18 12:29 PM, Joel M. Halpern wrote:
>>> Sorry for being a bit vague.  The issue with packet reordering when
>>> using flowlabel for ECMP was discussed in a blog post from a senior
>>> engineer (at a company other than my own) who as far as I know does not
>>> participate in the IETF.  He went through a number of issues in that.
>>>
>>> There are likely ways to work around the problems.  But since the draft
>>> does not even refer to the issue, there is a serious gap.
>>>
>>> Yours,
>>> Joel
>>>
>>> On 3/30/18 11:59 AM, Tom Herbert wrote:
>>>> On Fri, Mar 30, 2018 at 7:51 AM, Joel M. Halpern <jmh@joelhalpern.com>
>>>> wrote:
>>>>> I do not think this document is ready to be sent to the IETF and IESG
>>>>> for
>>>>> final approval.
>>>>> There are several kinds of problems.
>>>>>
>>>>> ECMP1: The document asserts that entropy information is put into the
>>>>> flow
>>>>> label.   I wish this worked.  I helped bring this idea forward years
>>>>> ago,
>>>>> independent of SRH.  Unfortunately, there are two related problems.
>>>>> First,
>>>>> adoption appears to be low.  Second, and more important, there are
>>>>> reports
>>>>> from the field that doing thi actually breaks other things and cause
>>>>> packet
>>>>> re-ordering under some circumstances.  As a result, vendors are
>>>>> suggesting
>>>>> that people turn it off even when it is available.
>>>>
>>>> Joel,
>>>>
>>>> I assume you're referring to:
>>>>
>>>> "If an array of layer-3 adjacencies is bound to the End.X SID, then
>>>> one entry of the array is selected based on a hash of the packet's
>>>> header (at least SA, DA, Flow Label)."
>>>>
>>>> The problem with flow label isn't that it can cause OOO packets, it's
>>>> interaction with stateful techniques in intermediate nodes like
>>>> firewalls and load balancer that require all packets for a flow to
>>>> consistently hit the same stateful device. When the flow label changes
>>>> in mid flow, packets might be routed to a different device that
>>>> doesn't have the flow state and hence are dropped.
>>>>
>>>> I imagine that one could argue that SRH is intended to be used in
>>>> closed domains so the operator could ensure there are no stateful
>>>> nodes present that would have an issue, but that makes correct
>>>> operation of the protocol conditional.
> 
> If it's conditional on "source nodes MUST conform to [RFC6437]",
> would that be a problem?
> 
>>>> IMO the fix for the flow label/ECMP problem needs to include an update
>>>> to RFC6437 specifying 1) the flow label can only be set by the source,
>>>> nodes in the path MUST NOT set it,
> 
> We debated this at great length years ago, and it was explicitly changed
> to the rules in RFC6437, because the previous rule was unrealistic. But
> it defines clear rules about what forwarding nodes may do.
> 
>>>> 2) if the flow label is set, it
>>>> MUST be an entropy identifier of the encapsulated transport flow so
>>>> that (SA, DA, flow label) is a good tuple representation of the flow,
> 
> Wait, "encapsulated"? The flow label refers to the header it occurs in,
> not to an encapsulated header, which it knows nothing about.
> 
> Ignoring this, that is the intention of RFC6437, apart from not misusing the
> word 'entropy'. Please point to any text in RFC6437 that isn't clear enough.
> 
>>>> 3) the flow label MUST be the same for all packets sent on a flow
>>>> including fragments.
> 
> Yes, there is a problem for fragmented packets if the flow label is set by
> a forwarding node, as mentioned at https://tools.ietf.org/html/rfc6437#page-7
> That isn't a problem if the source node sets the flow label, which is of
> course the primary recommendation. (Therefore, it isn't a problem
> in the RFC 6438 scenario, ECMP in tunnels.)
> 
> Should we be adding a BCP to that, requiring this choice:
> 
>        *  A forwarding node might use the 2-tuple to define a flow in all
>           cases.  In this case, subsequent load distribution would be
>           based only on IP addresses.
> ?
>      Brian
> 
>>>> Tom
>>>>
>>>>> ECMP2: If, with suitable analysis, we decide that this is actually
>>>>> safe to
>>>>> do, then the document needs to include both that analysis and clearly
>>>>> spelled out requirements that operators MUST verify that all their
>>>>> routers
>>>>> support this behavior and that they have enabled this behavior before
>>>>> turning on SRv6 usage.
>>>>>
>>>>> Structure: The text in section 2.3 says "It is assumed in this
>>>>> document that
>>>>> the SRH is added to the packet by its source".  This is at best
>>>>> disingenuous.  It is very clear that the value of this behavior lies
>>>>> primarily in use by the network, not by the packet sources.  Claiming
>>>>> otherwise results in a document with minimal utility.  Even this
>>>>> document
>>>>> itself disagrees with this assertion.  Tucked into the very next
>>>>> section is
>>>>> text saying that "outer header with an SRH applied to the incoming
>>>>> packet".
>>>>> If this behavior were a clearly spelled out requirement, rather than a
>>>>> "typically" and if the text in 2.3 were replaced with something
>>>>> realistic,
>>>>> then the document would at least be itnernally consistent and match the
>>>>> expected usages.
>>>>>
>>>>> Edge filtering: The text on edge filtering does not actually state that
>>>>> prevention of packets with SRH and a current DA of an internal node is
>>>>> mandatory.  Unless it is clearly stated, the security considerations
>>>>> text as
>>>>> currently written is significantly weakened.  If it is mandatory,
>>>>> then again
>>>>> the deployment section needs to note that an operator needs to verify
>>>>> that
>>>>> all of his edge devices support such filtering and have it properly
>>>>> enabled
>>>>> in order to use SRv6.
>>>>>
>>>>> Edge Filtering and hybrids: Other documents have talked about allowing
>>>>> external packets with SRv6 entries pointing to internal nodes (which
>>>>> means
>>>>> the DA upon arrival at the operator edge will be an internal node as I
>>>>> understand it).  It the intention is to permit that with appropriate
>>>>> security, then the edge filtering requirements need to be clear about
>>>>> the
>>>>> requirements for cryptographic validation at the edge.
>>>>>
>>>>> Yours,
>>>>> Joel
>>>>>
>>>>>
>>>>> On 3/29/18 4:30 PM, Bob Hinden wrote:
>>>>>>
>>>>>>
>>>>>> This message starts a two week 6MAN Working Group Last Call on
>>>>>> advancing:
>>>>>>
>>>>>>           Title           : IPv6 Segment Routing Header (SRH)
>>>>>>           Authors         : Stefano Previdi
>>>>>>                             Clarence Filsfils
>>>>>>                             John Leddy
>>>>>>                             Satoru Matsushima
>>>>>>                             Daniel Voyer
>>>>>>           Filename       : draft-ietf-6man-segment-routing-header-11.txt
>>>>>>           Pages          : 34
>>>>>>           Date           : 2018-03-28
>>>>>>
>>>>>>         
>>>>>> https://tools.ietf.org/html/draft-ietf-6man-segment-routing-header
>>>>>>
>>>>>> as a Proposed Standard.  Substantive comments and statements of support
>>>>>> for publishing this document should be directed to the mailing list.
>>>>>> Editorial suggestions can be sent to the author.  This last call will
>>>>>> end on 12 April 2018.
>>>>>>
>>>>>> An issue tracker will be setup to track issues raised on this document.
>>>>>>
>>>>>> Thanks,
>>>>>> Bob & Ole
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --------------------------------------------------------------------
>>>>>> IETF IPv6 working group mailing list
>>>>>> ipv6@ietf.org
>>>>>> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
>>>>>> --------------------------------------------------------------------
>>>>>>
>>>>>
>>>>> --------------------------------------------------------------------
>>>>> IETF IPv6 working group mailing list
>>>>> ipv6@ietf.org
>>>>> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
>>>>> --------------------------------------------------------------------
>>>
>>> --------------------------------------------------------------------
>>> IETF IPv6 working group mailing list
>>> ipv6@ietf.org
>>> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
>>> --------------------------------------------------------------------
>>>
>>
>> --------------------------------------------------------------------
>> IETF IPv6 working group mailing list
>> ipv6@ietf.org
>> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
>> --------------------------------------------------------------------
>>
> 
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> ipv6@ietf.org
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>