Re: [spring] “SRV6+” complexity in forwarding

Mark Smith <markzzzsmith@gmail.com> Thu, 19 September 2019 04:51 UTC

Return-Path: <markzzzsmith@gmail.com>
X-Original-To: ipv6@ietfa.amsl.com
Delivered-To: ipv6@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DDF3212011E; Wed, 18 Sep 2019 21:51:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.498
X-Spam-Level:
X-Spam-Status: No, score=-0.498 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tsPRw_K4Usjk; Wed, 18 Sep 2019 21:50:59 -0700 (PDT)
Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com [IPv6:2607:f8b0:4864:20::333]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7E5D01200A3; Wed, 18 Sep 2019 21:50:59 -0700 (PDT)
Received: by mail-ot1-x333.google.com with SMTP id 21so1893314otj.11; Wed, 18 Sep 2019 21:50:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=9PmxuuBJL4LriW8TXpGTHpjsGdCnWskSskE+m1Lb5z0=; b=InwE6aJgQvOH8GdCGmqcRKT606LOlKp3vZZSZT7NJjN5IOu0M5Plc3ZrJVqMdkJbsD gt8/aaftBdqQNRed9Rx58i7yEdyMVHkqRttdP/jXMlMOmel6rZIGqqkR/RKOOl72qn/p VYTw5vsjx7Oqj3h58lkKvu2W1DYybvnyMPmsJ6y7nIb8GEE/x/gctypagy0VSp4LeJXR TnO0CvSsPPU3WLZuc2+t9dlcsYjrZNMS/5M4DMB1NoeW1qUpulpuLh1tcFy6/kVPypua 3sVR4jOkKBWYMs59erCwJxf68Rr3mA1+rfFQm6It1hLFidBLsmmwyqpLap92rH/v7mVh ED6g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=9PmxuuBJL4LriW8TXpGTHpjsGdCnWskSskE+m1Lb5z0=; b=PkCcS+MnmWgmSruz3d+9buLvseeGJMYCejXNd3w7xsm0p6x9jUC4gjbbs+Ky3Q21a4 ZwDyn6zSj555S7x7y2uJIEo8DSeY16WfdGKjHcroJGhBxy1Rs0ed4KpaVrMMAt7oCzEP jKz3JYJtAx/Te1hcUHERghYc2gf6gITS7PZ9vzpfT+FgikSDI5ST4+1VCMTEYqpePVpF YEFebd+H3hlMW9LdVV3niUxRInsuViLdVOPyPBRSggRj1oDSYY4dunq7er5nBNEp+W/i tg/17Kb0TMrWOBUhsA6oWE0Kzomi5tU+lq8Bf/PuAIneUFvD/5akMqmhBIp/zkT35xLV wXQQ==
X-Gm-Message-State: APjAAAWC0VVW3Qh5iv+y+jNcYHN/R7nEdsQ7MBjZUgmbBnPHVnrBj5K+ EDf8ZKkb2W3V7mcrOBsmLWozz/UJ9yT2KJ2eLBk=
X-Google-Smtp-Source: APXvYqyWKbG1FK2M6nKb+pvrHIJUkWuygPboerra1HcQAU8yNcViytbt36+B/KPYYwSwJ9F9koKTg1XPiCXiqImqV9c=
X-Received: by 2002:a05:6830:18b:: with SMTP id q11mr5485462ota.94.1568868658819; Wed, 18 Sep 2019 21:50:58 -0700 (PDT)
MIME-Version: 1.0
References: <CAHd-QWtA21+2Sm616Fnw0D-eB7SNb_BeG8-A-MCLLFgTwSpOsg@mail.gmail.com> <BYAPR05MB54632F09C712ADB30138CFA9AEBE0@BYAPR05MB5463.namprd05.prod.outlook.com> <BYAPR19MB3415D21403394F8129A4BAD8FCB90@BYAPR19MB3415.namprd19.prod.outlook.com> <30491F13-C652-45C3-AB2B-95F765FBB4EA@juniper.net> <65C5CB04-3A2F-4F83-A7C8-2045154F93AE@cisco.com> <BYAPR05MB5463EC3250F2A303A3641839AEBA0@BYAPR05MB5463.namprd05.prod.outlook.com> <91CBADAD-EFE6-46E1-A9D3-DAA111357179@juniper.net> <CAOj+MMGyUFRPDqCBo5SbLX486o_9GLpM6Zxf8KSt1voWiqhkGQ@mail.gmail.com> <E8D473B5-3E8D-4339-9A79-0CAE30750A55@juniper.net> <CAOj+MMFOy5PyTo=jPJkVrQOctdWjsTbD=7ix-2n89vodKzT3gQ@mail.gmail.com> <2F604D74-51CF-4F2F-AEA9-1CBDEEA9B9F7@gmail.com> <F09C2D09-D769-4817-AF73-97D6ED1BC4BF@lapishills.com> <201909120857387140042@chinatelecom.cn> <1568259664564.62561@bell.ca> <CAO42Z2wQ_8GEE+=nAMFBj+ape9Vf7fARVoOwGdCiUxdffkyXgw@mail.gmail.com> <BYAPR05MB5463A04B05B4BD6AA294F7F0AEB00@BYAPR05MB5463.namprd05.prod.outlook.com> <6EA6F7C0-BEB2-4749-A6AB-62B1337213B2@cisco.com> <BYAPR05MB5463426F1668202EE5F183EFAE8F0@BYAPR05MB5463.namprd05.prod.outlook.com> <634900D2-FBCE-47CF-8907-C8B9CB3A4102@cisco.com> <CALx6S34=Tw-u4Hz-07-Rs-GjsungkqnD_fMoQnGc17u3VJhY1g@mail.gmail.com> <CAFqxzqYr7g2jzwJrhvjL_DXYZsDzbzqx01cy0zB1aBweDde1XQ@mail.gmail.com> <CAO42Z2yrjwRMykWxmEo5=18fMvuZdMtuyz5g1p=8oSzp_ro9Vw@mail.gmail.com> <CALx6S37cC7wT+=x2e=YD-7O+s2H5a-ksMpRGxwL_Wf_Qb2cD4A@mail.gmail.com>
In-Reply-To: <CALx6S37cC7wT+=x2e=YD-7O+s2H5a-ksMpRGxwL_Wf_Qb2cD4A@mail.gmail.com>
From: Mark Smith <markzzzsmith@gmail.com>
Date: Thu, 19 Sep 2019 14:50:32 +1000
Message-ID: <CAO42Z2yo79HG_a0eXucQ=BxR3FSREYayFm0OUyyWQqk+LdsN_A@mail.gmail.com>
Subject: Re: [spring] “SRV6+” complexity in forwarding
To: Tom Herbert <tom@herbertland.com>
Cc: Dirk Steinberg <dirk@lapishills.com>, Rob Shakir <robjs@google.com>, SPRING WG <spring@ietf.org>, 6man <6man@ietf.org>, "Darren Dukes (ddukes)" <ddukes@cisco.com>, Robert Raszuk <robert@raszuk.net>, Ron Bonica <rbonica@juniper.net>, "xiechf@chinatelecom.cn" <xiechf@chinatelecom.cn>, Tarek Saad <tsaad.net@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/ipv6/zyCQqp1QT-rEI40MDqxQ-ZFujZk>
X-BeenThere: ipv6@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "IPv6 Maintenance Working Group \(6man\)" <ipv6.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipv6>, <mailto:ipv6-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ipv6/>
List-Post: <mailto:ipv6@ietf.org>
List-Help: <mailto:ipv6-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipv6>, <mailto:ipv6-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 19 Sep 2019 04:51:02 -0000

On Thu, 19 Sep 2019 at 11:16, Tom Herbert <tom@herbertland.com> wrote:
>
> On Wed, Sep 18, 2019 at 5:33 PM Mark Smith <markzzzsmith@gmail.com> wrote:
> >
> >
> >
> > On Thu, 19 Sep 2019, 09:40 Dirk Steinberg, <dirk@lapishills.com> wrote:
> >>
> >> SRv6 does not require TLV processing for normal forwarding (use case: SP core).
> >
> >
> > +1
> >
> > The Internet scales because complexity is pushed towards the edges.
> >
> Mark,
>
> I agree, but that begs the question as to why SRV6 contains its own
> TLVs and authentication as opposed to using edge mechanisms like
> destination options after the routing header or AH.

Yes, agree entirely.

I think it might be a mindset thing.

If hosts and routers are imagined as devices, then it might seem
necessary to invent new mechanisms to perform e.g. authentication when
a packet crosses a router.

Once your mindset changes to hosts and routers being functions, and
you use the RFC 8200 definitions of those terms, it then becomes much
clearer that host mechanisms like AH and Dest Options can be used "in
the network", because a router as a device is performing both RFC 8200
routing and host functions.

It's a bit of mental hurdle to cross, however X-in-IPv6 tunnel
end-points are hosts, because they originate new packets (the outer
header), they're the ultimate destination of those packets, and the SA
and DA in the outer packet header are those of the devices at the ends
of the tunnel.

Once a tunnel end-point is viewed as a host, then all of the host
options become available to use e.g. AH/ESP EHs, Destination Options,
UDP/TCP etc.

> The obvious intent
> of putting TLVs in the routing header is that some intermediate
> destinations will process them. So then these "some" intermediate
> nodes will sustain the performance hit of TLV processing. If this is
> so bad as expected that TLVs are relegated to the slow path, then
> nobody turns them on because of the performance hit and the mechanism
> likely becomes yet another extensibility mechanism with the same
> demise as IPv4 options. History shows again and again that
> extensibility mechanisms are "use it or lose it". Requiring an
> extensibility mechanism is a good force function and motivation for
> implementers to make the mechanism work and be feasible.
>

I think the thing that has happened in fairly recent times is that the
"slow path" has become much faster, and by the sounds of it can be
made faster using ASICs for specific processing. So it's now possible
to "slow path" process packets at a fast enough rate and in enough of
a volume that it becomes feasible to do so.

I guess that HbH processing wasn't supported or enabled in the past
for a few reasons - customers didn't ask vendors to provide it,
because customers didn't have a use for it, and there was the
potential for a DoS on a control plane as past control planes were
fairly resource limited.

Regards,
Mark.


> Tom
>
> >>
> >> - Dirk
> >>
> >> On Wed, Sep 18, 2019 at 5:57 PM Tom Herbert <tom@herbertland.com> wrote:
> >>>
> >>> On Wed, Sep 18, 2019 at 6:42 AM Darren Dukes (ddukes) <ddukes@cisco.com> wrote:
> >>> >
> >>> > Hi Ron.
> >>> >
> >>> > I summarized my argument as follows:
> >>> > "Regardless of ASIC capabilities there are two performance penalties you will not escape with PSSI+CRH+PPSI: TLV parsing and multiple lookups.”
> >>> >
> >>> > You’ve confirmed this additional overhead for "SRv6+".  Thanks.
> >>> >
> >>>
> >>> Darren,
> >>>
> >>> How does one escape the performance penalty of TLV processing in SRV6?
> >>>
> >>> Tom
> >>>
> >>>
> >>> > You then say "So long as the ASIC can process enough packets per second to saturate the line cards, we are forwarding at full line rate."
> >>> >
> >>> > Yes this is true, but we can conclude: The complexity of "SRv6+" requires ASICs do much more work per packet vs SRv6.
> >>> >
> >>> > Thanks
> >>> >   Darren
> >>> >
> >>> >
> >>> > On Sep 16, 2019, at 9:59 PM, Ron Bonica <rbonica@juniper.net> wrote:
> >>> >
> >>> > Hi Darren,
> >>> >
> >>> > I think that your argument can be summarized as follows:
> >>> >
> >>> >
> >>> > SRv6 requires only two FIB searches
> >>> > SRv6+ requires 4 or more FIB searches
> >>> > Therefore, SRv6+ cannot possibly forward at line speed
> >>> >
> >>> >
> >>> > Have I summarized your argument correctly? If not, please set me straight. If so, please read on.
> >>> >
> >>> > First, SRv6+ never requires more than 4 FIB searches. The DOH that precedes the CRH contains, at most, one PSSI. Therefore SRv6+ requires four FIB searches, at most.
> >>> >
> >>> > Second, SRv6+ only requires 4 FIB searches the following case:
> >>> >
> >>> >
> >>> > The packet contains two instances of the DOH. (Most use-cases require only one.)
> >>> > The processing node is configured to process the PSSI. (Many ASIC-based devices, because of their role in the network, won’t support any per segment service instructions. This nodes will be configured to ignore the PSSI. That is why it is optional.)
> >>> >
> >>> >
> >>> > So, in most use-cases, SRv6+ requires only 3 FIB searches.
> >>> >
> >>> > So, you might now argue that:
> >>> >
> >>> >
> >>> > SRv6 requires only two FIB searches
> >>> > SRv6+ requires three and sometimes four FIB searches
> >>> > Therefore, SRv6+ cannot possibly forward at line speed
> >>> >
> >>> >
> >>> > Here, some slightly deeper thought might be required. A platform has two relevant resources:
> >>> >
> >>> >
> >>> > A route lookup ASIC, that can process some number of packets per second
> >>> > Some number of interfaces, that can forward some number of bits per second
> >>> >
> >>> >
> >>> > So long as the ASIC can process enough packets per second to saturate the line cards, we are forwarding at full line rate. So long as a platform has a sufficiently capable ASIC, it will be able to forward at line speed. But it’s a matter of how the platform is designed. If the ASIC is not sufficiently capable, of course, it will not forward at line speed.
> >>> >
> >>> > In your email, you say that I have been asked several times to report on the state of Juniper’s SRv6+ implementation. While I cannot provide details, you can assume that we wouldn’t be working on this if we thought that performance was going to be sub-optimal.
> >>> >
> >>> > You also suggest that Juniper’s is the only implementation. Are you sure that this is correct?
> >>> >
> >>> >                                                                                                                      Ron
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > Juniper Business Use Only
> >>> > From: Darren Dukes (ddukes) <ddukes@cisco.com>
> >>> > Sent: Monday, September 16, 2019 4:38 PM
> >>> > To: Ron Bonica <rbonica@juniper.net>
> >>> > Cc: Mark Smith <markzzzsmith@gmail.com>; EXT - daniel.bernier@bell.ca <daniel.bernier@bell.ca>; xiechf@chinatelecom.cn; SPRING WG <spring@ietf.org>; 6man <6man@ietf.org>; Robert Raszuk <robert@raszuk.net>; Rob Shakir <robjs@google.com>; Tarek Saad <tsaad.net@gmail.com>
> >>> > Subject: “SRV6+” complexity in forwarding
> >>> >
> >>> > Hi Ron, I agree ASICs are always improving, indeed this is evident in the number of successful SRv6 deployments and multiple vendor implementations at line rate on merchant silicon, and multiple vendor ASICs.
> >>> >
> >>> > Is “SRv6+” (PSSI+CRH+PPSI) implemented and deployed at line rate?
> >>> > You’ve been asked this several times.  Since you’re the only implementor(?) and one operator is claiming deployment or testing, I am curious.
> >>> >
> >>> > Regardless of ASIC capabilities there are two performance penalties you will not escape with PSSI+CRH+PPSI: TLV parsing and multiple lookups.
> >>> >
> >>> > Requiring all segments in a CRH segment list to process an arbitrary length DOH+set of PSSI’s and other options is always very expensive.
> >>> > - It is expensive in SRAM as previously discussed in these threads.
> >>> > - It is expensive in parsing logic to know and process a set of TLVs in any ASIC or NP.
> >>> >
> >>> > Spreading PSSI, CRH, PPSI operations in multiple headers and multiple identifiers you now have multiple lookups at a node.
> >>> > 1 - lookup destination address
> >>> > 2 - lookup one or more PSSI and future destination options.
> >>> > 3 - lookup the CRH label or PPSI label.
> >>> > 4 - lookup new destination address
> >>> >
> >>> > Compare this with SRv6.
> >>> > 1 - lookup destination address
> >>> > 2 - lookup new destination address
> >>> >
> >>> > While ASICs are more capable and will continue to be more capable, these technical performance problems you introduce with PSSI+CRH+PPSI will not go away.
> >>> >
> >>> > Darren
> >>> >
> >>> >
> >>> >
> >>> > On Sep 12, 2019, at 12:34 PM, Ron Bonica <rbonica=40juniper.net@dmarc.ietf..org> wrote:
> >>> >
> >>> >
> >>> > --------------------------------------------------------------------
> >>> > IETF IPv6 working group mailing list
> >>> > ipv6@ietf.org
> >>> > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> >>> > --------------------------------------------------------------------
> >>>
> >>> _______________________________________________
> >>> spring mailing list
> >>> spring@ietf.org
> >>> https://www.ietf.org/mailman/listinfo/spring
> >>
> >> _______________________________________________
> >> spring mailing list
> >> spring@ietf.org
> >> https://www.ietf.org/mailman/listinfo/spring