Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset forFlex-Algorithm

Ketan Talaulikar <ketant.ietf@gmail.com> Thu, 13 April 2023 05:44 UTC

Return-Path: <ketant.ietf@gmail.com>
X-Original-To: lsr@ietfa.amsl.com
Delivered-To: lsr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E3545C151709 for <lsr@ietfa.amsl.com>; Wed, 12 Apr 2023 22:44:14 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.752
X-Spam-Level:
X-Spam-Status: No, score=-0.752 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, NORMAL_HTTP_TO_IP=0.001, NUMERIC_HTTP_ADDR=1.242, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rtzonW1j4hJO for <lsr@ietfa.amsl.com>; Wed, 12 Apr 2023 22:44:10 -0700 (PDT)
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A30F9C14CE25 for <lsr@ietf.org>; Wed, 12 Apr 2023 22:44:09 -0700 (PDT)
Received: by mail-ed1-x529.google.com with SMTP id 4fb4d7f45d1cf-504718a2282so4085851a12.0 for <lsr@ietf.org>; Wed, 12 Apr 2023 22:44:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681364648; x=1683956648; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=HFcimyHiOZwAKEGM/+gVX613LwOaDB50NNNhbZzsOhk=; b=gs0Tf8a1kwejSehPsKOt0PXAW/tPnZn9a+EAqeVAiQjYp3uKdcZ/fVpJKxJT2TjzCU 1MkXsvoatH7Jl1UgdIgw0dKd6meCbOPYTh6/5/9XyuWVRQr71s6slV8HLQTLKpnuTkDS 1YNA5BqG4mKF3PgyEoO2EGFPEtk0jf9qFI5hG8B70E0xTRu9kofGPS/P8zr8Ct8/FIuD VMlQJZgKWJV78shLgvcFieboEDbcYMHrFhln7r523EEUsykKz+3oSNCOnqBOijYmd3BB toy1QRcBRayK2jZyCplZHnMEVHjfC1/mNiK/X3NqNk2+hJCRqkk8f+7auLFlUKgKU56/ zMGw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681364648; x=1683956648; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HFcimyHiOZwAKEGM/+gVX613LwOaDB50NNNhbZzsOhk=; b=XKg08d2F+5Fmgj35g4E3JA21ITAwNNoa5FgBOjcKukR5gMKC6Otu9ryI5zNH5FrkeN tNZxi/doG6kcqsRQuWSewDh5wppyJXuGKyKwlqkPvNYIkLJtNUMqWZlJvF0HvEOqOecg vJMN/JuPoCDMzPSalDpuZ4fzmZoHSZBaTX678Ceh0tEdBldSuHYSMO+9kZjHbYrTfT5C izkr34X1+zg9hEj3lTNNqrZvMjBs4a+WckeCxD55Y97b1rf2x3nr3NMOl6TIukAV76ry z5Wo7bVF/WQaftUDRS/kErhhFffCw6ePm59MwmU6Ojqe369V13VF1akr50cwH7sr2Jo/ 5bBw==
X-Gm-Message-State: AAQBX9cyeSoWtma4eOD6V5Bv4fdtO8g7iOVaXp21EVzosh5viBU+Zbwm W+8wPV+UxythTB+DzUqfLXFoRWMuGMfU+7n4Atw=
X-Google-Smtp-Source: AKy350aFaj3ncrok03yMEOXbeuOn9KjifWNtTe1GJp1LnjnuhD4T9krZk9DIWdp2UuHotruk3OyTCsncme4BoaGXsU4=
X-Received: by 2002:a50:9b0e:0:b0:504:7684:a23c with SMTP id o14-20020a509b0e000000b005047684a23cmr567753edi.8.1681364647781; Wed, 12 Apr 2023 22:44:07 -0700 (PDT)
MIME-Version: 1.0
References: <MWHPR05MB32312AD47458F4369D21381EDBB89@MWHPR05MB3231.namprd05.prod.outlook.com> <CH0PR05MB1027362770793A71FAD12B6F5DB849@CH0PR05MB10273.namprd05.prod.outlook.com> <CABY-gOPwP6F_NegtniQa9K4b+VUFX_ySVXxjEjOORsLD9zDJ3g@mail.gmail.com> <CH0PR05MB10273B00CB7B535691B15833FDB859@CH0PR05MB10273.namprd05.prod.outlook.com> <CH0PR05MB10273495993BEAD66CDAF7BB5DB889@CH0PR05MB10273.namprd05.prod.outlook.com> <AB077D7D-2A90-4289-B372-20933B32C9A6@gmail.com> <BY5PR11MB433744F23925142BEBE875BCC1969@BY5PR11MB4337.namprd11.prod.outlook.com> <CH0PR05MB10273595E93B90E20AF7B5E65DB9A9@CH0PR05MB10273.namprd05.prod.outlook.com> <BY5PR11MB43373E11C95ADF91A256983FC19A9@BY5PR11MB4337.namprd11.prod.outlook.com> <CH0PR05MB1027340A2303F94A773B1FF3BDB9B9@CH0PR05MB10273.namprd05.prod.outlook.com> <2b096436746bccd-00027.Richmail.00001030847411916931@chinamobile.com> <f3530b3b-f091-27b3-aacf-a3cd691cb9ce@cisco.com> <9f456617d3ed4769be02685b8daac4c1@h3c.com> <2bb05328-8a5b-92e6-a0bd-266da8492c0f@cisco.com> <978EE357-7229-4C92-A3AC-1A958CA18ACB@gmail.com> <abc5fe90672840e99754823e3ae2e226@h3c.com> <CAOj+MMHAKeo_BFk8NWhAKDNDcMi98kaVALqWO8uEPQd0pHbkjw@mail.gmail.com> <E817FC13-8E0D-4E6A-9765-3F6D4D5B4009@juniper.net> <CAH6gdPxzL6JLBc31YyUqreB6yGRddjmbLDej95qvmr9WZRRxEw@mail.gmail.com> <845A0DDD-D104-41E1-9B0A-7E536342C5E5@juniper.net> <CAH6gdPy8Z2jZx1wRF74VQuAkPNsS4Unkh0ZXWmK3Vs_f3GFh3g@mail.gmail.com> <CH0PR05MB10273F7F6AD80ECC1B546612EDB989@CH0PR05MB10273.namprd05.prod.outlook.com> <CAH6gdPwR0HjNXC7qLM55CurtH+vPFbM7WUrAZbrR14KB0F8Ykw@mail.gmail.com> <CH0PR05MB10273859776C4C506053F0587DB989@CH0PR05MB10273.namprd05.prod.outlook.com>
In-Reply-To: <CH0PR05MB10273859776C4C506053F0587DB989@CH0PR05MB10273.namprd05.prod.outlook.com>
From: Ketan Talaulikar <ketant.ietf@gmail.com>
Date: Thu, 13 Apr 2023 11:13:56 +0530
Message-ID: <CAH6gdPxzYr_+B4NEDsBq2n_EPJMDHbjKBja2ppdcrr6mvPxX_A@mail.gmail.com>
To: Louis Chan <louisc@juniper.net>
Cc: Krzysztof Szarkowicz <kszarkowicz@juniper.net>, Robert Raszuk <robert@raszuk.net>, linchangwang <linchangwang.04414@h3c.com>, Acee Lindem <acee.ietf@gmail.com>, Peter Psenak <ppsenak@cisco.com>, 程伟强 <chengweiqiang@chinamobile.com>, "Les Ginsberg (ginsbe" <ginsberg@cisco.com>, lsr <lsr@ietf.org>
Content-Type: multipart/alternative; boundary="00000000000091a9a405f93136ba"
Archived-At: <https://mailarchive.ietf.org/arch/msg/lsr/wume4GNmWIVP0wbCJqgjaQf9WG0>
Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset forFlex-Algorithm
X-BeenThere: lsr@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Link State Routing Working Group <lsr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/lsr>, <mailto:lsr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/lsr/>
List-Post: <mailto:lsr@ietf.org>
List-Help: <mailto:lsr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/lsr>, <mailto:lsr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Apr 2023 05:44:15 -0000

Hi Louis,

One last bit from my side as well ...

You are correct that this topic is too wide to discuss on this thread.
Also, I sense that you perhaps have the use-case and applicability in your
mind. Could I request you to consider putting that into the draft so we
could understand the motivation?

Right now, I see on this thread that perhaps your original motivation for
this draft is being mixed up with running a large number of FlexAlgos in
the network.

Thanks,
Ketan

On Thu, Apr 13, 2023 at 10:01 AM Louis Chan <louisc@juniper.net> wrote:

> Hi Ketan,
>
>
>
> Just a short response.
>
>
>
> If you remember ATM days, there are VP shaping/policing and VC
> shaping/policing. It can solve quite complicated QOS problem.
>
> Here we are not doing the costly ATM again.
>
>
>
> With kind of hierarchical QOS readily available in ASIC today, it
> potentially solves some complex multipoint to multipoint QOS problem.
>
> But first, it requires labeling the packet, like VCI and VPI.
>
>
>
> I am going to stop here because it would be too much to discuss.
>
>
>
> Rgds
>
> Louis
>
>
>
> *From:* Ketan Talaulikar <ketant.ietf@gmail.com>
> *Sent:* Thursday, April 13, 2023 12:02 PM
> *To:* Louis Chan <louisc@juniper.net>
> *Cc:* Krzysztof Szarkowicz <kszarkowicz@juniper.net>; Robert Raszuk <
> robert@raszuk.net>; linchangwang <linchangwang.04414@h3c.com>; Acee
> Lindem <acee.ietf@gmail.com>; Peter Psenak <ppsenak@cisco.com>; 程伟强 <
> chengweiqiang@chinamobile.com>; Les Ginsberg (ginsbe <ginsberg@cisco.com>;
> lsr <lsr@ietf.org>
> *Subject:* Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset
> forFlex-Algorithm
>
>
>
> *[External Email. Be cautious of content]*
>
>
>
> Hi Louis,
>
>
>
> First we need to ascertain if it is necessary before we get things flooded
> into IGPs. Then we can get to the evaluation of whether or how "evil" it is.
>
>
>
> The VLAN analogy seems incorrect to me and a debate on that may be
> orthogonal to this topic. I'll wait for the "necessity" to be first
> described before taking a bite at this PIZZA ;-)
>
>
>
> Once again, thanks for your work on this document.
>
>
>
> Thanks,
>
> Ketan
>
>
>
> On Thu, Apr 13, 2023 at 9:15 AM Louis Chan <louisc@juniper.net> wrote:
>
> Hi Ketan,
>
>
>
> First, I like “PIZZA” more than “VFA”, and at least it is closer to “real”
> and tasty instead of “virtual” and boring. :)
>
>
>
> For VFA, it is just a further identification method. History has VLAN
> first introduced in ethernet. People might think it was already enough in
> that period. But later, market asked for stacked vlan, and find its
> application.
>
>
>
> Having VFA/PIZZA might give more possibilities for future usage. It is
> only ~30B advertisement per VFA per node, and it would not be a big harm.
> And there is no further header overhead (or cell tax) in forwarding plane,
> and make it easy of vendor interop. This is more important.
>
>
>
> I have no intention to swamp IGP with control plane kind of info, like QOS
> profile.  My view is to leave IGP as slim as possible for quick reaction to
> network changes.
>
>
>
> If it is a necessary evil, it should be minimum. My take would be getting
> minimum but useful enough info into FAD. In this case, the advertisement
> size is small, with ease of management.  That is what I refer to “good to
> have” info in previous email.
>
>
>
> Rgds
>
> Louis
>
>
>
> *From:* Ketan Talaulikar <ketant.ietf@gmail.com>
> *Sent:* Thursday, April 13, 2023 12:06 AM
> *To:* Krzysztof Szarkowicz <kszarkowicz@juniper.net>
> *Cc:* Robert Raszuk <robert@raszuk.net>; linchangwang <
> linchangwang.04414@h3c.com>; Acee Lindem <acee.ietf@gmail.com>; Peter
> Psenak <ppsenak@cisco.com>; 程伟强 <chengweiqiang@chinamobile.com>; Louis
> Chan <louisc@juniper.net>; Les Ginsberg (ginsbe <ginsberg@cisco.com>; lsr
> <lsr@ietf.org>
> *Subject:* Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset
> forFlex-Algorithm
>
>
>
> *[External Email. Be cautious of content]*
>
>
>
> Hi Krzysztof,
>
>
>
> I got the impression that the use-case/need for these VFA SIDs in the
> first place was to carry some indication in the packet for local treatment
> at each hop (e.g,, bandwidth profile or QoS treatment?) in the data path
> since the forwarding is based on FlexAlgo.
>
>
>
> If not, then I am not sure that I follow the use-case or motivation for
> VFA (or pizza ;-) as Louis calls it) in the first place.
>
>
>
> Thanks,
>
> Ketan
>
>
>
>
>
> On Wed, Apr 12, 2023 at 9:28 PM Krzysztof Szarkowicz <
> kszarkowicz@juniper.net> wrote:
>
> Hi Ketan,
>
>
>
> I agree with you. The draft doesn’t propose to carry ’service bandwidth
> profile’ parameters in the IGP.
>
>
>
> The draft is dealing only with label/SID assignment/generation and
> distribution.
>
>
>
> Thanks,
>
> Krzysztof
>
>
>
>
>
> On 2023 -Apr-12, at 17:49, Ketan Talaulikar <ketant.ietf@gmail.com> wrote:
>
>
>
>
>
> *[External Email. Be cautious of content]*
>
>
>
>
>
> Hi Krzysztof,
>
>
>
> A further question is if it is necessary to carry this "service bandwidth
> profile" information in the IGPs in the first place. Why not indicate this
> just in the packet? Wouldn't that be a better way to help scale IGPs by not
> introducing this into IGPs in the first place? ;-)
>
>
>
> One such simple solution is proposed by
> https://datatracker.ietf.org/doc/draft-decraene-mpls-slid-encoded-entropy-label-id/
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-decraene-mpls-slid-encoded-entropy-label-id/__;!!NEt6yMaO-gk!DxCsNxydz3b_s_iNl0j_6pyHLD1_vLYmeqwZbd_OBYdhfs3BBaQ8bIXbBsB-4Yigax-NkpPUA1EmCDwYmDNrSqU0Lg$>
>
>
>
> Thanks,
>
> Ketan
>
>
>
>
>
> On Wed, Apr 12, 2023 at 9:13 PM Krzysztof Szarkowicz <kszarkowicz=
> 40juniper.net@dmarc.ietf.org> wrote:
>
> Hi,
>
>
>
> It is the question, if we could for example have more than 20 (e.g. 200),
> for 200 different service bandwidth guarantees (but having only one - or
> handful - SPF calculation domains, where one SPF calculation domain is one
> ‘legacy’ flex-algo ). The challenge is with SPP calculations, once the
> number of flex-algos goes high.
>
>
>
> Thanks,
>
> Krzysztof
>
>
>
>
>
> On 2023 -Apr-12, at 17:13, Robert Raszuk <robert@raszuk.net> wrote:
>
>
>
>
>
> *[External Email. Be cautious of content]*
>
>
>
>
>
>
>
> Ok you can use 20 flex algos today with no extension. Is going to another
> level of nesting really necessary ?
>
>
>
> On Wed, Apr 12, 2023 at 4:52 PM linchangwang <linchangwang.04414@h3c.com>
> wrote:
>
> Hi Acee
>
>
>
> An operator's backbone network is divided into different flex algorithms
> planes according to different SLA requirements of users.
>
> A flex algo represents a service requirement, such as bandwidth
> requirements.
>
> 20 flex algorithms represent 20 different service bandwidth guarantees,
> corresponding to different resource requirements.
>
>
>
> Thanks,
>
> Changwang lin
>
>
>
> *From:* Acee Lindem [mailto:acee.ietf@gmail.com]
> *Sent:* Wednesday, April 12, 2023 10:12 PM
> *To:* Peter Psenak
> *Cc:* linchangwang (RD); 程伟强; Louis Chan; Les Ginsberg (ginsbe; lsr;
> Krzysztof Szarkowicz
> *Subject:* Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset
> forFlex-Algorithm
>
>
>
> Hi Weiqiang,
>
>
>
> I’m also curious as to how you are using 20 different flex algorithms. Is
> this just a hypothetical scenario
>
> to illustrate the mathematics or do you have an actual use case?
>
>
>
> On Apr 12, 2023, at 09:31, Peter Psenak <ppsenak@cisco.com> wrote:
>
>
>
> Changwang,
>
> please see inline (##PP2):
>
>
> On 12/04/2023 15:13, linchangwang wrote:
>
> Hi Peter
>  Please see inline [changwang lin].
>
> We've met the same problem when applying Flex Algo in SRv6 network.
>
> what problem exactly, can you please describe it?
> [changwang lin]
> Advertisement size of per Flex-Algo Adj-SID in the network
> Related to F(# of node, # of FA, # of links)
> For a node with 1,000 links and 20 Flex-Algo
>    n x 20 x 1000
>    MPLS-SR:If n = 10 bytes, it is 200K bytes
>    SRv6:   If n = 24 bytes, it is 400K+ bytes
> If 500 nodes:
>    MPLS-SR:it is 200K*500   =  100000k bytes
>    SRv6:   it is 400K+ * 500  = 200000k bytes
> If interface mtu=1500, lsp length = 1497
>  LSPs num:
>    MPLS-SR:10000k bytes/1497 = 66800  lsps
>    SRv6:   20000k bytes/1497 = 160320 lsps
> The number of LSPs is too large, and IS-IS needs to periodically refresh
> LSPs,
> resulting in a decrease in ISIS performance and unstable network operation.
>
>
> ##PP2
> above is hardly a realistic estimation.
>
> In a network with 1k nodes, not every node will have 1k links.
>
> Advertising large number of LSPs is not caused by Adj-SIDs.
> With TE enabled the amount of data flooded per link is larger than
> advertisement of the 20 Adj-SID. The problem you are highlighting is not
> specific to Adj-SIDs, it's generic.
>
> LSP refresh time can be set to 18 hours and any reasonable implementation
> does not refresh all LSPs at the same time.
>
> So we need to optimize on the control surface to save LSP space.
>
>
> ##PP2
> with all the respect, I don't agree. The problem as you described it does
> not exist.
>
> Through the optimization notification mechanism mentioned in these two
> documents,
> we have greatly saved LSP space for IS-IS and improved the performance of
> IS-IS flex algo in large-scale networking.
> At the same time, through the VFA mechanism, in other non flex algo
> application scenarios,
>  such as network slicing scenarios, the LSP space of IS-IS can also be
> saved
>
>
> ##PP2
> it seems to me you are trying to fix the implementation problem with the
> protocol changes, which is never a good idea.
>
> thanks,
> Peter
>
> thanks,
> Changwang lin
> -----Original Message-----
> From: Lsr [mailto:lsr-bounces@ietf.org] On Behalf Of Peter Psenak
> Sent: Wednesday, April 12, 2023 7:10 PM
> To: 程伟强; Louis Chan; Les Ginsberg (ginsbe; Acee Lindem
> Cc: lsr; Krzysztof Szarkowicz
> Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset
> forFlex-Algorithm
> Weiqiang,
> please see inline (##PP):
> On 12/04/2023 12:05, 程伟强 wrote:
>
> Hi Louis and Les,
>
>
> My two cents from operator perspective.
>
>
> We've met the same problem when applying Flex Algo in SRv6 network.
>
> what problem exactly, can you please describe it?
> [changwang lin]
> Advertisement size of per Flex-Algo Adj-SID in the network
> Related to F(# of node, # of FA, # of links)
> For a node with 1,000 links and 20 Flex-Algo
>    n x 20 x 1000
>    MPLS-SR:If n = 10 bytes, it is 200K bytes
>    SRv6:   If n = 24 bytes, it is 400K+ bytes
> If 500 nodes:
>    MPLS-SR:it is 200K*500   =  100000k bytes
>    SRv6:   it is 400K+ * 500  = 200000k bytes
> If interface mtu=1500, lsp length = 1497
>  LSP num:
>    MPLS-SR:10000k bytes/1497 = 66800  lsps
>    SRv6:   20000k bytes/1497 = 160320 lsps
> The number of LSPs is too large, and IS-IS needs to periodically refresh
> LSPs,
> resulting in a decrease in ISIS performance and unstable network operation.
> So we need to optimize on the control surface to save LSP space.
> Through the optimization notification mechanism mentioned in these two
> documents,
> we have greatly saved LSP space for IS-IS and improved the performance of
> IS-IS flex algo in large-scale networking.
> At the same time, through the VFA mechanism, in other non flex algo
> application scenarios,
>  such as network slicing scenarios, the LSP space of IS-IS can also be
> saved
>
>
>
> As the number of slices and the scale of the network increases, the
> convergence issue which is caused by SIDs  advertising and flooding
> becomes more and more serious.
>
>
> Due to the problem, it is impossible to apply Flex-Algo in the large
> network, such as the network with more than 1000 routers.
>
> flex-algo has been successfully deployed in a networks that have more
> that 1k nodes.
> Maybe you want deploy the flex-algo for something that it was not
> designed for.
>
>
>
> I believe Louis'draft provides a good idea to resolve this problem.
> Similar solution for SRv6 SIDs is described in another draft.
>
> Again, what problem exactly?
>  From what I see the drafts tries to pack algo SIDs to save space in
> LSP. I don't see how it helps to to deploy flex-algo in a large scale
> network.
> thanks,
> Peter
>
>
>
> About the SIDs assignment, I think it is better to have a scheduled
> assignment than a random assignment as Les mentioned.
>
>
> [1]
> https://datatracker.ietf.org/doc/draft-cheng-lsr-isis-srv6-sid-block/
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-cheng-lsr-isis-srv6-sid-block/__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUND7v7my7$>
> <https://datatracker.ietf.org/doc/draft-cheng-lsr-isis-srv6-sid-block/
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-cheng-lsr-isis-srv6-sid-block/__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUND7v7my7$>
> >
>
>
>
> Thanks,
>
> Weiqiang Cheng
>
>
>
>     ----邮件原文----
>     *发件人:*Louis Chan  <louisc=40juniper.net@dmarc.ietf.org>
>     *收件
>     人:*"Les Ginsberg (ginsberg)" <ginsberg@cisco.com>,Acee Lindem <
> acee.ietf@gmail.com>
>     *抄 送:
>     *lsr  <lsr@ietf.org>,Krzysztof Szarkowicz  <kszarkowicz@juniper.net>,Weiqiang
> Cheng  <chengweiqiang@chinamobile.com>
>     *发送时间:*2023-04-12 10:45:56
>     *主题:*Re: [Lsr] IETF-
>     116 LSR - IGP extensions for Advertising Offset forFlex-Algorithm
>
>     Hi Les,
>
>     Thanks for the prompt reply. Please see inline for clarification [lc2].
>
>     /Louis
>
>     *From:* Les Ginsberg (ginsberg) <ginsberg@cisco.com>
>     *Sent:* Tuesday, April 11, 2023 11:03 PM
>     *To:* Louis Chan <louisc@juniper.net>; Acee Lindem <
> acee.ietf@gmail.com>
>     *Cc:* lsr <lsr@ietf.org>; Krzysztof Szarkowicz
>     <kszarkowicz@juniper.net>; Weiqiang Cheng
>     <chengweiqiang@chinamobile.com>
>     *Subject:* RE: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>     Offset for Flex-Algorithm
>
>     *[External Email. Be cautious of content]*
>
>     Louis -
>
>     Please see inline.
>
>      > -----Original Message-----
>
>      > From: Louis Chan <louisc@juniper.net <mailto:louisc@juniper.net>>
>
>      > Sent: Monday, April 10, 2023 11:01 PM
>
>      > To: Les Ginsberg (ginsberg) <ginsberg@cisco.com
>     <mailto:ginsberg@cisco.com>>; Acee Lindem
>
>      > <acee.ietf@gmail.com <mailto:acee.ietf@gmail.com>>
>
>      > Cc: lsr <lsr@ietf.org <mailto:lsr@ietf.org>>; Krzysztof
>     Szarkowicz <kszarkowicz@juniper.net <mailto:kszarkowicz@juniper.net>>;
>
>      > Weiqiang Cheng <chengweiqiang@chinamobile.com
>     <mailto:chengweiqiang@chinamobile.com>>
>
>      > Subject: RE: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>     Offset for
>
>      > Flex-Algorithm
>
>      >
>
>      > Hi Les,
>
>      >
>
>      > Thanks for your questions. Please see inline [lc] below.
>
>      >
>
>      > /Louis
>
>      >
>
>      > -----Original Message-----
>
>      > From: Les Ginsberg (ginsberg) <ginsberg@cisco.com
>     <mailto:ginsberg@cisco.com>>
>
>      > Sent: Saturday, April 8, 2023 7:34 AM
>
>      > To: Acee Lindem <acee.ietf@gmail.com
>     <mailto:acee.ietf@gmail.com>>; Louis Chan <louisc@juniper.net
>     <mailto:louisc@juniper.net>>
>
>      > Cc: lsr <lsr@ietf.org <mailto:lsr@ietf.org>>
>
>      > Subject: RE: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>     Offset for
>
>      > Flex-Algorithm
>
>      >
>
>      > [External Email. Be cautious of content]
>
>      >
>
>      >
>
>      > OK - since Acee opened the door - here are some comments from me -
>
>      > starting with the most important.
>
>      >
>
>      > (BTW - I still have limited enthusiasm for this draft.)
>
>      >
>
>      > 1)The proposal places some restrictions on how operators
>     provision their
>
>      > network in terms of assigning SIDs and reserving space for future
>
>      > assignments.
>
>      > If operators do not use compatible assignment schemes, then this
>     will never
>
>      > get deployed. It is therefore not enough to come with a nice idea
>     - you must
>
>      > have some enthusiasm from the operator community.
>
>      >
>
>      >
>
>      > [lc] If the operator only wants to deploy flex-algo, there is no
>     change in their
>
>      > Node-sid numbering scheme. For the Adj-sid, these are local
>     labels with local
>
>      > significant only, and there is no need for any special planning
>     for Adj-sid,
>
>      > unless you are suggesting they want to make fixed assignment of
>     Adj-sid
>
>      > label for each link. Even with fixed, the proposed draft has
>     benefit on that. I
>
>      > will explain later.
>
>      >
>
>     [LES:] Let's discuss this in the context of prefix-sids - the same
>     applies to adj-sids.
>
>     Today (i.e., in the absence of your proposal) an operator is free to
>     assign any label within the SRGB for a given prefix/algo pair so
>     long as it is not assigned to some other prefix/algo context.
>
>     Your proposal places some new restrictions. Now, for a given
>     flex-algo, whenever an operator assigns a given label for a prefix
>     in Algo 0 (call it Label-A0), they must guarantee that
>     "Label-A0+offset" for an  advertised flex-algo specific offset is
>     available to be assigned for the prefix/flex-algo pair - and this
>     must be true for all prefixes advertised in algo 0.
>
>     This is certainly possible to do, but is not guaranteed to be the
>     case in current deployments.
>
>     For example - and this is only an example...today an operator might
>     utilize a provisioning tool to assign prefix-sids for all supported
>     algorithms on all nodes in the network.
>
>     To do this, the tool might maintain a database of assigned labels.
>     When provisioning a new node/prefix/algorithm, the logic in the tool
>     might simply take the next available label in the database.
>
>     The result of this would not be consistent with the requirements of
>     your draft.
>
>     Which is why I say in order to deploy the extension you propose,
>     such an operator would have to modify its provisioning tool.
>
>     [lc2] There might be some misunderstanding of our proposal. Let me
>     give some examples.
>
>     Case 1: Flex-Algo only
>
>     Prefix offset advertisement: “no”
>
>     Adj-sid offset advertisement: yes
>
>     In slide 8’s example, FA129 is using label “402001”, and the
>     advertisement of this label is using existing methods.
>
>     e.g. SRGB = 400000-460000
>
>     FA129: index 2001 (preferred value), or one can choose 111, 222
>
>     FA130 (new): index 3001 (preferred value), or 333, 4444
>
>     This does not change how the operator to assign label for prefix-sid
>     with their current method. Any index/label could be used for FA
>     prefix within SRGB.
>
>     The only change is the Adj-sid label allocation, but this “mostly”
>     is  only “local” to one node. There is no effect on global label
>     allocation.
>
>     This draft will be compatible to what operators are doing today.
>
>     Case 2: VFA only
>
>     Prefix offset advertisement: yes
>
>     Adj-sid offset advertisement: yes
>
>     I agree, with VFA, there would be impact to global allocation to
>     node-sid/prefix-sid. But VFA is a totally new concept. No one has
>     deployed that yet.
>
>     There is no impact to operators which stick to deploy only Flex-algo.
>
>     Other case: Flex-Algo w/o Adj-sid offset
>
>                    Continue the example of Case#1 above
>
>                    Another FA131 is added, but no Adj-sid offset is
>     advertised
>
>                    The question would be
>
>       *
>
>         Either allow this configuration, and FA131 will fallback using
>         Algo 0’s  adj-sid
>
>       *
>
>         Or, disallow this configuration
>
>     I tend to “allow” such configuration with mix of FA129, FA130 (with
>     adj-sid offset) and FA131 (w/o adj-sid offset)
>
>     [/lc2]
>
>     Could this be done? Sure.
>
>     Do operators want to do this? I do not know.
>
>     But since this would be necessary in order to use your proposed
>     extension, it is necessary to gauge operator enthusiasm for making
>     such changes in order to know whether there is any point in
>     proceeding with  your proposal.
>
>      > In slide 8 of the below presentation
>
>      >
>
> https://datatracker.ietf.org/meeting/116/materials/slides-116-lsr-03-ietf116-
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting/116/materials/slides-116-lsr-03-ietf116-__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNBlx2nlh$>
> <
> https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting/116/materials/slides-116-lsr-03-ietf116-igp-adv-offset-01__;!!NEt6yMaO-gk!Bl7Swe9ql9VT0qGkD6FoZZzTWT2fmIx55eSncdmMgoCJetJ5-80micuqnqk79yewGB-BleOfrYpSjfI$
> >
>
>     >  igp-adv-offset-01
>     <
> https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting/116/materials/slides-116-lsr-03-ietf116-igp-adv-offset-01__;!!NEt6yMaO-gk!Bl7Swe9ql9VT0qGkD6FoZZzTWT2fmIx55eSncdmMgoCJetJ5-80micuqnqk79yewGB-BleOfrYpSjfI$
> >
>
>      >
>
>      > FA129 is a prefix-sid (400201) allocated by operator, and it can
>     be any label.
>
>      > There is no connection to how Adj-sid is derived.
>
>      > Per Flex-Algo adj-sid assignment is not affecting network wide label
>
>      > assignment from operation perspective. Each node could have
>     different local
>
>      > block for such adj-sid assignment. One might need to estimate
>     total possible
>
>      > number of link in one chassis for such allocation, or it could be
>     estimated by
>
>      > OS software itself. I also mentioned in the session, if there is
>     100 x FA with
>
>      > 1000 links (high end), it is only 100k labels. Is it difficult to
>     allocate such.
>
>     [LES:] Your proposal does not reduce the number of labels which need
>     to be "allocated" and installed in forwarding, it only reduces the
>     number of bytes used to advertise this information in LSPs/LSAs.
>
>     [lc2] You are correct.
>
>     I think, at the same time, it helps reducing time for global
>     convergence since the advertisement size is smaller, especially in
>     network with long diameter with multi-hops.
>
>     Also, in
>     https://datatracker.ietf.org/doc/draft-ietf-lsr-isis-fast-flooding/
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-lsr-isis-fast-flooding/__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNHGf6W4I$>
>     <https://datatracker.ietf.org/doc/draft-ietf-lsr-isis-fast-flooding/
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-lsr-isis-fast-flooding/__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNHGf6W4I$>>
>  (which you participated in)
>
>      >>>
>
>        As IS-IS is deployed in greater scale both in the number of nodes in
>
>         an area and in the number of neighbors per node, the impact of the
>
>         historic flooding rates becomes more significant.  Consider the
>
>         bringup or failure of a node with 1000 neighbors.  This will result
>
>         in a minimum of 1000 LSP updates.  At typical LSP flooding rates
>     used
>
>         today (33 LSPs/second), it would take 30+ seconds simply to send
> the
>
>         updated LSPs to a given neighbor.  Depending on the diameter of the
>
>         network, achieving a consistent LSDB on all nodes in the network
>
>         could easily take a minute or more.
>
>     <<<
>
>     This proposed draft will certainly help.
>
>     [/lc2]
>
>      >
>
>      > So, this is why I do not understand your question in full.
>
>      >
>
>      > If the operator plans to use VFA, that would be a different
>     discussion. VFA,
>
>      > today, does not exist in deployment.
>
>      >
>
>     [LES:] My comments here have nothing to do with VFA.
>
>      > [/lc]
>
>      >
>
>      > Have you discussed this idea with any operators?
>
>      >
>
>      > [lc] I include Wei-qiang from China mobile in this thread. He has
>     shown his
>
>      > need on this kind of solution. Maybe, he could give his
>     perspective here. [/lc]
>
>      >
>
>      > If so, what has been their response?
>
>      > If they are open to the idea, how might they migrate from their
>     existing
>
>      > assignment schemes to an assignment scheme compatible with the
>
>      > proposal?
>
>      >
>
>      > These are questions that need to be answered before considering
>     this idea.
>
>      >
>
>      > [lc] In slide 8, if you see these label numbers
>
>      >
>
>      > Prefix-sid: 400001, 402001, 406001, 407001
>
>      > Adj-sid: 32, 2032, 6032, 7032
>
>      >
>
>      > From operator perspective or troubleshooting perspective, value
>     xxx001
>
>      > represent the same node, and value x032 represent the same link.
> This
>
>      > makes things more organized and easier to understand.
>
>      >
>
>      > If all are random labels, I do not see any benefit at all.
>
>      > [/lc]
>
>     [LES:] I am not commenting on whether the label assignment scheme
>     you propose is better or worse than any other.
>
>     I am only pointing out that you are imposing new restrictions on how
>     labels are allocated.
>
>     As you are not in charge of how operators provision their networks
>     (nor am I), it is presumptuous of you to think that simply because
>     you think this is a better way to do things that operators will be
>     happy to modify their existing networks  to conform to your proposed
>     restrictions.
>
>     This isn’t academia - you need to vet this with the operator community.
>
>     [lc2] Please refer to the examples at the top. The picture should be
>     clear by now. There is no restriction to what is deployed today. [/lc2]
>
>      >
>
>      > 2)Section 5 Compatibilty
>
>      >
>
>      > There is no "compatibility" with legacy nodes - because all nodes
>     in the
>
>      > network have to have a consistent understanding of what SID is
>     assigned to a
>
>      > given context (for prefixes and adjacencies) since they might
>     need to install
>
>      > forwarding entries for that context.
>
>      > I do not see any point in deploying this until all nodes support
>     it. If you did do
>
>      > so, you would need to advertise old and new forms - which does the
>
>      > opposite of what you are trying to achieve. Instead of reducing
>     LSP space
>
>      > used you would increase it.
>
>      >
>
>      > [lc] If the operator just plans to use only Flex-Algo, and no
>     VFA, it should be
>
>      > compatible with legacy implementation. If legacy nodes do not
>     understand
>
>      > adj-sid offset notation, these nodes could just ignore it. The
>     forwarding
>
>      > plane should work with co-existence of old and new nodes. Per
>     flex-algo adj-
>
>      > sid is only local significant to one node. New nodes should
>     detect whether
>
>      > legacy nodes exist in the network via such new extension
>     advertisement.
>
>      > And new nodes should use only algo 0 adj-sid from legacy nodes
>     for any TI-
>
>      > LFA.
>
>     [LES:] Consider a network of 100 nodes.
>
>     Let's say the "left-hand-side" of the network consist of legacy
>     nodes who do not understand your new advertisements.
>
>     The "right-hand-side" of the network consists of upgraded nodes who
>     support the new advertisements.
>
>     Consider nodes PE-LEFT and PE-RIGHT.
>
>     PE-RIGHT advertises a prefix-SID of 1000 for 2.2.2.2/32
> <https://urldefense.com/v3/__http:/2.2.2.2/32__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNEj8LL78$>,
> and an
>     offset of 1000 for flex-algo 128.
>
>     PE-LEFT supports flex-algo 128 and wants to install a forwarding
>     entry for 2.2.2.2 for flex-algo 128.
>
>     It looks in the LSPs originated by PE-RIGHT. It does not see any
>     assigned SID for 2.2.2.2/32
> <https://urldefense.com/v3/__http:/2.2.2.2/32__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNEj8LL78$>
> flex-algo 128.
>
>     It cannot create a forwarding entry. And neither can any other node
>     in the left hand side of the network.
>
>     When PE-RIGHT stops advertising the explicit prefix SID for
>     2.2.2.2/32
> <https://urldefense.com/v3/__http:/2.2.2.2/32__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNEj8LL78$>
> Algo 128, all legacy nodes are unable to create
>     forwarding entries for the prefix/algo tuple.
>
>     This isn’t backwards compatible. 
>
>     In general, you cannot advertise information legacy nodes require in
>     a new container that legacy nodes do not understand and claim that
>     you are backwards compatible.
>
>     [lc2] Please refers to the examples for clarification.
>
>      1.
>
>         For existing Flex-Algo deployment, it would be compatible. There
>         is no change in the container format on how prefix-sid
>         2.2.2.2/32
> <https://urldefense.com/v3/__http:/2.2.2.2/32__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNEj8LL78$>
> in FA128 is advertised.
>
>      1.
>
>         For new VFA, it would not be compatible. But….it does not mean
>         that we could not have VFA running in the same network.
>
>     There could be procedures to enhance such. With your example, one
>     workaround could be:
>
>     For VFA 600, PE-RIGHT detects that PE-LEFT does not participate in
>     VFA600 (due to no offset advertisement seen),
>
>       *
>
>         Either, it spawns new CSPF for VFA600 instead of sharing FA129’s
>         result. Bypass PE-LEFT as a result.
>
>       *
>
>         Or, it uses legacy node FA129 prefix-sid and adj-sid as
>         replacement (note: this method needs more comment)
>
>     In either ways, VFA600 could work without issue even with legacy
>     nodes co-existence.
>
>     After PE-LEFT upgraded, VFA600 would be using FA129 CSPF result
>     instead, and save CPU resources in each node.
>
>     Another question: do we need FAD for VFA600? Currently, no. Not
>     mandatory.
>
>     But it could be considered if “good to have” parameters are passed
>     along with FAD.
>
>     [/lc2]
>
>      >
>
>      > I do not see a major problem. Please give me an example to
>     illustrate your
>
>      > concern if possible.
>
>      >
>
>      > Of course, we need to do double check on the claim and possibly lab
>
>      > verification to see if the backward compatibility could be
>     achieved. It could
>
>      > be vendor specific.
>
>      >
>
>      > [/lc]
>
>      >
>
>      >
>
>      > What does deserve discussion is a "hitless migration strategy".
>     When full
>
>      > support is available, if you were to switch to the new scheme,
>     you would
>
>      > want to do so without changing any existing SIDs as this would avoid
>
>      > forwarding disruption. Which means operators would have to modify
>     their
>
>      > SID assignment scheme in advance of deploying the new scheme.
>
>      >
>
>      > [lc] For VFA, there would be issue for legacy nodes. I agree. In
>     this case,
>
>      > solution could be
>
>      >         - either have a fallback plan for newer nodes if
>     detection of legacy nodes
>
>      > exist in the network. E.g. spawn new CSPF
>
>      >         - or, totally not to deploy VFA unless all nodes are
>     upgraded.
>
>      >
>
>      > Section 5 is not updated with VFA inclusion.
>
>     [LES:] My comments have nothing to do with VFA.
>
>     Please reconsider them after you have understood the backwards
>     compatibility issues.
>
>      >
>
>      > [/lc]
>
>      >
>
>      > 3)Virtual Flex Algorithm
>
>      >
>
>      > You have introduced a new concept with very little explanation of
>     what it is
>
>      > nor how it can be supported.
>
>      > For example, how would we determine which nodes support a given VFA?
>
>      > Since the algorithm value must be greater than 255, it cannot be
>     advertised in
>
>      > the existing SR Algorithm sub-TLV.
>
>      >
>
>      > If you are serious about this idea, please provide a more complete
>
>      > discussion.
>
>      >
>
>      > [lc] We could illustrate application examples in next
>     presentation. For
>
>      > ethernet, we have port, and then we have VLAN and stacked vlan.
>     History
>
>      > has some hints on this.
>
>      >
>
>     [LES:] You are writing a normative specification. Hoping that all
>     readers/implementors have the same "intuition" isn’t sufficient.
>
>         Les
>
>      > [/lc]
>
>      >
>
>      > 4)Section 4.3
>
>      >
>
>      > "R" and "N" flags are now defined in prefix attributes sub-TLV
>     (RFC7794)
>
>      > They were originally defined in the SR sub-TLV because RFC 7794
>     did not exist
>
>      > at the time.
>
>      > The only reason they continue to exist in RFC 8667 is for backwards
>
>      > compatibility with early implementation of SR-MPLS based on early
>     drafts of
>
>      > what became RFC 8667.
>
>      > Please do not introduce them in new sub-TLVs - there is no need.
>
>      >
>
>      > [lc] noted with thanks [/lc]
>
>      >
>
>      > 5)ADJ-SIDs are NOT allocated from the SRGB as they are local in
>     scope.
>
>      > They MAY be allocated from the SRLB - or outside either GB range.
>
>      > Please correct the document in this regard.
>
>      >
>
>      > [lc] noted [/lc]
>
>      >
>
>      >    Les
>
>      >
>
>      > > -----Original Message-----
>
>      > > From: Lsr <lsr-bounces@ietf.org <mailto:lsr-bounces@ietf.org>>
>     On Behalf Of Acee Lindem
>
>      > > Sent: Friday, April 7, 2023 11:43 AM
>
>      > > To: Louis Chan <louisc@juniper.net <mailto:louisc@juniper.net>>
>
>      > > Cc: lsr <lsr@ietf.org <mailto:lsr@ietf.org>>
>
>      > > Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>
>      > > Offset for Flex-Algorithm
>
>      > >
>
>      > > Hi Louis,
>
>      > >
>
>      > > In the interest of initiating discussion, I would like to
>     propose the
>
>      > > term "Flex Algorithm Traffic Class (FATC)" for the sub-division of
>
>      > > flex-algorithm traffic referred to in the draft as “Virtual
>     Flex Algorithm”.
>
>      > >
>
>      > > Also, in your terminology, you refer referred to TLVs and
>     fields with
>
>      > > simply “Algorithm” when RFC 9350 uses “Flex Algorithm”.
>
>      > >
>
>      > > Thanks,
>
>      > > Acee
>
>      > >
>
>      > >
>
>      > >
>
>      > > _______________________________________________
>
>      > > Lsr mailing list
>
>      > > Lsr@ietf.org <mailto:Lsr@ietf.org>
>
>      > >
>     https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/lsr_
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/lsr_> <
> https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/lsr_>
>
>      > > _;!!NEt6yMaO-gk!B9ufrV6k-
>
>      > c7mtP9JUiXbrF3NCkZ15_UMLBjV_fnJovfz18M5VkkI2F
>
>      > > EoixpkxsfMnbqwbR0bpHgoS9E$
>
>      >
>
>      > Juniper Business Use Only
>
>     Juniper Business Use Only
>
>
>     Juniper Business Use Only
>
> _______________________________________________
> Lsr mailing list
> Lsr@ietf.org
> https://www.ietf.org/mailman/listinfo/lsr
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/lsr__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNLUwWHuZ$>
>
> -------------------------------------------------------------------------------------------------------------------------------------
> 本邮件及其附件含有新华三集团的保密信息,仅限于发送给上面地址中列出
> 的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、
> 或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本
> 邮件!
> This e-mail and its attachments contain confidential information from New
> H3C, which is
> intended only for the person or entity whose address is listed above. Any
> use of the
> information contained herein in any way (including, but not limited to,
> total or partial
> disclosure, reproduction, or dissemination) by persons other than the
> intended
> recipient(s) is prohibited. If you receive this e-mail in error, please
> notify the sender
> by phone or email immediately and delete it!
>
>
>
> _______________________________________________
> Lsr mailing list
> Lsr@ietf.org
> https://www.ietf.org/mailman/listinfo/lsr
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/lsr__;!!NEt6yMaO-gk!GCoxXhuL7ryqRodFUI2f-GPQa1_Vjs9WLIB_XJtQUiKVcm6Yrxqx5p-_PmGOQFNseVacNf7mx4QUNLUwWHuZ$>
>
>
>
> _______________________________________________
> Lsr mailing list
> Lsr@ietf.org
> https://www.ietf.org/mailman/listinfo/lsr
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/lsr__;!!NEt6yMaO-gk!DxCsNxydz3b_s_iNl0j_6pyHLD1_vLYmeqwZbd_OBYdhfs3BBaQ8bIXbBsB-4Yigax-NkpPUA1EmCDwYmDMh734fjw$>
>
>
>
>
>
> Juniper Business Use Only
>
>
> Juniper Business Use Only
>