Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset forFlex-Algorithm

Peter Psenak <ppsenak@cisco.com> Tue, 18 April 2023 07:31 UTC

Return-Path: <ppsenak@cisco.com>
X-Original-To: lsr@ietfa.amsl.com
Delivered-To: lsr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 745D4C14CE40 for <lsr@ietfa.amsl.com>; Tue, 18 Apr 2023 00:31:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.598
X-Spam-Level:
X-Spam-Status: No, score=-14.598 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NCZ6xPIOUm-x for <lsr@ietfa.amsl.com>; Tue, 18 Apr 2023 00:31:06 -0700 (PDT)
Received: from aer-iport-5.cisco.com (aer-iport-5.cisco.com [173.38.203.67]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4D308C14F747 for <lsr@ietf.org>; Tue, 18 Apr 2023 00:31:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=59647; q=dns/txt; s=iport; t=1681803065; x=1683012665; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=tJnoQq1yP+HP3+P7YHhEu8QJuEOGXCLsxCz6zrsN7SM=; b=mpcWdMU2Vo0njEih3ihxMkozt8UpKIVEeygKxze0/XmADuostQGy3RGR zTwy7J9kaaXiENlXrNKCnU8abA3AB7UH+HJPSvFVvDJZk2kTXX1neHca3 QpZPu6lZDkHB9W8M0oxjlBnZFxHdPD0r0TqIpw75yPaRyxzayYtsfU19J A=;
X-IronPort-AV: E=Sophos;i="5.99,206,1677542400"; d="scan'208";a="4578246"
Received: from aer-iport-nat.cisco.com (HELO aer-core-11.cisco.com) ([173.38.203.22]) by aer-iport-5.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 18 Apr 2023 07:31:03 +0000
Received: from [10.209.204.188] ([10.209.204.188]) by aer-core-11.cisco.com (8.15.2/8.15.2) with ESMTP id 33I7V2H4093577; Tue, 18 Apr 2023 07:31:02 GMT
Message-ID: <9a5eae20-07e4-46fe-fbc2-2d1770ab67ae@cisco.com>
Date: Tue, 18 Apr 2023 09:31:01 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.9.1
Content-Language: en-US
To: Louis Chan <louisc@juniper.net>, Weiqiang Cheng <chengweiqiang@chinamobile.com>, 'Peter Psenak' <ppsenak=40cisco.com@dmarc.ietf.org>, 'linchangwang' <linchangwang.04414@h3c.com>, "'Les Ginsberg (ginsbe'" <ginsberg@cisco.com>, 'Acee Lindem' <acee.ietf@gmail.com>
Cc: 'lsr' <lsr@ietf.org>, Krzysztof Szarkowicz <kszarkowicz@juniper.net>
References: <MWHPR05MB32312AD47458F4369D21381EDBB89@MWHPR05MB3231.namprd05.prod.outlook.com> <9f456617d3ed4769be02685b8daac4c1@h3c.com> <2bb05328-8a5b-92e6-a0bd-266da8492c0f@cisco.com> <40e2d16ac5464bb19acf343dce22fa0d@h3c.com> <27aca645-0f44-e0a5-ddd6-1ad09650e71f@cisco.com> <CH0PR05MB1027331ABC14E0CC7BE13D5A6DB989@CH0PR05MB10273.namprd05.prod.outlook.com> <16e8b360-3566-11c1-c9d6-39d6ad00178f@cisco.com> <CH0PR05MB102733B612F837AC22B0AA750DB999@CH0PR05MB10273.namprd05.prod.outlook.com> <6b48eee2-3e9b-7d01-03ef-3fc2c2e5ab5d@cisco.com> <CH0PR05MB10273D3BD48DECED88DCBB154DB999@CH0PR05MB10273.namprd05.prod.outlook.com> <ac27c8d a-5bbb-3dfe-80 b0-c78d1de42c96@cisco.com> <051e01d96ebb$e9e0e4f0$bda2aed0$@com> <316ef57a-9237-26e5-ed90-0935f546f8ef@cisco.com> <CH0PR05MB10273F1267E24A3D511D19F8EDB999@CH0PR05MB10273.namprd05.prod.outlook.com> <af6b7923-db36-f84e-4605-5f2f706f5897@cisco.com> <CH0PR05MB1027344A1DFC190100AD7BF58DB9D9@CH0PR05MB10273.namprd05.prod.outlook.com>
From: Peter Psenak <ppsenak@cisco.com>
In-Reply-To: <CH0PR05MB1027344A1DFC190100AD7BF58DB9D9@CH0PR05MB10273.namprd05.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 8bit
X-Outbound-SMTP-Client: 10.209.204.188, [10.209.204.188]
X-Outbound-Node: aer-core-11.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/lsr/6_sVnqgvO6WtIFJSXxH3fY2nDJQ>
Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset forFlex-Algorithm
X-BeenThere: lsr@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Link State Routing Working Group <lsr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/lsr>, <mailto:lsr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/lsr/>
List-Post: <mailto:lsr@ietf.org>
List-Help: <mailto:lsr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/lsr>, <mailto:lsr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Apr 2023 07:31:11 -0000

Loius,

please see inline:

On 18/04/2023 06:26, Louis Chan wrote:
> Hi Peter,
> 
> Such an answer to my questions does not help much.
> 
> Today, most control plane has multiple GB of memory. 250KB (or 400KB) is a small portion of that. Then, why we still have seen issues in scaling Flex-Algo in reality?
> 
> For the flooding enhancement, it might help. The problem would be the weakest control plane in the network. You probably need a network simulation to prove the effectiveness. Do you agree?
> 
> Then, what is required to enhance in OSPF too?

the mechanisms described in ISIS fast flooding draft are applicable to 
OSPF as well.

> 
> For our proposed solution using offset, it "must" help because it reduces the overall size for either ISIS or OSPF deployment scenario, even applicable to the weakest control plane in the network.
> 
> For other questions listed below, I expect you could answer directly.
> It is a mutual respect and also be fair to all of us, as I tried to answer all challenging questions whenever I could.

I thought I have answered all of them already during the discussion. Let 
me try again, please see below.


> 
> Rgds
> Louis
> 
> -----Original Message-----
> From: Peter Psenak <ppsenak@cisco.com>
> Sent: Friday, April 14, 2023 10:38 PM
> To: Louis Chan <louisc@juniper.net>; Weiqiang Cheng <chengweiqiang@chinamobile.com>; 'Peter Psenak' <ppsenak=40cisco.com@dmarc.ietf.org>; 'linchangwang' <linchangwang.04414@h3c.com>; 'Les Ginsberg (ginsbe' <ginsberg@cisco.com>; 'Acee Lindem' <acee.ietf@gmail.com>
> Cc: 'lsr' <lsr@ietf.org>; Krzysztof Szarkowicz <kszarkowicz@juniper.net>
> Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset forFlex-Algorithm
> 
> [External Email. Be cautious of content]
> 
> 
> Louis,
> 
> On 14/04/2023 16:26, Louis Chan wrote:
>> Hi Peter (plus all),
>>
>> I do not think emotional statement could help. We should stick to facts and evidence, and be professional.
> 
> the fact is that with 9k LSP size and 255 LSPs, where each algo ADJ-SID
> takes up to 9 bytes, you have enough space to encode 250k of them.
> Do you need more? RFC7356 gives you the answer.
> 
> thanks,
> Peter
> 
>>
>> Please calm down, and enjoy your weekend first. We should have a drink when we meet.
>>
>> But there are questions in my mind. I must apologize being a bit blunt in asking following questions.
>>
>>
>> 1. When you said that
>> "I know bunch that deployed it and are happy with it. They needed few algos, which is what I consider the right usage of it."
>>
>> do these customers deploy the draft " draft-ietf-lsr-algorithm-related-adjacency-sid"?

not necessarily, but they do not deploy 100s of algos. I have questioned 
that usage of flex-algo already.

Using the analogy. If someone wants to redistribute 500k routes from BGP 
to ISIS, are we going to change the way we encode the prefix 
advertisememnt in ISIS? Unlikely. It's a bad design in a first place and 
whoever does it needs to face the consequencies. We can not modify 
protocol encodings to fix bad network designs.

>>
>> If no, why would you have conclusion that similar problem as Weiqiang have seen would not happen with your draft?
>>
>>
>> 2. When Weiqiang and others here points out they have live observation and analysis about the IGP flooding issues in Flex-Algo, why would you deny this kind of fact? Could you prove that their claims are wrong?

I have responded to that already. You claim that because we need to 
generate and flood multiple LSPs to fit the data, we slow the 
convergence. First, not necessarily - we have fast floodong mechanisms 
and as Chris pointed out, we only need to flood all LSPs in special 
cases. Second, there are many other reasons why we may need to generate 
many LSPs - e.g., lot of prefixes due to redistribution, many links with 
TE data, etc. Are we going to chnage all these encodings?


>>
>> 3. Also, I have a simple analysis of 2 core nodes. Could you response to that? Is the analysis reasonable or not?


your analysis has issues:
- you used LSP MTU of 1500. If you change to 9k, you will end up with 44 
LSPs
- if you use fast flooding, flooding 44 LSPs even across 10 hops may 
take you few hundred ms.


>>
>> 4. I have asked you a direct question of a real "concern", but not a personal preference

>>
>> " there are many TE attributes flooded for TE purposes, e.g., reserved/unreserved bandwidth (per pool), SRLGs, affinities, ...
>> You are picking Ajd-SID, but that is not the only thing that is advertised per link. You may have many SRLGs, many affinities, etc."
>>
>> Is this a real problem? or Is it just a personal preference statement?
>> Could you be more specific on the problem that you refer to?

my point was that there were many reasons why we may need advertise 
extensive data and generate many LSPs from a single router. The problem 
is not specific to flex-algo Adj-SIDs. And that we shoudl not chnage 
existing encodings to solve that problem. The problem can be solved by 
the generic mechanisms like fast flooding and larger LSP MTU.

thaks,
Peter

>>
>> Rgds
>> Louis
>>
>> -----Original Message-----
>> From: Peter Psenak <ppsenak@cisco.com>
>> Sent: Friday, April 14, 2023 7:03 PM
>> To: Weiqiang Cheng <chengweiqiang@chinamobile.com>; 'Peter Psenak' <ppsenak=40cisco.com@dmarc.ietf.org>; Louis Chan <louisc@juniper.net>; 'linchangwang' <linchangwang.04414@h3c.com>; 'Les Ginsberg (ginsbe' <ginsberg@cisco.com>; 'Acee Lindem' <acee.ietf@gmail.com>
>> Cc: 'lsr' <lsr@ietf.org>; Krzysztof Szarkowicz <kszarkowicz@juniper.net>
>> Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset forFlex-Algorithm
>>
>> [External Email. Be cautious of content]
>>
>>
>> Weiqiang,
>>
>> On 14/04/2023 12:28, Weiqiang Cheng wrote:
>>> Hi Peter,
>>> As far as I know, Flex Algo is not really deployed by operators by now, mainly because of the scalability concerns.
>>
>> I know bunch that deployed it and are happy with it. They needed few algos, which is what I consider the right usage of it.
>>
>> thanks,
>> Peter
>>
>>> In fact, the Luois team and our team did not know each other before. However, we proposed solutions for MPLS-SR and SRv6 respectively almost at the same time. So you can see it is a common problem for those operators who want to deploy FA at scale.
>>>
>>> As discussion in last few days, my observation is that we've agreed the problem is there.
>>> I think it's a good time to address it now.I suggest that We can discuss more on the solutions further, instead of discussing the requirements.
>>>
>>> B.R.
>>> Weiqiang Cheng
>>>
>>>
>>> -----邮件原件-----
>>> 发件人: Lsr [mailto:lsr-bounces@ietf.org] 代表 Peter Psenak
>>> 发送时间: 2023年4月14日 16:51
>>> 收件人: Louis Chan; linchangwang; 程伟强; Les Ginsberg (ginsbe; Acee Lindem
>>> 抄送: lsr; Krzysztof Szarkowicz
>>> 主题: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising Offset
>>> forFlex-Algorithm
>>>
>>> Louis,
>>>
>>> On 14/04/2023 10:25, Louis Chan wrote:
>>>> Hi Peter,
>>>>
>>>> I do not think we should divert the focus. It is about efficiency in packing information.
>>>
>>> trying to change the way the protocol encodes existing data is
>>> something that we should not do, unless there is a blocking issue that
>>> does not allow protocol to work anymore with existing encoding. You
>>> have not provided evidence of that. All the claims so far have been
>>> around the lines of implementation efficiency and can be addressed by
>>> existing protocol mechanism.
>>>
>>>>
>>>> If some important info is required to pass as "necessary evil", then it should.
>>>>
>>>> Actually, I am looking forward to applying similar method for other
>>>> metric or ASLA, proposed by someone, and not necessary by me. And I
>>>> have seen some drafts are addressing this kind of issues. > e.g. For
>>>> FA200 and FA201, most link metrics are sharing the same
>>> values, but only 5% difference in order to achieve some desired
>>> behavior. In this case, we should find a way to pack it efficiently.
>>>
>>> maybe you need a new version of the protocol to do what you propose.
>>>
>>> thanks,
>>> Peter
>>>
>>>>
>>>> Rgds
>>>> Louis
>>>>
>>>> -----Original Message-----
>>>> From: Peter Psenak <ppsenak@cisco.com>
>>>> Sent: Friday, April 14, 2023 3:32 PM
>>>> To: Louis Chan <louisc@juniper.net>; linchangwang
>>>> <linchangwang.04414@h3c.com>; 程伟强 <chengweiqiang@chinamobile.com>;
>>>> Les Ginsberg (ginsbe <ginsberg@cisco.com>; Acee Lindem
>>>> <acee.ietf@gmail.com>
>>>> Cc: lsr <lsr@ietf.org>; Krzysztof Szarkowicz
>>>> <kszarkowicz@juniper.net>
>>>> Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>>>> Offset forFlex-Algorithm
>>>>
>>>> [External Email. Be cautious of content]
>>>>
>>>>
>>>> Louis,
>>>>
>>>> On 14/04/2023 07:38, Louis Chan wrote:
>>>>> Hi Peter,
>>>>>
>>>>> For Adj-sid and additional TE requirement, I am not sure what you refer to.
>>>>> TE requirement or metric requirement? If it is metric related, we have ASLA draft to address some TE related problem.
>>>>> (To me, to reduce ASLA advertisement is required too.)
>>>>>
>>>>> What TE problem are you referring to? Could you give an example to illustrate you concern?
>>>>
>>>> there are many TE attributes flooded for TE purposes, e.g., reserved/unreserved bandwidth (per pool), SRLGs, affinities, ...
>>>>
>>>> You are picking Ajd-SID, but that is not the only thing that is advertised per link. You may have many SRLGs, many affinities, etc.
>>>>
>>>>
>>>> Peter
>>>>
>>>>>
>>>>> Rgds
>>>>> Louis
>>>>>
>>>>> -----Original Message-----
>>>>> From: Peter Psenak <ppsenak@cisco.com>
>>>>> Sent: Thursday, April 13, 2023 5:09 PM
>>>>> To: Louis Chan <louisc@juniper.net>; linchangwang
>>>>> <linchangwang.04414@h3c.com>; 程伟强 <chengweiqiang@chinamobile.com>;
>>>>> Les Ginsberg (ginsbe <ginsberg@cisco.com>; Acee Lindem
>>>>> <acee.ietf@gmail.com>
>>>>> Cc: lsr <lsr@ietf.org>; Krzysztof Szarkowicz
>>>>> <kszarkowicz@juniper.net>
>>>>> Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>>>>> Offset forFlex-Algorithm
>>>>>
>>>>> [External Email. Be cautious of content]
>>>>>
>>>>>
>>>>> Loius,
>>>>>
>>>>> there are many reasons why we need to advertise additional data for adjacency - TE being a major one. You are trying to optimize the Adj-SID only, which is not the major contributor anyway. The problem is not specific to Adj-SID.
>>>>>
>>>>> In terms of convergence, if you are worried about the flooding speed, there is a draft in progress:
>>>>>
>>>>> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-i
>>>>> et
>>>>> f-lsr-isis-fast-flooding/__;!!NEt6yMaO-gk!B1_2-Ht7ugsG_Wz7Ii0QGroMZy
>>>>> o9 HTCDcsgefE_pSNK-2bIfNoekducPbd5X3kQ5eMiRk98Jb_PYYg$
>>>>>
>>>>> thanks,
>>>>> Peter
>>>>>
>>>>>
>>>>>
>>>>> On 13/04/2023 10:52, Louis Chan wrote:
>>>>>> Hi Peter/all,
>>>>>>
>>>>>> Here I do a simple analysis on this scaling issue.
>>>>>>
>>>>>> 1. Assume a network with these parameters
>>>>>> - 20 x Flex-algo
>>>>>> - 2 x core nodes with 1,000 links
>>>>>> - network diameter with 5 hops
>>>>>>
>>>>>> 2. Just check out the additional advertisement size from core nodes following ChangWang example.
>>>>>>
>>>>>> For 1 core node,
>>>>>> n x 20 x 1000
>>>>>> MPLS-SR:If n = 10 bytes, it is 200K bytes per core node
>>>>>>
>>>>>> With 2 core nodes, it is 400KB in total
>>>>>>
>>>>>> LSP num: 400KB/1500 = 267 lsps, at least
>>>>>>
>>>>>> 3. About the delivery/flooding rate, from
>>>>>> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-
>>>>>> ie
>>>>>> t
>>>>>> f-lsr-isis-fast-flooding/__;!!NEt6yMaO-gk!B1_2-Ht7ugsG_Wz7Ii0QGroMZ
>>>>>> yo
>>>>>> 9 HTCDcsgefE_pSNK-2bIfNoekducPbd5X3kQ5eMiRk98Jb_PYYg$
>>>>>>>>>
>>>>>>         As IS-IS is deployed in greater scale both in the number of nodes in
>>>>>>          an area and in the number of neighbors per node, the impact of the
>>>>>>          historic flooding rates becomes more significant.  Consider the
>>>>>>          bringup or failure of a node with 1000 neighbors.  This will result          <--- 1000 adj links
>>>>>>          in a minimum of 1000 LSP updates.  At typical LSP flooding rates used                <--- imply 1000 LSP updates
>>>>>>          today (33 LSPs/second), it would take 30+ seconds simply to send the         <--- 33lsp/s
>>>>>>          updated LSPs to a given neighbor.  Depending on the diameter of the
>>>>>>          network, achieving a consistent LSDB on all nodes in the network
>>>>>>          could easily take a minute or more.                                          <--- at least double
>>>>>> <<<
>>>>>>
>>>>>> 267/33 = 8.1 sec
>>>>>>
>>>>>>
>>>>>> With a network diameter of 5, the additional time for delivering the consistent LSDB to all remote nodes =
>>>>>> m x 8.1 sec,    where 1 < m < 5 due to inefficiency or implementation issue
>>>>>>
>>>>>> It is likely 16+ sec, according to the above description in that draft.
>>>>>>
>>>>>> If using offset solution, it is around 0.008sec x 2 = 0.016sec in additional. This number is small.
>>>>>>
>>>>>> Additional of 16+ sec is significant in global convergence time.
>>>>>>
>>>>>> 4. This model/example does not include nodes from second layer, which also has 2 x 1,000 adj-sid in the reverse direction. The total number would be estimated bigger than 30+ sec.
>>>>>>
>>>>>> Should this be improved?
>>>>>>
>>>>>> 5. Flooding could be in all directions. Larger size of advertisement could delay some important update in busy period. i.e. smaller size in advertisement is better.
>>>>>> And I assume this draft with offset would also reduce the refresh requirement.
>>>>>>
>>>>>> Rgds
>>>>>> Louis
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Peter Psenak <ppsenak@cisco.com>
>>>>>> Sent: Wednesday, April 12, 2023 11:26 PM
>>>>>> To: linchangwang <linchangwang.04414@h3c.com>; 程伟强
>>>>>> <chengweiqiang@chinamobile.com>; Louis Chan <louisc@juniper.net>;
>>>>>> Les Ginsberg (ginsbe <ginsberg@cisco.com>; Acee Lindem
>>>>>> <acee.ietf@gmail.com>
>>>>>> Cc: lsr <lsr@ietf.org>; Krzysztof Szarkowicz
>>>>>> <kszarkowicz@juniper.net>
>>>>>> Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>>>>>> Offset forFlex-Algorithm
>>>>>>
>>>>>> [External Email. Be cautious of content]
>>>>>>
>>>>>>
>>>>>> Hi Changwang,
>>>>>>
>>>>>> please see inline ##PP3:
>>>>>>
>>>>>> On 12/04/2023 16:46, linchangwang wrote:
>>>>>>> Hi Peter,
>>>>>>>
>>>>>>>
>>>>>>> Please see inline [changwang lin].
>>>>>>>
>>>>>>>> Changwang,
>>>>>>>
>>>>>>>> please see inline (##PP2):
>>>>>>>
>>>>>>>
>>>>>>> On 12/04/2023 15:13, linchangwang wrote:
>>>>>>>> Hi Peter
>>>>>>>>
>>>>>>>>          Please see inline [changwang lin].
>>>>>>>>
>>>>>>>>> We've met the same problem when applying Flex Algo in SRv6 network.
>>>>>>>>
>>>>>>>> what problem exactly, can you please describe it?
>>>>>>>>
>>>>>>>> [changwang lin]
>>>>>>>> Advertisement size of per Flex-Algo Adj-SID in the network
>>>>>>>> Related to F(# of node, # of FA, # of links) For a node with
>>>>>>>> 1,000 links and
>>>>>>>> 20 Flex-Algo
>>>>>>>>            n x 20 x 1000
>>>>>>>>            MPLS-SR:If n = 10 bytes, it is 200K bytes
>>>>>>>>            SRv6:   If n = 24 bytes, it is 400K+ bytes
>>>>>>>> If 500 nodes:
>>>>>>>>            MPLS-SR:it is 200K*500   =  100000k bytes
>>>>>>>>            SRv6:   it is 400K+ * 500  = 200000k bytes
>>>>>>>> If interface mtu=1500, lsp length = 1497
>>>>>>>>          LSPs num:
>>>>>>>>            MPLS-SR:10000k bytes/1497 = 66800  lsps
>>>>>>>>            SRv6:   20000k bytes/1497 = 160320 lsps
>>>>>>>>
>>>>>>>> The number of LSPs is too large, and IS-IS needs to periodically
>>>>>>>> refresh LSPs, resulting in a decrease in ISIS performance and unstable network operation.
>>>>>>>
>>>>>>> ##PP2
>>>>>>> above is hardly a realistic estimation.
>>>>>>>
>>>>>>> In a network with 1k nodes, not every node will have 1k links.
>>>>>>>
>>>>>>> Advertising large number of LSPs is not caused by Adj-SIDs.
>>>>>>> With TE enabled the amount of data flooded per link is larger than
>>>>>>> advertisement of the 20 Adj-SID. The problem you are highlighting
>>>>>>> is not specific to Adj-SIDs, it's generic.
>>>>>>>
>>>>>>> LSP refresh time can be set to 18 hours and any reasonable
>>>>>>> implementation does not refresh all LSPs at the same time.
>>>>>>>
>>>>>>> [changwang lin]
>>>>>>> This problem exists in actual operator networking, it can be calculated based on an actual network as follows:
>>>>>>>         One network with 200 nodes
>>>>>>>         One node with 20 interfaces
>>>>>>>         One interface with 32 flex algos Each interface is assigned
>>>>>>> two types of end. x, one PSP and one non PSP, with each end. x
>>>>>>> occupying
>>>>>>> 30 bytes An nbr tlv with basic bandwidth, EAG, and interface
>>>>>>> address is approximately 140 bytes Number of LSPs in the entire
>>>>>>> network: 140
>>>>>>> * 20 * 32 * 200/1497=12000 LSPs
>>>>>>>
>>>>>>> The performance of IGP has always been affected by the size of the
>>>>>>> entire network's LSDB, and even if the periodic flooding time is reduced, there will still be convergence issues.
>>>>>>
>>>>>> ##PP3
>>>>>> I don't see any relationship between the convergence and the fact that you have to advertise 20 ADJ-SIDs per link. If there is one, it's an implementation problem.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> So we need to optimize on the control surface to save LSP space.
>>>>>>>
>>>>>>> ##PP2
>>>>>>> with all the respect, I don't agree. The problem as you described
>>>>>>> it does not exist.
>>>>>>> [changwang lin]
>>>>>>> In the actual deployment of MPLS-SR and SRv6 networks, as the
>>>>>>> number of flex algos and interfaces increases, the space occupied by adj-sids becomes larger and larger.
>>>>>>> This is the actual problem when deploying flex algos.
>>>>>>
>>>>>> ##PP3
>>>>>> I don't see any protocol problem.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> Through the optimization notification mechanism mentioned in
>>>>>>>> these two documents, we have greatly saved LSP space for IS-IS and improved the performance of IS-IS flex algo in large-scale networking.
>>>>>>>> At the same time, through the VFA mechanism, in other non flex algo application scenarios,
>>>>>>>>          such as network slicing scenarios, the LSP space of IS-IS
>>>>>>>> can also be saved
>>>>>>>
>>>>>>> ##PP2
>>>>>>> it seems to me you are trying to fix the implementation problem
>>>>>>> with the protocol changes, which is never a good idea.
>>>>>>>
>>>>>>> [changwang lin]
>>>>>>> In actual deployment, a flex algo corresponds to the SLA
>>>>>>> requirements of a service, such as different bandwidth guarantees,
>>>>>>> 128 flex algos can correspond to 128 service requirements.
>>>>>>> When the flex algo specification increases, the corresponding number of LSPs rapidly increases, making it impossible to deploy large flex algo applications.
>>>>>>> And the mechanism of this draft can greatly improve the space utilization of LSP..
>>>>>>
>>>>>> ##PP3
>>>>>> I appreciate your effort, but I don't believe the proposed compression is needed, nor that it addresses the problem you have.
>>>>>>
>>>>>> The amount of data being flooded per adjacency may potentially be large and Adj-SIDs only represent a fraction of it - even with 32 Adj-SIDs per link you are not hitting any protocol limitation.
>>>>>>
>>>>>> thanks,
>>>>>> Peter
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> thanks,
>>>>>>> Changwang lin
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Peter
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Changwang lin
>>>>>>>>
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Lsr [mailto:lsr-bounces@ietf.org] On Behalf Of Peter Psenak
>>>>>>>> Sent: Wednesday, April 12, 2023 7:10 PM
>>>>>>>> To: 程伟强; Louis Chan; Les Ginsberg (ginsbe; Acee Lindem
>>>>>>>> Cc: lsr; Krzysztof Szarkowicz
>>>>>>>> Subject: Re: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>>>>>>>> Offset forFlex-Algorithm
>>>>>>>>
>>>>>>>> Weiqiang,
>>>>>>>>
>>>>>>>> please see inline (##PP):
>>>>>>>>
>>>>>>>> On 12/04/2023 12:05, 程伟强 wrote:
>>>>>>>>> Hi Louis and Les,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> My two cents from operator perspective.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> We've met the same problem when applying Flex Algo in SRv6 network.
>>>>>>>>
>>>>>>>> what problem exactly, can you please describe it?
>>>>>>>>
>>>>>>>> [changwang lin]
>>>>>>>> Advertisement size of per Flex-Algo Adj-SID in the network
>>>>>>>> Related to F(# of node, # of FA, # of links) For a node with
>>>>>>>> 1,000 links and
>>>>>>>> 20 Flex-Algo
>>>>>>>>            n x 20 x 1000
>>>>>>>>            MPLS-SR:If n = 10 bytes, it is 200K bytes
>>>>>>>>            SRv6:   If n = 24 bytes, it is 400K+ bytes
>>>>>>>> If 500 nodes:
>>>>>>>>            MPLS-SR:it is 200K*500   =  100000k bytes
>>>>>>>>            SRv6:   it is 400K+ * 500  = 200000k bytes
>>>>>>>> If interface mtu=1500, lsp length = 1497
>>>>>>>>          LSP num:
>>>>>>>>            MPLS-SR:10000k bytes/1497 = 66800  lsps
>>>>>>>>            SRv6:   20000k bytes/1497 = 160320 lsps
>>>>>>>>
>>>>>>>> The number of LSPs is too large, and IS-IS needs to periodically
>>>>>>>> refresh LSPs, resulting in a decrease in ISIS performance and unstable network operation.
>>>>>>>>
>>>>>>>> So we need to optimize on the control surface to save LSP space.
>>>>>>>> Through the optimization notification mechanism mentioned in
>>>>>>>> these two documents, we have greatly saved LSP space for IS-IS and improved the performance of IS-IS flex algo in large-scale networking.
>>>>>>>> At the same time, through the VFA mechanism, in other non flex algo application scenarios,
>>>>>>>>          such as network slicing scenarios, the LSP space of IS-IS
>>>>>>>> can also be saved
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> As the number of slices and the scale of the network increases,
>>>>>>>>> the convergence issue which is caused by SIDs  advertising and
>>>>>>>>> flooding becomes more and more serious.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Due to the problem, it is impossible to apply Flex-Algo in the
>>>>>>>>> large network, such as the network with more than 1000 routers.
>>>>>>>>
>>>>>>>> flex-algo has been successfully deployed in a networks that have
>>>>>>>> more that 1k nodes.
>>>>>>>>
>>>>>>>> Maybe you want deploy the flex-algo for something that it was not
>>>>>>>> designed for.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I believe Louis'draft provides a good idea to resolve this problem.
>>>>>>>>> Similar solution for SRv6 SIDs is described in another draft.
>>>>>>>>
>>>>>>>> Again, what problem exactly?
>>>>>>>>
>>>>>>>>          From what I see the drafts tries to pack algo SIDs to save
>>>>>>>> space in LSP. I don't see how it helps to to deploy flex-algo in
>>>>>>>> a large scale network.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Peter
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> About the SIDs assignment, I think it is better to have a
>>>>>>>>> scheduled assignment than a random assignment as Les mentioned.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/dra
>>>>>>>>> ft
>>>>>>>>> -
>>>>>>>>> c
>>>>>>>>> heng-lsr-isis-srv6-sid-block/__;!!NEt6yMaO-gk!EX4RtSVg7I0jO4M9ZM
>>>>>>>>> rt c j Gl2sqxcDIFHFZRnk4pocYHUxv9_pAb8cfeVdGYX_3gOEDi1kmvLCcctw$
>>>>>>>>> <https://urldefense.com/v3/__https://datatracker.ietf.org/doc/dr
>>>>>>>>> af
>>>>>>>>> t
>>>>>>>>> -
>>>>>>>>> cheng-lsr-isis-srv6-sid-block/__;!!NEt6yMaO-gk!EX4RtSVg7I0jO4M9Z
>>>>>>>>> Mr t c
>>>>>>>>> jGl2sqxcDIFHFZRnk4pocYHUxv9_pAb8cfeVdGYX_3gOEDi1kmvLCcctw$ >
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Weiqiang Cheng
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>             ----邮件原文----
>>>>>>>>>             *发件人:*Louis Chan  <louisc=40juniper.net@dmarc.ietf.org>
>>>>>>>>>             *收件
>>>>>>>>>             人:*"Les Ginsberg (ginsberg)" <ginsberg@cisco.com>,Acee Lindem <acee.ietf@gmail.com>
>>>>>>>>>             *抄 送:
>>>>>>>>>             *lsr  <lsr@ietf.org>,Krzysztof Szarkowicz  <kszarkowicz@juniper.net>,Weiqiang Cheng  <chengweiqiang@chinamobile.com>
>>>>>>>>>             *发送时间:*2023-04-12 10:45:56
>>>>>>>>>             *主题:*Re: [Lsr] IETF-
>>>>>>>>>             116 LSR - IGP extensions for Advertising Offset
>>>>>>>>> forFlex-Algorithm
>>>>>>>>>
>>>>>>>>>             Hi Les,
>>>>>>>>>
>>>>>>>>>             Thanks for the prompt reply. Please see inline for clarification [lc2].
>>>>>>>>>
>>>>>>>>>             /Louis
>>>>>>>>>
>>>>>>>>>             *From:* Les Ginsberg (ginsberg) <ginsberg@cisco.com>
>>>>>>>>>             *Sent:* Tuesday, April 11, 2023 11:03 PM
>>>>>>>>>             *To:* Louis Chan <louisc@juniper.net>; Acee Lindem <acee.ietf@gmail.com>
>>>>>>>>>             *Cc:* lsr <lsr@ietf.org>; Krzysztof Szarkowicz
>>>>>>>>>             <kszarkowicz@juniper.net>; Weiqiang Cheng
>>>>>>>>>             <chengweiqiang@chinamobile.com>
>>>>>>>>>             *Subject:* RE: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>>>>>>>>>             Offset for Flex-Algorithm
>>>>>>>>>
>>>>>>>>>             *[External Email. Be cautious of content]*
>>>>>>>>>
>>>>>>>>>             Louis -
>>>>>>>>>
>>>>>>>>>             Please see inline.
>>>>>>>>>
>>>>>>>>>              > -----Original Message-----
>>>>>>>>>
>>>>>>>>>              > From: Louis Chan <louisc@juniper.net
>>>>>>>>> <mailto:louisc@juniper.net>>
>>>>>>>>>
>>>>>>>>>              > Sent: Monday, April 10, 2023 11:01 PM
>>>>>>>>>
>>>>>>>>>              > To: Les Ginsberg (ginsberg) <ginsberg@cisco.com
>>>>>>>>>             <mailto:ginsberg@cisco.com>>; Acee Lindem
>>>>>>>>>
>>>>>>>>>              > <acee.ietf@gmail.com <mailto:acee.ietf@gmail.com>>
>>>>>>>>>
>>>>>>>>>              > Cc: lsr <lsr@ietf.org <mailto:lsr@ietf.org>>; Krzysztof
>>>>>>>>>             Szarkowicz <kszarkowicz@juniper.net
>>>>>>>>> <mailto:kszarkowicz@juniper.net>>;
>>>>>>>>>
>>>>>>>>>              > Weiqiang Cheng <chengweiqiang@chinamobile.com
>>>>>>>>>             <mailto:chengweiqiang@chinamobile.com>>
>>>>>>>>>
>>>>>>>>>              > Subject: RE: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>>>>>>>>>             Offset for
>>>>>>>>>
>>>>>>>>>              > Flex-Algorithm
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > Hi Les,
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > Thanks for your questions. Please see inline [lc] below.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > /Louis
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > -----Original Message-----
>>>>>>>>>
>>>>>>>>>              > From: Les Ginsberg (ginsberg) <ginsberg@cisco.com
>>>>>>>>>             <mailto:ginsberg@cisco.com>>
>>>>>>>>>
>>>>>>>>>              > Sent: Saturday, April 8, 2023 7:34 AM
>>>>>>>>>
>>>>>>>>>              > To: Acee Lindem <acee.ietf@gmail.com
>>>>>>>>>             <mailto:acee.ietf@gmail.com>>; Louis Chan <louisc@juniper.net
>>>>>>>>>             <mailto:louisc@juniper.net>>
>>>>>>>>>
>>>>>>>>>              > Cc: lsr <lsr@ietf.org <mailto:lsr@ietf.org>>
>>>>>>>>>
>>>>>>>>>              > Subject: RE: [Lsr] IETF-116 LSR - IGP extensions for Advertising
>>>>>>>>>             Offset for
>>>>>>>>>
>>>>>>>>>              > Flex-Algorithm
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [External Email. Be cautious of content]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > OK - since Acee opened the door - here are some
>>>>>>>>> comments from me -
>>>>>>>>>
>>>>>>>>>              > starting with the most important.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > (BTW - I still have limited enthusiasm for this
>>>>>>>>> draft.)
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > 1)The proposal places some restrictions on how operators
>>>>>>>>>             provision their
>>>>>>>>>
>>>>>>>>>              > network in terms of assigning SIDs and reserving
>>>>>>>>> space for future
>>>>>>>>>
>>>>>>>>>              > assignments.
>>>>>>>>>
>>>>>>>>>              > If operators do not use compatible assignment schemes, then this
>>>>>>>>>             will never
>>>>>>>>>
>>>>>>>>>              > get deployed. It is therefore not enough to come with a nice idea
>>>>>>>>>             - you must
>>>>>>>>>
>>>>>>>>>              > have some enthusiasm from the operator community.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] If the operator only wants to deploy flex-algo, there is no
>>>>>>>>>             change in their
>>>>>>>>>
>>>>>>>>>              > Node-sid numbering scheme. For the Adj-sid, these are local
>>>>>>>>>             labels with local
>>>>>>>>>
>>>>>>>>>              > significant only, and there is no need for any special planning
>>>>>>>>>             for Adj-sid,
>>>>>>>>>
>>>>>>>>>              > unless you are suggesting they want to make fixed assignment of
>>>>>>>>>             Adj-sid
>>>>>>>>>
>>>>>>>>>              > label for each link. Even with fixed, the proposed draft has
>>>>>>>>>             benefit on that. I
>>>>>>>>>
>>>>>>>>>              > will explain later.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>             [LES:] Let's discuss this in the context of prefix-sids - the same
>>>>>>>>>             applies to adj-sids.
>>>>>>>>>
>>>>>>>>>             Today (i.e., in the absence of your proposal) an operator is free to
>>>>>>>>>             assign any label within the SRGB for a given prefix/algo pair so
>>>>>>>>>             long as it is not assigned to some other prefix/algo context.
>>>>>>>>>
>>>>>>>>>             Your proposal places some new restrictions. Now, for a given
>>>>>>>>>             flex-algo, whenever an operator assigns a given label for a prefix
>>>>>>>>>             in Algo 0 (call it Label-A0), they must guarantee that
>>>>>>>>>             "Label-A0+offset" for an  advertised flex-algo specific offset is
>>>>>>>>>             available to be assigned for the prefix/flex-algo pair - and this
>>>>>>>>>             must be true for all prefixes advertised in algo 0.
>>>>>>>>>
>>>>>>>>>             This is certainly possible to do, but is not guaranteed to be the
>>>>>>>>>             case in current deployments.
>>>>>>>>>
>>>>>>>>>             For example - and this is only an example...today an operator might
>>>>>>>>>             utilize a provisioning tool to assign prefix-sids for all supported
>>>>>>>>>             algorithms on all nodes in the network.
>>>>>>>>>
>>>>>>>>>             To do this, the tool might maintain a database of assigned labels.
>>>>>>>>>             When provisioning a new node/prefix/algorithm, the logic in the tool
>>>>>>>>>             might simply take the next available label in the database.
>>>>>>>>>
>>>>>>>>>             The result of this would not be consistent with the requirements of
>>>>>>>>>             your draft.
>>>>>>>>>
>>>>>>>>>             Which is why I say in order to deploy the extension you propose,
>>>>>>>>>             such an operator would have to modify its provisioning tool.
>>>>>>>>>
>>>>>>>>>             [lc2] There might be some misunderstanding of our proposal. Let me
>>>>>>>>>             give some examples.
>>>>>>>>>
>>>>>>>>>             Case 1: Flex-Algo only
>>>>>>>>>
>>>>>>>>>             Prefix offset advertisement: “no”
>>>>>>>>>
>>>>>>>>>             Adj-sid offset advertisement: yes
>>>>>>>>>
>>>>>>>>>             In slide 8’s example, FA129 is using label “402001”, and the
>>>>>>>>>             advertisement of this label is using existing methods.
>>>>>>>>>
>>>>>>>>>             e.g. SRGB = 400000-460000
>>>>>>>>>
>>>>>>>>>             FA129: index 2001 (preferred value), or one can choose
>>>>>>>>> 111,
>>>>>>>>> 222
>>>>>>>>>
>>>>>>>>>             FA130 (new): index 3001 (preferred value), or 333,
>>>>>>>>> 4444
>>>>>>>>>
>>>>>>>>>             This does not change how the operator to assign label for prefix-sid
>>>>>>>>>             with their current method. Any index/label could be used for FA
>>>>>>>>>             prefix within SRGB.
>>>>>>>>>
>>>>>>>>>             The only change is the Adj-sid label allocation, but this “mostly”
>>>>>>>>>             is  only “local” to one node. There is no effect on global label
>>>>>>>>>             allocation.
>>>>>>>>>
>>>>>>>>>             This draft will be compatible to what operators are doing today.
>>>>>>>>>
>>>>>>>>>             Case 2: VFA only
>>>>>>>>>
>>>>>>>>>             Prefix offset advertisement: yes
>>>>>>>>>
>>>>>>>>>             Adj-sid offset advertisement: yes
>>>>>>>>>
>>>>>>>>>             I agree, with VFA, there would be impact to global allocation to
>>>>>>>>>             node-sid/prefix-sid. But VFA is a totally new concept. No one has
>>>>>>>>>             deployed that yet.
>>>>>>>>>
>>>>>>>>>             There is no impact to operators which stick to deploy only Flex-algo.
>>>>>>>>>
>>>>>>>>>             Other case: Flex-Algo w/o Adj-sid offset
>>>>>>>>>
>>>>>>>>>                            Continue the example of Case#1 above
>>>>>>>>>
>>>>>>>>>                            Another FA131 is added, but no Adj-sid offset is
>>>>>>>>>             advertised
>>>>>>>>>
>>>>>>>>>                            The question would be
>>>>>>>>>
>>>>>>>>>               *
>>>>>>>>>
>>>>>>>>>                 Either allow this configuration, and FA131 will fallback using
>>>>>>>>>                 Algo 0’s  adj-sid
>>>>>>>>>
>>>>>>>>>               *
>>>>>>>>>
>>>>>>>>>                 Or, disallow this configuration
>>>>>>>>>
>>>>>>>>>             I tend to “allow” such configuration with mix of FA129, FA130 (with
>>>>>>>>>             adj-sid offset) and FA131 (w/o adj-sid offset)
>>>>>>>>>
>>>>>>>>>             [/lc2]
>>>>>>>>>
>>>>>>>>>             Could this be done? Sure.
>>>>>>>>>
>>>>>>>>>             Do operators want to do this? I do not know.
>>>>>>>>>
>>>>>>>>>             But since this would be necessary in order to use your proposed
>>>>>>>>>             extension, it is necessary to gauge operator enthusiasm for making
>>>>>>>>>             such changes in order to know whether there is any point in
>>>>>>>>>             proceeding with  your proposal.
>>>>>>>>>
>>>>>>>>>              > In slide 8 of the below presentation
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>> https://urldefense.com/v3/__https://datatracker.ietf.org/meeting
>>>>>>>>> /1
>>>>>>>>> 1
>>>>>>>>> 6
>>>>>>>>> /materials/slides-116-lsr-03-ietf116-__;!!NEt6yMaO-gk!EX4RtSVg7I
>>>>>>>>> 0j
>>>>>>>>> O
>>>>>>>>> 4
>>>>>>>>> M9ZMrtcjGl2sqxcDIFHFZRnk4pocYHUxv9_pAb8cfeVdGYX_3gOEDi1knGgpMXww
>>>>>>>>> $
>>>>>>>>> <https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting
>>>>>>>>> /1
>>>>>>>>> 1
>>>>>>>>> 6
>>>>>>>>> /materials/slides-116-lsr-03-ietf116-igp-adv-offset-01__;!!NEt6y
>>>>>>>>> Ma
>>>>>>>>> O
>>>>>>>>> -
>>>>>>>>> gk!Bl7Swe9ql9VT0qGkD6FoZZzTWT2fmIx55eSncdmMgoCJetJ5-80micuqnqk79
>>>>>>>>> ye
>>>>>>>>> w
>>>>>>>>> G
>>>>>>>>> B-BleOfrYpSjfI$>
>>>>>>>>>
>>>>>>>>>             >  igp-adv-offset-01
>>>>>>>>>
>>>>>>>>> <https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting
>>>>>>>>> /1
>>>>>>>>> 1
>>>>>>>>> 6
>>>>>>>>> /materials/slides-116-lsr-03-ietf116-igp-adv-offset-01__;!!NEt6y
>>>>>>>>> Ma
>>>>>>>>> O
>>>>>>>>> -
>>>>>>>>> gk!Bl7Swe9ql9VT0qGkD6FoZZzTWT2fmIx55eSncdmMgoCJetJ5-80micuqnqk79
>>>>>>>>> ye
>>>>>>>>> w
>>>>>>>>> G
>>>>>>>>> B-BleOfrYpSjfI$>
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > FA129 is a prefix-sid (400201) allocated by operator, and it can
>>>>>>>>>             be any label.
>>>>>>>>>
>>>>>>>>>              > There is no connection to how Adj-sid is derived.
>>>>>>>>>
>>>>>>>>>              > Per Flex-Algo adj-sid assignment is not affecting
>>>>>>>>> network wide label
>>>>>>>>>
>>>>>>>>>              > assignment from operation perspective. Each node could have
>>>>>>>>>             different local
>>>>>>>>>
>>>>>>>>>              > block for such adj-sid assignment. One might need to estimate
>>>>>>>>>             total possible
>>>>>>>>>
>>>>>>>>>              > number of link in one chassis for such allocation, or it could be
>>>>>>>>>             estimated by
>>>>>>>>>
>>>>>>>>>              > OS software itself. I also mentioned in the session, if there is
>>>>>>>>>             100 x FA with
>>>>>>>>>
>>>>>>>>>              > 1000 links (high end), it is only 100k labels. Is it difficult to
>>>>>>>>>             allocate such.
>>>>>>>>>
>>>>>>>>>             [LES:] Your proposal does not reduce the number of labels which need
>>>>>>>>>             to be "allocated" and installed in forwarding, it only reduces the
>>>>>>>>>             number of bytes used to advertise this information in LSPs/LSAs.
>>>>>>>>>
>>>>>>>>>             [lc2] You are correct.
>>>>>>>>>
>>>>>>>>>             I think, at the same time, it helps reducing time for global
>>>>>>>>>             convergence since the advertisement size is smaller, especially in
>>>>>>>>>             network with long diameter with multi-hops.
>>>>>>>>>
>>>>>>>>>             Also, in
>>>>>>>>>
>>>>>>>>> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/dra
>>>>>>>>> ft
>>>>>>>>> -
>>>>>>>>> ietf-lsr-isis-fast-flooding/__;!!NEt6yMaO-gk!EX4RtSVg7I0jO4M9ZMr
>>>>>>>>> tc j Gl2sqxcDIFHFZRnk4pocYHUxv9_pAb8cfeVdGYX_3gOEDi1kl_1f1RjQ$
>>>>>>>>>
>>>>>>>>> <https://urldefense.com/v3/__https://datatracker.ietf.org/doc/dr
>>>>>>>>> af
>>>>>>>>> t
>>>>>>>>> -
>>>>>>>>> ietf-lsr-isis-fast-flooding/__;!!NEt6yMaO-gk!EX4RtSVg7I0jO4M9ZMr
>>>>>>>>> tc j G l2sqxcDIFHFZRnk4pocYHUxv9_pAb8cfeVdGYX_3gOEDi1kl_1f1RjQ$
>>>>>>>>>> (which you participated in)
>>>>>>>>>
>>>>>>>>>              >>>
>>>>>>>>>
>>>>>>>>>                As IS-IS is deployed in greater scale both in the
>>>>>>>>> number of nodes in
>>>>>>>>>
>>>>>>>>>                 an area and in the number of neighbors per node,
>>>>>>>>> the impact of the
>>>>>>>>>
>>>>>>>>>                 historic flooding rates becomes more significant.
>>>>>>>>> Consider the
>>>>>>>>>
>>>>>>>>>                 bringup or failure of a node with 1000 neighbors.
>>>>>>>>> This will result
>>>>>>>>>
>>>>>>>>>                 in a minimum of 1000 LSP updates.  At typical LSP flooding rates
>>>>>>>>>             used
>>>>>>>>>
>>>>>>>>>                 today (33 LSPs/second), it would take 30+ seconds
>>>>>>>>> simply to send the
>>>>>>>>>
>>>>>>>>>                 updated LSPs to a given neighbor.  Depending on
>>>>>>>>> the diameter of the
>>>>>>>>>
>>>>>>>>>                 network, achieving a consistent LSDB on all nodes
>>>>>>>>> in the network
>>>>>>>>>
>>>>>>>>>                 could easily take a minute or more.
>>>>>>>>>
>>>>>>>>>             <<<
>>>>>>>>>
>>>>>>>>>             This proposed draft will certainly help.
>>>>>>>>>
>>>>>>>>>             [/lc2]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > So, this is why I do not understand your question in full.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > If the operator plans to use VFA, that would be a different
>>>>>>>>>             discussion. VFA,
>>>>>>>>>
>>>>>>>>>              > today, does not exist in deployment.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>             [LES:] My comments here have nothing to do with VFA.
>>>>>>>>>
>>>>>>>>>              > [/lc]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > Have you discussed this idea with any operators?
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] I include Wei-qiang from China mobile in this thread. He has
>>>>>>>>>             shown his
>>>>>>>>>
>>>>>>>>>              > need on this kind of solution. Maybe, he could give his
>>>>>>>>>             perspective here. [/lc]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > If so, what has been their response?
>>>>>>>>>
>>>>>>>>>              > If they are open to the idea, how might they migrate from their
>>>>>>>>>             existing
>>>>>>>>>
>>>>>>>>>              > assignment schemes to an assignment scheme
>>>>>>>>> compatible with the
>>>>>>>>>
>>>>>>>>>              > proposal?
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > These are questions that need to be answered before considering
>>>>>>>>>             this idea.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] In slide 8, if you see these label numbers
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > Prefix-sid: 400001, 402001, 406001, 407001
>>>>>>>>>
>>>>>>>>>              > Adj-sid: 32, 2032, 6032, 7032
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > From operator perspective or troubleshooting perspective, value
>>>>>>>>>             xxx001
>>>>>>>>>
>>>>>>>>>              > represent the same node, and value x032 represent
>>>>>>>>> the same link. This
>>>>>>>>>
>>>>>>>>>              > makes things more organized and easier to understand.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > If all are random labels, I do not see any benefit at all.
>>>>>>>>>
>>>>>>>>>              > [/lc]
>>>>>>>>>
>>>>>>>>>             [LES:] I am not commenting on whether the label assignment scheme
>>>>>>>>>             you propose is better or worse than any other.
>>>>>>>>>
>>>>>>>>>             I am only pointing out that you are imposing new restrictions on how
>>>>>>>>>             labels are allocated.
>>>>>>>>>
>>>>>>>>>             As you are not in charge of how operators provision their networks
>>>>>>>>>             (nor am I), it is presumptuous of you to think that simply because
>>>>>>>>>             you think this is a better way to do things that operators will be
>>>>>>>>>             happy to modify their existing networks  to conform to your proposed
>>>>>>>>>             restrictions.
>>>>>>>>>
>>>>>>>>>             This isn’t academia - you need to vet this with the operator community.
>>>>>>>>>
>>>>>>>>>             [lc2] Please refer to the examples at the top. The picture should be
>>>>>>>>>             clear by now. There is no restriction to what is
>>>>>>>>> deployed today. [/lc2]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > 2)Section 5 Compatibilty
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > There is no "compatibility" with legacy nodes - because all nodes
>>>>>>>>>             in the
>>>>>>>>>
>>>>>>>>>              > network have to have a consistent understanding of what SID is
>>>>>>>>>             assigned to a
>>>>>>>>>
>>>>>>>>>              > given context (for prefixes and adjacencies) since they might
>>>>>>>>>             need to install
>>>>>>>>>
>>>>>>>>>              > forwarding entries for that context.
>>>>>>>>>
>>>>>>>>>              > I do not see any point in deploying this until all nodes support
>>>>>>>>>             it. If you did do
>>>>>>>>>
>>>>>>>>>              > so, you would need to advertise old and new forms -
>>>>>>>>> which does the
>>>>>>>>>
>>>>>>>>>              > opposite of what you are trying to achieve. Instead of reducing
>>>>>>>>>             LSP space
>>>>>>>>>
>>>>>>>>>              > used you would increase it.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] If the operator just plans to use only Flex-Algo, and no
>>>>>>>>>             VFA, it should be
>>>>>>>>>
>>>>>>>>>              > compatible with legacy implementation. If legacy nodes do not
>>>>>>>>>             understand
>>>>>>>>>
>>>>>>>>>              > adj-sid offset notation, these nodes could just ignore it. The
>>>>>>>>>             forwarding
>>>>>>>>>
>>>>>>>>>              > plane should work with co-existence of old and new nodes. Per
>>>>>>>>>             flex-algo adj-
>>>>>>>>>
>>>>>>>>>              > sid is only local significant to one node. New nodes should
>>>>>>>>>             detect whether
>>>>>>>>>
>>>>>>>>>              > legacy nodes exist in the network via such new extension
>>>>>>>>>             advertisement.
>>>>>>>>>
>>>>>>>>>              > And new nodes should use only algo 0 adj-sid from legacy nodes
>>>>>>>>>             for any TI-
>>>>>>>>>
>>>>>>>>>              > LFA.
>>>>>>>>>
>>>>>>>>>             [LES:] Consider a network of 100 nodes.
>>>>>>>>>
>>>>>>>>>             Let's say the "left-hand-side" of the network consist of legacy
>>>>>>>>>             nodes who do not understand your new advertisements.
>>>>>>>>>
>>>>>>>>>             The "right-hand-side" of the network consists of upgraded nodes who
>>>>>>>>>             support the new advertisements.
>>>>>>>>>
>>>>>>>>>             Consider nodes PE-LEFT and PE-RIGHT.
>>>>>>>>>
>>>>>>>>>             PE-RIGHT advertises a prefix-SID of 1000 for 2.2.2.2/32, and an
>>>>>>>>>             offset of 1000 for flex-algo 128.
>>>>>>>>>
>>>>>>>>>             PE-LEFT supports flex-algo 128 and wants to install a forwarding
>>>>>>>>>             entry for 2.2.2.2 for flex-algo 128.
>>>>>>>>>
>>>>>>>>>             It looks in the LSPs originated by PE-RIGHT. It does not see any
>>>>>>>>>             assigned SID for 2.2.2.2/32 flex-algo 128.
>>>>>>>>>
>>>>>>>>>             It cannot create a forwarding entry. And neither can any other node
>>>>>>>>>             in the left hand side of the network.
>>>>>>>>>
>>>>>>>>>             When PE-RIGHT stops advertising the explicit prefix SID for
>>>>>>>>>             2.2.2.2/32 Algo 128, all legacy nodes are unable to create
>>>>>>>>>             forwarding entries for the prefix/algo tuple.
>>>>>>>>>
>>>>>>>>>             This isn’t backwards compatible. 
>>>>>>>>>
>>>>>>>>>             In general, you cannot advertise information legacy nodes require in
>>>>>>>>>             a new container that legacy nodes do not understand and claim that
>>>>>>>>>             you are backwards compatible.
>>>>>>>>>
>>>>>>>>>             [lc2] Please refers to the examples for clarification.
>>>>>>>>>
>>>>>>>>>              1.
>>>>>>>>>
>>>>>>>>>                 For existing Flex-Algo deployment, it would be compatible. There
>>>>>>>>>                 is no change in the container format on how prefix-sid
>>>>>>>>>                 2.2.2.2/32 in FA128 is advertised.
>>>>>>>>>
>>>>>>>>>              1.
>>>>>>>>>
>>>>>>>>>                 For new VFA, it would not be compatible. But….it does not mean
>>>>>>>>>                 that we could not have VFA running in the same network.
>>>>>>>>>
>>>>>>>>>             There could be procedures to enhance such. With your example, one
>>>>>>>>>             workaround could be:
>>>>>>>>>
>>>>>>>>>             For VFA 600, PE-RIGHT detects that PE-LEFT does not participate in
>>>>>>>>>             VFA600 (due to no offset advertisement seen),
>>>>>>>>>
>>>>>>>>>               *
>>>>>>>>>
>>>>>>>>>                 Either, it spawns new CSPF for VFA600 instead of sharing FA129’s
>>>>>>>>>                 result. Bypass PE-LEFT as a result.
>>>>>>>>>
>>>>>>>>>               *
>>>>>>>>>
>>>>>>>>>                 Or, it uses legacy node FA129 prefix-sid and adj-sid as
>>>>>>>>>                 replacement (note: this method needs more comment)
>>>>>>>>>
>>>>>>>>>             In either ways, VFA600 could work without issue even with legacy
>>>>>>>>>             nodes co-existence.
>>>>>>>>>
>>>>>>>>>             After PE-LEFT upgraded, VFA600 would be using FA129 CSPF result
>>>>>>>>>             instead, and save CPU resources in each node.
>>>>>>>>>
>>>>>>>>>             Another question: do we need FAD for VFA600? Currently, no. Not
>>>>>>>>>             mandatory.
>>>>>>>>>
>>>>>>>>>             But it could be considered if “good to have” parameters are passed
>>>>>>>>>             along with FAD.
>>>>>>>>>
>>>>>>>>>             [/lc2]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > I do not see a major problem. Please give me an example to
>>>>>>>>>             illustrate your
>>>>>>>>>
>>>>>>>>>              > concern if possible.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > Of course, we need to do double check on the claim
>>>>>>>>> and possibly lab
>>>>>>>>>
>>>>>>>>>              > verification to see if the backward compatibility could be
>>>>>>>>>             achieved. It could
>>>>>>>>>
>>>>>>>>>              > be vendor specific.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [/lc]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > What does deserve discussion is a "hitless migration strategy".
>>>>>>>>>             When full
>>>>>>>>>
>>>>>>>>>              > support is available, if you were to switch to the new scheme,
>>>>>>>>>             you would
>>>>>>>>>
>>>>>>>>>              > want to do so without changing any existing SIDs as
>>>>>>>>> this would avoid
>>>>>>>>>
>>>>>>>>>              > forwarding disruption. Which means operators would have to modify
>>>>>>>>>             their
>>>>>>>>>
>>>>>>>>>              > SID assignment scheme in advance of deploying the new scheme.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] For VFA, there would be issue for legacy nodes. I agree. In
>>>>>>>>>             this case,
>>>>>>>>>
>>>>>>>>>              > solution could be
>>>>>>>>>
>>>>>>>>>              >         - either have a fallback plan for newer nodes if
>>>>>>>>>             detection of legacy nodes
>>>>>>>>>
>>>>>>>>>              > exist in the network. E.g. spawn new CSPF
>>>>>>>>>
>>>>>>>>>              >         - or, totally not to deploy VFA unless all nodes are
>>>>>>>>>             upgraded.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > Section 5 is not updated with VFA inclusion.
>>>>>>>>>
>>>>>>>>>             [LES:] My comments have nothing to do with VFA.
>>>>>>>>>
>>>>>>>>>             Please reconsider them after you have understood the backwards
>>>>>>>>>             compatibility issues.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [/lc]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > 3)Virtual Flex Algorithm
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > You have introduced a new concept with very little explanation of
>>>>>>>>>             what it is
>>>>>>>>>
>>>>>>>>>              > nor how it can be supported.
>>>>>>>>>
>>>>>>>>>              > For example, how would we determine which nodes support a given VFA?
>>>>>>>>>
>>>>>>>>>              > Since the algorithm value must be greater than 255, it cannot be
>>>>>>>>>             advertised in
>>>>>>>>>
>>>>>>>>>              > the existing SR Algorithm sub-TLV.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > If you are serious about this idea, please provide
>>>>>>>>> a more complete
>>>>>>>>>
>>>>>>>>>              > discussion.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] We could illustrate application examples in next
>>>>>>>>>             presentation. For
>>>>>>>>>
>>>>>>>>>              > ethernet, we have port, and then we have VLAN and stacked vlan.
>>>>>>>>>             History
>>>>>>>>>
>>>>>>>>>              > has some hints on this.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>             [LES:] You are writing a normative specification. Hoping that all
>>>>>>>>>             readers/implementors have the same "intuition" isn’t sufficient.
>>>>>>>>>
>>>>>>>>>                 Les
>>>>>>>>>
>>>>>>>>>              > [/lc]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > 4)Section 4.3
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > "R" and "N" flags are now defined in prefix attributes sub-TLV
>>>>>>>>>             (RFC7794)
>>>>>>>>>
>>>>>>>>>              > They were originally defined in the SR sub-TLV because RFC 7794
>>>>>>>>>             did not exist
>>>>>>>>>
>>>>>>>>>              > at the time.
>>>>>>>>>
>>>>>>>>>              > The only reason they continue to exist in RFC 8667
>>>>>>>>> is for backwards
>>>>>>>>>
>>>>>>>>>              > compatibility with early implementation of SR-MPLS based on early
>>>>>>>>>             drafts of
>>>>>>>>>
>>>>>>>>>              > what became RFC 8667.
>>>>>>>>>
>>>>>>>>>              > Please do not introduce them in new sub-TLVs - there is no need.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] noted with thanks [/lc]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > 5)ADJ-SIDs are NOT allocated from the SRGB as they are local in
>>>>>>>>>             scope.
>>>>>>>>>
>>>>>>>>>              > They MAY be allocated from the SRLB - or outside either GB range.
>>>>>>>>>
>>>>>>>>>              > Please correct the document in this regard.
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > [lc] noted [/lc]
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              >    Les
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > > -----Original Message-----
>>>>>>>>>
>>>>>>>>>              > > From: Lsr <lsr-bounces@ietf.org <mailto:lsr-bounces@ietf.org>>
>>>>>>>>>             On Behalf Of Acee Lindem
>>>>>>>>>
>>>>>>>>>              > > Sent: Friday, April 7, 2023 11:43 AM
>>>>>>>>>
>>>>>>>>>              > > To: Louis Chan <louisc@juniper.net
>>>>>>>>> <mailto:louisc@juniper.net>>
>>>>>>>>>
>>>>>>>>>              > > Cc: lsr <lsr@ietf.org <mailto:lsr@ietf.org>>
>>>>>>>>>
>>>>>>>>>              > > Subject: Re: [Lsr] IETF-116 LSR - IGP extensions
>>>>>>>>> for Advertising
>>>>>>>>>
>>>>>>>>>              > > Offset for Flex-Algorithm
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>>              > > Hi Louis,
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>>              > > In the interest of initiating discussion, I would like to
>>>>>>>>>             propose the
>>>>>>>>>
>>>>>>>>>              > > term "Flex Algorithm Traffic Class (FATC)" for
>>>>>>>>> the sub-division of
>>>>>>>>>
>>>>>>>>>              > > flex-algorithm traffic referred to in the draft as “Virtual
>>>>>>>>>             Flex Algorithm”.
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>>              > > Also, in your terminology, you refer referred to TLVs and
>>>>>>>>>             fields with
>>>>>>>>>
>>>>>>>>>              > > simply “Algorithm” when RFC 9350 uses “Flex Algorithm”.
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>>              > > Thanks,
>>>>>>>>>
>>>>>>>>>              > > Acee
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>>              > > _______________________________________________
>>>>>>>>>
>>>>>>>>>              > > Lsr mailing list
>>>>>>>>>
>>>>>>>>>              > > Lsr@ietf.org <mailto:Lsr@ietf.org>
>>>>>>>>>
>>>>>>>>>              > >
>>>>>>>>>
>>>>>>>>> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinf
>>>>>>>>> o/
>>>>>>>>> l
>>>>>>>>> s
>>>>>>>>> r_
>>>>>>>>> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinf
>>>>>>>>> o/
>>>>>>>>> l
>>>>>>>>> s
>>>>>>>>> r_>
>>>>>>>>>
>>>>>>>>>              > > _;!!NEt6yMaO-gk!B9ufrV6k-
>>>>>>>>>
>>>>>>>>>              > c7mtP9JUiXbrF3NCkZ15_UMLBjV_fnJovfz18M5VkkI2F
>>>>>>>>>
>>>>>>>>>              > > EoixpkxsfMnbqwbR0bpHgoS9E$
>>>>>>>>>
>>>>>>>>>              >
>>>>>>>>>
>>>>>>>>>              > Juniper Business Use Only
>>>>>>>>>
>>>>>>>>>             Juniper Business Use Only
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>             Juniper Business Use Only
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Lsr mailing list
>>>>>>>> Lsr@ietf.org
>>>>>>>> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo
>>>>>>>> /l
>>>>>>>> s
>>>>>>>> r
>>>>>>>> __;!!NEt6yMaO-gk!EX4RtSVg7I0jO4M9ZMrtcjGl2sqxcDIFHFZRnk4pocYHUxv9
>>>>>>>> _p
>>>>>>>> A
>>>>>>>> b
>>>>>>>> 8cfeVdGYX_3gOEDi1kljo7ufGA$
>>>>>>>> -----------------------------------------------------------------
>>>>>>>> --
>>>>>>>> -
>>>>>>>> -
>>>>>>>> ----------------------------------------------------------------
>>>>>>>> 本邮件及其附件含有新华三集团的保密信息,仅限于发送给上面地址中列出
>>>>>>>> 的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、
>>>>>>>> 或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本
>>>>>>>> 邮件!
>>>>>>>> This e-mail and its attachments contain confidential information
>>>>>>>> from New H3C, which is intended only for the person or entity
>>>>>>>> whose address is listed above. Any use of the information
>>>>>>>> contained herein in any way (including, but not limited to, total
>>>>>>>> or partial disclosure, reproduction, or dissemination) by persons
>>>>>>>> other than the intended
>>>>>>>> recipient(s) is prohibited. If you receive this e-mail in error,
>>>>>>>> please notify the sender by phone or email immediately and delete it!
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Juniper Business Use Only
>>>>>>
>>>>>
>>>>>
>>>>> Juniper Business Use Only
>>>>>
>>>>
>>>>
>>>> Juniper Business Use Only
>>>>
>>>
>>>
>>> _______________________________________________
>>> Lsr mailing list
>>> Lsr@ietf.org
>>> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/lsr_
>>> _;!!NEt6yMaO-gk!CpAiLsLUgtEYd391pIWnSINqSV8P0hoPZ2Cw_WvaUKFwhaQvBgbhFQ
>>> dYgVCu33ZaXGY7RiI9idFxyQ$
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Lsr mailing list
>>> Lsr@ietf.org
>>> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/lsr_
>>> _;!!NEt6yMaO-gk!CpAiLsLUgtEYd391pIWnSINqSV8P0hoPZ2Cw_WvaUKFwhaQvBgbhFQ
>>> dYgVCu33ZaXGY7RiI9idFxyQ$
>>>
>>
>>
>> Juniper Business Use Only
>>
> 
> 
> Juniper Business Use Only
>