Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call

Huaimo Chen <huaimo.chen@futurewei.com> Mon, 25 May 2020 02:25 UTC

Return-Path: <huaimo.chen@futurewei.com>
X-Original-To: lsr@ietfa.amsl.com
Delivered-To: lsr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 92B0A3A07CE for <lsr@ietfa.amsl.com>; Sun, 24 May 2020 19:25:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.078
X-Spam-Level:
X-Spam-Status: No, score=-2.078 tagged_above=-999 required=5 tests=[AC_DIV_BONANZA=0.001, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, T_REMOTE_IMAGE=0.01, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=futurewei.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SFAIQB3oTqjz for <lsr@ietfa.amsl.com>; Sun, 24 May 2020 19:25:45 -0700 (PDT)
Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-eopbgr760123.outbound.protection.outlook.com [40.107.76.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E9AF03A07C4 for <lsr@ietf.org>; Sun, 24 May 2020 19:25:44 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jeVhSe4B7y4vlWo7dP2styggsKvPsr/3t/2Ew8fVhg5/Z2C9q+N1yuNTSfMCxnD8nIPXrdWchcMU5AHc/Y8YJZs5nJSHh8UQfxOKbhO9f6VEsYFvGAfz4czHAu3cx4AUZ8M6QNPyImaWE3lDNDwHfAUJzDZBlZ6t4kkc1BPu2IewWAzVqR6TYmIT5LxYkfi3kqCJ6cs3ie7xA4OtOwNeEateja66tFjq69qDfsq+b1puzGGkkxh1uOMc8rFHa+0zzvJ6aMVm2b1uXv60m/+X6AlhH8Mfd89yCQMh4UiTss780FXP0gwRUU6sAlyONm1pejNlc7EB8/WaFjaXMKRL2w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4Ch9ulHfdowGm7OyWjAMwsHWZ2HEz9bwQSarknc5H2Y=; b=HjQSnpA2g4nb7VXF/GJuPLNLpmoZhMEQl8scrKZzkJAQrdszwXXaGztmDP7u5EIDoRg52jw3K0LvS/y3jEss/g7fUqcmaQpttbBOD74fxvvE+Ipup+H/ai/GyQQQ44GrQBVOfeY3MW71d/J9jZZiFAqID+0CBmE3RdA+3/Gd10ffRkjxFPiCzXPAHDv9DRW2LuI9dFJtnDqrYqt0dTVE4fhiXn/Gu2po/2Y0iTM1P0HbxtxlXvubI+GknDHBiYcJl4xPwnGDvEwJGBqnOLhsjmrRUN36KLZ2+dPYcr14qk6gZRLbemckMjxqO0fptZ2Rkv2VmztGzuxc1jlBJNH5mA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=futurewei.com; dmarc=pass action=none header.from=futurewei.com; dkim=pass header.d=futurewei.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Futurewei.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4Ch9ulHfdowGm7OyWjAMwsHWZ2HEz9bwQSarknc5H2Y=; b=snqoWQIuJlOghgWcha12gTTArMHGa9ReDtLci4Fbbh3HGm9oD8a61S9RbydqC5FMMG6QFZZFcNj0MJI4d8iXKBZsfeLxtI6cE/gouWuJ/7YV9q62cS37J/A6ug9oZ7ThAxqYaZ6Rm/337tHjyExr2h5XBIpNfZTc8bao2OSFHUo=
Received: from MN2PR13MB3117.namprd13.prod.outlook.com (2603:10b6:208:13a::20) by MN2PR13MB4039.namprd13.prod.outlook.com (2603:10b6:208:269::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.8; Mon, 25 May 2020 02:25:41 +0000
Received: from MN2PR13MB3117.namprd13.prod.outlook.com ([fe80::d5b6:8550:9c40:eec2]) by MN2PR13MB3117.namprd13.prod.outlook.com ([fe80::d5b6:8550:9c40:eec2%7]) with mapi id 15.20.3045.009; Mon, 25 May 2020 02:25:41 +0000
From: Huaimo Chen <huaimo.chen@futurewei.com>
To: Gyan Mishra <hayabusagsm@gmail.com>
CC: "Acee Lindem (acee)" <acee@cisco.com>, "lsr@ietf.org" <lsr@ietf.org>
Thread-Topic: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call
Thread-Index: AQHWKvCWbO+rNwui80Sty0q3wo/aI6iuQDRQgAC6fICAAVrmiIAAx0uAgADIYX+AAL9sAIAAAVeagAA06ID//+MZAIAARoiA///qD4CAAHjEgIAAvJSBgAC7/oCAAAhqAIADKXWP
Date: Mon, 25 May 2020 02:25:41 +0000
Message-ID: <MN2PR13MB3117FEC5B0C03845F5B49A96F2B30@MN2PR13MB3117.namprd13.prod.outlook.com>
References: <63330FD6-EDE8-4938-AA85-948279C57E34@cisco.com> <DM6PR06MB509842BA0E42A4938ACB2B94EEB80@DM6PR06MB5098.namprd06.prod.outlook.com> <CABNhwV3jQ=C6ynJBsKhJW_b=hkq7CKPFG2ci0vXE_0jZH9KtXQ@mail.gmail.com> <MN2PR13MB3117C43CA052E4E29D7428CBF2B60@MN2PR13MB3117.namprd13.prod.outlook.com> <CABNhwV2Y3kemTdHYBamZwAB+m5DefxqTC0shPan2cGHd9TARTg@mail.gmail.com> <MN2PR13MB3117E84C778B11D71FAF8135F2B70@MN2PR13MB3117.namprd13.prod.outlook.com> <CABNhwV1W3EDJpqUjLGj3-E-8L1R=75oAgtHX==9qkfrvrZy2ZQ@mail.gmail.com> <MN2PR13MB3117A5E68B341A203266C64AF2B70@MN2PR13MB3117.namprd13.prod.outlook.com> <CABNhwV3e_qm=RmTQyQ+yCWN+pt_d1Aw8-W9gkRwH+_kAyphu8Q@mail.gmail.com> <424B17B1-D66D-4708-A819-855E972841AF@cisco.com> <CABNhwV2cDoNotaePw30eZa5wE61NC_vh8cFaZHh_nPqs1ibFsg@mail.gmail.com> <E093695B-78DE-4332-B4E7-87343077624E@cisco.com> <CABNhwV1cfOE+trZGM-s44xU0su5RUgL2c7ubRCdc3EFCWsdujw@mail.gmail.com> <MN2PR13MB31179DB87C96DA218F9EB1D0F2B40@MN2PR13MB3117.namprd13.prod.outlook.com> <CABNhwV0CC6wBrPkiA+vKjpX--JPBPfRzJ=0GDP3BqH4NPaecKQ@mail.gmail.com>, <CABNhwV2ZrPpH-dDQ5jQ_FwsG6ezCqZ7Kk925avFwaQkA3Wt73w@mail.gmail.com>
In-Reply-To: <CABNhwV2ZrPpH-dDQ5jQ_FwsG6ezCqZ7Kk925avFwaQkA3Wt73w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=futurewei.com;
x-originating-ip: [2601:199:4300:8e5a:74ce:8eb8:ed16:ceb1]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 537dd6ba-cf56-42b3-aa07-08d80052ebc7
x-ms-traffictypediagnostic: MN2PR13MB4039:
x-microsoft-antispam-prvs: <MN2PR13MB403988223FE1FE6206424FF2F2B30@MN2PR13MB4039.namprd13.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-forefront-prvs: 0414DF926F
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: NhkMTaapI9IO8xa+IsS4WIfmSa76361j8WyumEre4BmyuurKC8eB+/vbmrlr5qxgUXmUaeW0//wPQ5jlDHzgxCvf7kzxcpXc4SZOCqjVro2cCx8oJmoXi2V0dvHLWVcvW9LYWITwKUfiuujsrs/dPhGYJV/4cmPylW9qM+TFsWxJHyPLIZ2ya2Etkqv/0pn0JTzhTZTE2TbhPNftM6dxsqtNfeoB3DAMkXfJBSWwvftvT4z04WjIGgAB4e+nz7J8gawfy22AGjy3TIcDINmWGMucxSIeBw74YZyNZvLeK6sN/tv4nKBWseLtA2en+87+Vc1RGm/8CnuKb2tDrA3fl1rOAdNHP4+SWZlIONFUFJjcX7MDOf2U3rx63vOOM1IEc7FoEZ6q+qdpRVCfSjwJGg==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR13MB3117.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39830400003)(376002)(136003)(396003)(366004)(30864003)(2906002)(53546011)(4326008)(86362001)(6916009)(55016002)(44832011)(7696005)(71200400001)(6506007)(478600001)(9686003)(54906003)(316002)(19627405001)(52536014)(186003)(8676002)(166002)(66556008)(66946007)(33656002)(64756008)(76116006)(8936002)(966005)(66446008)(5660300002)(66476007)(579004)(559001); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: sQaHyLhTLI71GPBSPNpKcCLIDcgtEvkczG/9n4xhjaB3ygCzDVorbYr5+WTvRqvPKywbueNj4ALHqpmOAt60cycDddrw/QUN/dlt1WXXL57E9xI8AIZ8B5ReZXNnb9mNm7+PDIVpx1Y0+yDctLlsJv07WkDbtkHP9djj8Kw1oVyp4znScff6ZVkBeZSlKhdewjMTPSnEQCJO71z5qZcekBnL2gbDDqRzdbxWJFYjMegdFRrkf1fUjx6D9X3Xm+/8IeenKCXRtGhKOTMf3NXA53CtTqa6ouV6rVjyQal25+/DTNzXtvvAHLZu0E/wtXyXBSwNzeE2xi5xOQxMMQboWZL0jcUl1lLqDr/9/U0suWVW3igdYHF21IgS5lRbSyWHGgEXzkiBd6sbdPkZSrXzDktQpGGjJCNqiCjRVxQye9gHaYvSg5TweaHZMlYFMxcpTsCK2wEYODRlVcgV/cTaYoAgvtlgADKHk9PMeMLbSkEckJkG/5iZQKvsKDp7XPQnuKnWLROiU4tNItERzSTgrdIRc/ut7tw1wyAuluOUdqqGTU2G662qikT72W0FEeL0
x-ms-exchange-transport-forked: True
Content-Type: multipart/alternative; boundary="_000_MN2PR13MB3117FEC5B0C03845F5B49A96F2B30MN2PR13MB3117namp_"
MIME-Version: 1.0
X-OriginatorOrg: Futurewei.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 537dd6ba-cf56-42b3-aa07-08d80052ebc7
X-MS-Exchange-CrossTenant-originalarrivaltime: 25 May 2020 02:25:41.4795 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 0fee8ff2-a3b2-4018-9c75-3a1d5591fedc
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 2ryVggn6x642iWMqTU7VB3SQ93w4dOCYrQzHAxuu/JdVg+O5jhbJYTSjPRGqdg1WGLfZ378drDLsHIbvIgzXGw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR13MB4039
Archived-At: <https://mailarchive.ietf.org/arch/msg/lsr/HiTFiINPi-BWM78wGSWezPAjev0>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call
X-BeenThere: lsr@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Link State Routing Working Group <lsr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/lsr>, <mailto:lsr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/lsr/>
List-Post: <mailto:lsr@ietf.org>
List-Help: <mailto:lsr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/lsr>, <mailto:lsr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 25 May 2020 02:25:52 -0000

Hi Gyan,

    Thanks much for your comments and suggestions. My answers/explanations are inline below.

Best Regards,
Huaimo
________________________________
From: Gyan Mishra <hayabusagsm@gmail.com>
Sent: Friday, May 22, 2020 9:11 PM
To: Huaimo Chen <huaimo.chen@futurewei.com>
Cc: Acee Lindem (acee) <acee@cisco.com>; lsr@ietf.org <lsr@ietf.org>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call

Hi Huaimo

Thinking about physical topology a little further as with the FT algorithm it makes sense to decouple the FT from the physical however the overall coverage is based on those basic parameters of degree number of links and nodes as well as width and depth of the topology.
[HC]: We may add/think more about physical topology and reduce/simplify the description about parameters in the draft.

Thinking about it further the clos leaf spine DC style topology is much worse in terms of flooding due to physical topology that leafs don’t physical connection and are interconnected via the spine creating massive ECMP and overload of the spine super spine.  So the physical DC leaf spine topology really exacerbates the flooding due to the non full mesh nature of the topology.
[HC]: You are right. The clos leaf spine topology is much worse. The algorithm in the draft can be used to compute FT for the clos leaf spine topology and reduce the number of links used for flooding significantly.

With a service provider core which is typically a mesh style multi tier multiple concentric circles mix of full and partial mesh and hub spoke type physical topologies.

So with flooding with full mesh every node connects with every node so you don’t have an overload of flooding being funneled through a super spine as you do with a leaf spine DC architecture.  So in a full mesh which is the best case scenario for flooding perspective from my point of view which each node as root connects to all nodes and then you can iteratively grow the flooding scope and it’s pretty controlled and you can limit the redundant flooding.  When the physical topology changes to a much more partially meshed the ECMP paths grow astronomically as does the flooding and longer convergence time.

So I think in the slide deck and draft look at scenario where their is tremendous ECMP from partial meshing and how the flood reduction algorithm improves the flooding and convergence.
[HC]: They should have more scenarios including clos spine leafs. The reason for not having them in the draft is that "drawing" topology of clos spine leaf to illustrate the algorithm in details is very hard.

Kind regards

Gyan

On Fri, May 22, 2020 at 8:41 PM Gyan Mishra <hayabusagsm@gmail.com<mailto:hayabusagsm@gmail.com>> wrote:
Hi Huaimo

Thank you for the slide deck.  That really helps understand the algorithm.

I will let you know if I have any questions.

The goal of the algorithm is to speed up convergence by limiting the convergence scope and expanding out 1 link at a time until all nodes and links are part of the flood scope.  From the examples in the slide deck I did not see mentioned what is done with the dynamic flooding algorithm where when you have a meshed network in the slide 5 examples, how do you limit the flooding on redundant links duplicate flooding with the many meshed paths as you iteratively grow the FT nodes scope Cq().  I believe with the dynamic flooding it does a degree of 2 so 2 links between nodes.

I think with clos spine leaf the mesh is much more intensive and problematic with ECMP then a circular topology nodal mesh that results in duplicate redundant flooding that slows down convergence.  With spine leaf it’s like an X horizontal width axis and then depth is spine to leaf links.  With spine leaf as you grow sideways and the spine expand the redundant ECMP grows and redundant flooding grows exponentially and is much worse then circular nodal mesh.

Thank you

Gyan

On Fri, May 22, 2020 at 10:05 AM Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>> wrote:
Hi Gyan,

    Thanks much for your questions. My answers/explanations are inline below.

Best Regards,
Huaimo
________________________________
From: Gyan Mishra <hayabusagsm@gmail.com<mailto:hayabusagsm@gmail.com>>
Sent: Thursday, May 21, 2020 10:13 PM
To: Acee Lindem (acee) <acee@cisco.com<mailto:acee@cisco.com>>
Cc: Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>>; lsr@ietf.org<mailto:lsr@ietf.org> <lsr@ietf.org<mailto:lsr@ietf.org>>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call


I think for both drafts it would be good to have a stick diagram at a minimum.
[HC]: A diagram for illustrating the steps of the algorithm is in Appendix.
I majored in EE with math  minor and both drafts are hard to follow without a diagram with labels showing the exact topology examples.

Four main topologies - full mesh, partial mesh and leaf spine ( 2 tier) and CLOS ( 3 tier) - real world scenario’s.

>From the interim meeting was their a slide deck visual topology shared?
[HC]: The set of slides presented in IETF 105 is attached.

That would help.

https://datatracker.ietf.org/doc/html/draft-cc-lsr-flooding-reduction-08<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-cc-lsr-flooding-reduction-08&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931306983861&sdata=IFYiJFuxf4UAMRmjxDf%2BTICOWslqJxCWjWwYYJM%2F6uQ%3D&reserved=0>

4.1

Step 1 - Build Cq candidate queue


For each node B on FT whose D is one (from minimum to maximum
       node ID), find a link L attached to B such that L's remote node R
       has minimum D and ID, add link L between B and R into FT and
       increase B's D and R's D by one.  Return FT.

My interpretation is the algorithm an iterative loop and starts with any node in the candidate list so could start from edge and work left to right or vice versa.  For loop to build FT from Cq = with each node B with directly attached node R via link L.  Increment node B from Cq until the FT is build for all nodes.  Instead of starting with a wide diameter FT matching physical topology we start micro topology with D=1 and go node by node.
[HC]: This (For each node B on FT .... Return FT.) is an iterative loop after every node is added to FT. This loop connects each node whose D = 1 (i.e., there is only one link attached to it on FT) to a node through a link on FT (i.e., add the link to FT) to make every node on FT to have its D > 1. Page 5 of the attached slides shows some details for this loop.

4.2


1.  Finding and removing the first element with node A in Cq that is
       not on FT and one PH's D in PHs < MaxD and PH's D < PH's ConMaxD.

       If A is root R0, then add the element into FT

       otherwise  (i.e., A != R0 with one PH's D in PHs < MaxD and PH's
          D < PH's ConMaxD.  Assume that PH is the first one in PHs
          whose D < MaxD and PH's D < PH's ConMaxD), PH's D++, and add A
          with D = 1 and PHs = {PH} into FT.

       Note:  if there is no element with a node in Cq satisfying the
          conditions, then algorithm may be restarted from R0, ++MaxD,
          Cq = {(R0,D=0,PHs = { })}, FT = { };

This is similar to 4.1 with micro topology with 2 directly connected nodes
[HC]: It is similar to 1. in section 4.1.

https://www.ietf.org/id/draft-chen-lsr-dynamic-flooding-algorithm-00.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fid%2Fdraft-chen-lsr-dynamic-flooding-algorithm-00.html&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931306988838&sdata=DlhahSNtm%2Ft406GcNtbGGq7uhFKfAqHO72Cm0vQdIo8%3D&reserved=0>

My interpretation of the algorithm from the verbiage.

So this draft starts in the middle of the topology and is geared towards leaf spine 2 or 3 tier clos topology.

For this one we build the bi graph so that could be leaf connected to two spines or spine connected to two leafs and iteratively build out the FT topology 3 node arch  spine leaf spine or leaf spine leaf.

Kind regards

Gyan

On Thu, May 21, 2020 at 7:01 PM Acee Lindem (acee) <acee@cisco.com<mailto:acee@cisco.com>> wrote:

Hi Gyan,



From: Gyan Mishra <hayabusagsm@gmail.com<mailto:hayabusagsm@gmail.com>>
Date: Thursday, May 21, 2020 at 4:20 PM
To: Acee Lindem <acee@cisco.com<mailto:acee@cisco.com>>
Cc: Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>>, "lsr@ietf.org<mailto:lsr@ietf.org>" <lsr@ietf.org<mailto:lsr@ietf.org>>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call





Yes and I have started reviewing the LSR mail archives as I am late in the discussion for this.



If you have link to any particular dates in the archives would be helpful.



No – it was more an “evolution” than an “epiphany”.



That sums up all the questions I have so if all answered in the archives then I am all set.



I support the draft moving forward as flood reduction is a much needed feature.



I think overall both drafts will really help large data centers with any overlay underlay vxlan spine leaf meshed CLOS highly meshed topology that scales out horizontally with a large spine footprint - this is a much much needed feature.  This also helps eliminate complexity of the workaround of using BGP in the underlay which to me adds tremendous complexity.



Here is the other recently presented dynamic flooding draft.



https://datatracker.ietf.org/doc/draft-chen-lsr-dynamic-flooding-algorithm/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-chen-lsr-dynamic-flooding-algorithm%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931306993818&sdata=f93gTdRXOpDko4jB0G%2B1Ixebs%2FrDHfmJ7WCWExPD5EI%3D&reserved=0>



You can see they both make use of the dynamic flooding infra in https://datatracker.ietf.org/doc/draft-ietf-lsr-dynamic-flooding/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-lsr-dynamic-flooding%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931306998798&sdata=y7jC5eotw%2BebdD4dDfvO7pSlh8JUYurUyAC4kFJDpfY%3D&reserved=0>



Thanks,

Acee









Kind regards



Gyan



On Thu, May 21, 2020 at 4:07 PM Acee Lindem (acee) <acee@cisco.com<mailto:acee@cisco.com>> wrote:

Speaking as WG Co-chair:



Hi Gyan,



I guess you’ve joined this discussion late. It might be a good idea to review the LSR mailing list archives.



From: Gyan Mishra <hayabusagsm@gmail.com<mailto:hayabusagsm@gmail.com>>
Date: Thursday, May 21, 2020 at 1:51 PM
To: Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>>
Cc: Acee Lindem <acee@cisco.com<mailto:acee@cisco.com>>, "lsr@ietf.org<mailto:lsr@ietf.org>" <lsr@ietf.org<mailto:lsr@ietf.org>>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call







On Thu, May 21, 2020 at 11:01 AM Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>> wrote:

Hi Gyan,



    Thanks much for your questions.



 Gyan> How does this draft compare to the WG LC draft for flood reduction.  Would they be two eventual standard options in the operators toolbox  or competing features for optimized flood reduction. Would having two flood reduction features standardized versus one default IGP flood reduction feature be confusing for operators.  Just as it was confusing to me.

https://datatracker.ietf.org/doc/draft-ietf-lsr-dynamic-flooding/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-lsr-dynamic-flooding%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931306998798&sdata=y7jC5eotw%2BebdD4dDfvO7pSlh8JUYurUyAC4kFJDpfY%3D&reserved=0>



    [HC]: This draft plus the draft for flooding reduction can provide two modes of flooding reductions (i.e., centralized mode and distributed mode). The latter describes the two modes, but does not include any flooding topology computation algorithm for the distributed mode. The former proposes a flooding topology computation algorithm to be used in the distributed mode.



   Gyan> It maybe a good idea for both documents to reference each other as to how they play together or  in some respects if any provide a homogeneous complete holistic solution for flooding problem being solved.



It is a good idea to have this new draft reference the dynamic-flooding but not vice-versa. The existing dynamic flooding draft provides a framework for any dynamic flooding algorithm that provides a separate flooding topology. This algorithm is just the first to be formally proposed in a draft. Note that another algorithm was presented during the LSR virtual interim which replaced IETF 107.



In my opinion it may not be a bad idea even to combine both drafts so the solution is complete and holistic.  This will also make the overall specification flow nicely.



No – we’re not going to do this.



Thanks,

Acee



I know that efforts were made by LSR to have a common IGP solution, however there are many inherent differences between ISIS and OSPF that from IGP Link state protocol perspective you can treat like apples to apples but really it’s apples and oranges.  Maybe it might we wise to have separate draft for both and have references linking together as the same algorithm concept and mathematically however the actual code implementation would vary as the LSDB link state data structures are completely different.



Later = Dynamic flooding = 2 modes Centralized leader based and distributed



https://datatracker.ietf.org/doc/draft-ietf-lsr-dynamic-flooding/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-lsr-dynamic-flooding%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307003777&sdata=b5DAj1yMUCRnkNtCjslcw8%2BhFQvZDFS%2FQZ1j9eRX5ts%3D&reserved=0>



This later draft per section excerpt provides both centralized and distributed algorithm see below



6.4<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-lsr-dynamic-flooding-05%23section-6.4&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307008751&sdata=jKC3VcXH%2Bo64b4sgsoce8QmETmJm4vLNdCucfrLyCAg%3D&reserved=0>.  Area Leader Responsibilities



   If the Area Leader operates in centralized mode, it MUST advertise

   algorithm 0 in its Area Leader Sub-TLV.  In order for Dynamic

   Flooding to be enabled it also MUST compute and advertise a flooding

   topology for the area.  The Area Leader may update the flooding

   topology at any time, however, it should not destabilize the network

   with undue or overly frequent topology changes.  If the Area Leader

   operates in centralized mode and needs to advertise a new flooding

   topology, it floods the new flooding topology on both the new and old

   flooding topologies.



   If the Area Leader operates in distributed mode, it MUST advertise a

   non-zero algorithm in its Area Leader Sub-TLV.



   When the Area Leader advertises algorithm 0 in its Area Leader Sub-

   TLV and does not advertise a flooding topology, Dynamic Flooding is

   disabled for the area.  Note this applies whether the Area Leader

   intends to operate in centralized mode or in distributed mode.



   Note that once Dynamic Flooding is enabled, disabling it risks

   destabilizing the network.



6.5<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-lsr-dynamic-flooding-05%23section-6.5&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307013733&sdata=1XXbBG%2Fo%2Ft6zGyxqPQdbMZQLET3%2FaxrKj516zl1Ayn8%3D&reserved=0>.  Distributed Flooding Topology Calculation



   If the Area Leader advertises a non-zero algorithm in its Area Leader

   Sub-TLV, all nodes in the area that support Dynamic Flooding and the

   value of algorithm advertised by the Area Leader MUST compute the

   flooding topology based on the Area Leader's advertised algorithm.



   Nodes that do not support the value of algorithm advertised by the

   Area Leader MUST continue to use standard flooding mechanism as

   defined by the protocol.



   Nodes that do not support the value of algorithm advertised by the

   Area Leader MUST be considered as Dynamic Flooding incapable nodes by

   the Area Leader.



   If the value of the algorithm advertised by the Area Leader is from

   the range 128-254 (private distributed algorithms), it is the responsibility of the

network operator to guarantee that all nodes in the area have a common understanding of what the given algorithm value represents.



Former = WG LC

https://datatracker.ietf.org/doc/html/draft-cc-lsr-flooding-reduction-08<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-cc-lsr-flooding-reduction-08&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307018706&sdata=zf%2FK80oG8tz90g15eVrkGeyGc5fhBym5PJezCFfSi9A%3D&reserved=0>



Provides algorithm for distributed mode for this draft as well as the algorithm to be used for distributed mode later dynamic flooding draft



Separate question-



In light of the flooding algorithm and seeing that at the bottom of section 6.5 mentions flooding algorithm are the IANA codepoints reserved for flooding unique and non overlapping with the flex algo codepoints I believe 0-127.  I would think the flooding algorithm range of values is completely separate since a different function then flex algo values.



https://datatracker.ietf.org/doc/html/draft-ietf-lsr-flex-algo-07<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-lsr-flex-algo-07&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307023687&sdata=MkDKud9pIJUwwoN%2F7w71fnOXfwu38sk4qwkGmDspUKI%3D&reserved=0>





Best Regards,

Huaimo

________________________________

From: Gyan Mishra <hayabusagsm@gmail.com<mailto:hayabusagsm@gmail.com>>
Sent: Thursday, May 21, 2020 10:36 AM

To: Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>>
Cc: Acee Lindem (acee) <acee=40cisco.com@dmarc.ietf.org<mailto:40cisco.com@dmarc.ietf.org>>; lsr@ietf.org<mailto:lsr@ietf.org> <lsr@ietf.org<mailto:lsr@ietf.org>>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call







On Wed, May 20, 2020 at 11:59 PM Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>> wrote:

Hi Gyan,



    Thanks much for your questions. My answers are inline below with [HC].



Best Regards,

Huaimo

________________________________

From: Gyan Mishra <hayabusagsm@gmail.com<mailto:hayabusagsm@gmail.com>>
Sent: Wednesday, May 20, 2020 11:14 AM
To: Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>>
Cc: Acee Lindem (acee) <acee=40cisco.com@dmarc.ietf.org<mailto:40cisco.com@dmarc.ietf.org>>; lsr@ietf.org<mailto:lsr@ietf.org> <lsr@ietf.org<mailto:lsr@ietf.org>>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call



Huaimo



This is a much needed feature that operators have been needing for densely meshed topologies that commonly exist in data centers to accommodate very high bandwidth E-W traffic.

[HC]: Thank you very much.



Below is link from Cisco which has introduced feature for dynamic flooding in clos high density ECMP data center topologies.



Please look at the feature description and it does seem to be exactly the same as this draft.  Please confirm.

[HC]: It seems different.



There maybe other vendors due to industry demand have to get the feature deployed before it reaches standards vendor consensus with the IETF.



https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-743015.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cisco.com%2Fc%2Fen%2Fus%2Fproducts%2Fcollateral%2Fswitches%2Fnexus-9000-series-switches%2Fwhite-paper-c11-743015.html&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307028662&sdata=NM0O3bq44tQ5GxZWm08rVjOh34mJjI1QsQ8DKFMgrAI%3D&reserved=0>



We are testing this feature and planning to deploy but wanted to ensure that this is the same as the draft on the standards track.

[HC]: The feature appears implemented draft "Dynamic Flooding on Dense Graphs", which does not include any flooding topology computation algorithm..



    Gyan> How does this draft compare to the WG LC draft for flood reduction.  Would they be two eventual standard options in the operators toolbox  or competing features for optimized flood reduction. Would having two flood reduction features standardized versus one default IGP flood reduction feature be confusing for operators.  Just as it was confusing to me.





https://datatracker.ietf.org/doc/draft-ietf-lsr-dynamic-flooding/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-lsr-dynamic-flooding%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307033643&sdata=tk5GwzA0o%2Bheui0Hl3nIpNwmNXM0QW%2FEXRbDQav8y54%3D&reserved=0>



Kind regards



Gyan





On Tue, May 19, 2020 at 11:52 PM Huaimo Chen <huaimo.chen@futurewei.com<mailto:huaimo.chen@futurewei.com>> wrote:

Hi Gyan,



    Thank you very much for your questions and support.



    This Flooding Topology Computation algorithm can be used in the Dynamic Flooding on Dense Graphs to compute the flooding topologies for the data center clos dense meshed topologies with many ECMP paths. It can be used by the area leader in the centralized mode to compute the flooding topology.



Best Regards,

Huaimo

________________________________

From: Lsr <lsr-bounces@ietf.org<mailto:lsr-bounces@ietf.org>> on behalf of Gyan Mishra <hayabusagsm@gmail.com<mailto:hayabusagsm@gmail.com>>
Sent: Tuesday, May 19, 2020 2:39 AM
To: Yanhe Fan <yfan@casa-systems.com<mailto:yfan@casa-systems.com>>
Cc: lsr@ietf.org<mailto:lsr@ietf.org> <lsr@ietf.org<mailto:lsr@ietf.org>>; Acee Lindem (acee) <acee=40cisco.com@dmarc.ietf.org<mailto:40cisco.com@dmarc.ietf.org>>
Subject: Re: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call





I support WG adoption and have a few questions related to the draft.



Does this flooding algorithm use the dynamic flooding algorithm used in data center clos dense meshed topologies with many ECMP paths where the flood is decoupled from the physical topology.  In the dynamic flooding algorithm mentioned in centralized mode the flooding is computed by the area leader and distributed to all nodes.  In distributed mode each mode the area leader determines the algorithm and then each node computes the flooding topology based on the algorithm.



This dynamic algorithm for optimized flood reduction would reduce the amount of redundant flooding in highly densely meshed ospf or Isis topologies.  So this optimization of flooding would improving overall link state routing protocol convergence.



Gyan



On Mon, May 18, 2020 at 3:37 PM Yanhe Fan <yfan@casa-systems.com<mailto:yfan@casa-systems.com>> wrote:

Support it as a co-author.



Thanks,

Yanhe



From: Lsr <lsr-bounces@ietf.org<mailto:lsr-bounces@ietf.org>> On Behalf Of Acee Lindem (acee)
Sent: Friday, May 15, 2020 3:40 PM
To: lsr@ietf.org<mailto:lsr@ietf.org>
Subject: [Lsr] Flooding Topology Computation Algorithm - draft-cc-lsr-flooding-reduction-08 Working Group Adoption Call



This begins a 3 week (due to holidays) WG adoption call for the “Flooding Topology Computation Algorithm” draft. Please issue your support or objection to this list by 11:59 PM, UTC on June 5th, 2020. Here is a URL for your convenience.



https://datatracker.ietf.org/doc/draft-cc-lsr-flooding-reduction/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-cc-lsr-flooding-reduction%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307038619&sdata=Rg7ZsHHsYJnJS7SpYUCLRsrDZ86Gi0iyKD2vMcjbboM%3D&reserved=0>



Thanks,
Acee

_______________________________________________
Lsr mailing list
Lsr@ietf.org<mailto:Lsr@ietf.org>
https://www.ietf.org/mailman/listinfo/lsr<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Flsr&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307043598&sdata=JDfLRjkZ4PI%2BwVcV0Mm0LXqq4Pzl3wz7BWWpv0zTtq0%3D&reserved=0>

--

Error! Filename not specified.<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307048576&sdata=gHSFT4GR6ZWS2MoTOPOVT%2BtTnF3wtTfRi5r1u6nzmcI%3D&reserved=0>

Gyan <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307053548&sdata=5RCW%2FwKABspmTndYt2yddKU2cGmYRpylfBQrM44LVZ0%3D&reserved=0> Mishra

<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307058534&sdata=6kQPY2lr7kCb1D8TK8FyEcLUhEw%2BXbGo5wq25P1%2BCN8%3D&reserved=0>Network Solutions Architect

<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307063514&sdata=KYMkolhI78NOupCfu2t4loUYoqPOz4DhWyVE5B6YefI%3D&reserved=0>

M 301 502-1347
13101 Columbia Pike<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307068486&sdata=efWfnCnZLIuib%2FkjHpXFoRTG%2FtueVoh5yTltBhLr4y4%3D&reserved=0>
Silver Spring, MD<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307073469&sdata=DU1ttb6xRduWgtpsBD6WNSWjma%2BtmtQdS%2FfOkth%2FwYc%3D&reserved=0>



--

Error! Filename not specified.<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307078448&sdata=0Oy2TQby8nTywn4XD8Zlsf8pbX34m0xOcwFcPdDl6u0%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307083424&sdata=DDxMBzfPAQ4XWIlzTfnZSp%2Fn5Q2Q1eAqZffx%2BHM4aig%3D&reserved=0>13101 Columbia Pike<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307083424&sdata=KSSlepfFHvtyqtOAbwIAdqkXy9RF%2FdOfo7%2FDz5RHp7k%3D&reserved=0>
Silver Spring, MD<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307088401&sdata=8N6QAF6sCbXNbIhQQDT%2B7LbdWxWK5PX5kcfqeSA8PW0%3D&reserved=0>

<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307093381&sdata=gM%2FzFPLLYGGKVIywDfe1VLMFtva0uGeYVwqjk4as53g%3D&reserved=0>



--

Error! Filename not specified.<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307098359&sdata=RFI30Fh8pkVGzCIo0t%2FuhBIYcu6cFYMQnfAKH9TCXXM%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307103340&sdata=JJvVXt2V%2B4qdM6D8hsCCltvguXzAGUPqzJAF2cSurVo%3D&reserved=0>
Silver Spring, MD<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307108314&sdata=zZ9exHtpFs%2B7d%2BTLGDwewg8ogHEksDYKQ%2FlHoXfS5R4%3D&reserved=0>

<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307113289&sdata=%2FvTes4sdQnRrA93lMZPAu%2BJeOuSZS%2BHN2NDIol5vkYU%3D&reserved=0>



<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307118270&sdata=CClF2UDqnAIwJ%2FfohMZudsssA%2FOMvzcgQfNPGQaPFqk%3D&reserved=0>

--

Error! Filename not specified.<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307123246&sdata=hspNepcNSicG01BlQPInlTrzCEDP7PrhsN5l0U9Qps4%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307128228&sdata=Zb1%2FHn2R9VxjacjDLBnZIb97XOKZvEMRn7QJS2LUN0s%3D&reserved=0>
Silver Spring, MD<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307133207&sdata=6ggk25pVaku9AIy6JyzPcXFYp9HxBqyxJUCsS2O7cOk%3D&reserved=0>

<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307133207&sdata=6ggk25pVaku9AIy6JyzPcXFYp9HxBqyxJUCsS2O7cOk%3D&reserved=0>



<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307138181&sdata=UYVHVeDZVxKw00dHyJFAM%2Ft4RrHG1Q5xsnfU2%2FQh89E%3D&reserved=0>

--

[Image removed by sender.]<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307143161&sdata=X%2FWFBYrgiaP3mP0VBmEbu9FmwKXpEBlXX%2F7N6IlbQMM%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307148138&sdata=9KuBH1b%2FrqYPVPz1x0oh6WOhHTgJwGYW4oTqijSBVns%3D&reserved=0>
Silver Spring, MD<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307153117&sdata=A%2F3aTskhdp8bYtLKDT9VQLCNNifuZm%2F3F2YRciVyH5E%3D&reserved=0>

<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307158095&sdata=iW%2F09yn2PMwEzUAIXaBwVId9QrSNXbS1z7s8Vl9Ts78%3D&reserved=0>



<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%250D%250A%2BColumbia%2BPike%2B%250D%250A%2BSilver%250D%250A%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307163074&sdata=o6fgwcMsNL9oCFlB%2BUx%2FmtsvLiz%2BcUS5njwbbChsjlg%3D&reserved=0>
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307168050&sdata=mixHFVEqF4qQ4i%2B6JbES5OP1g%2FSb%2FrxDgIv9Rww4jf4%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307173031&sdata=zguxTuADiqiiAZ22lAVWQD8f6JNBQwU9hgD1QktRULs%3D&reserved=0>
Silver Spring, MD<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307173031&sdata=zguxTuADiqiiAZ22lAVWQD8f6JNBQwU9hgD1QktRULs%3D&reserved=0>

<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fsearch%2F13101%2BColumbia%2BPike%2B%250D%250A%2BSilver%2BSpring%2C%2BMD%3Fentry%3Dgmail%26source%3Dg&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307178009&sdata=37qV2uXfgIRo455Ws84yQpkIjpQxU6VM6O8mSCRJjVc%3D&reserved=0>
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307182993&sdata=nHxpOihZ9RJbYVyIpCHoF22OEPNqU2vfUUlGkIY5wIA%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD

--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=02%7C01%7Chuaimo.chen%40futurewei.com%7C4c12ec234b2a452db84408d7feb65044%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637257931307187963&sdata=i9TTvZIGnP2LjLgUmMY013K5JYHVlHSoDmhMSfI8cbc%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD