Re: [Lsr] Thoughts on the area proxy and flood reflector drafts.

[Tony Li <[e-mail]>]

Hi Chris,

Thank you for your thoughtful comments.


> A simplified model of this design can be seen as a horseshoe of horseshoes. the Major horseshoe is L2 and the minor horseshoes are L1. Each L1 area has 2 L2 routers for redundancy (I'll consider more though), and all L2 routers are full-mesh connected to support that redundancy.
> 
> Telco
> 
> 
> But let's map this to a more DC centric view (I guess?) where each L1 area now has 10 L2 routers instead of 2 (i.e., 10%, but that could be change that later if need be).


The key point here is not a DC centric view, but one of scale. The above worked just fine back when we were terminating T1’s and E1’s.  Now, PE routers easily have terabit capacity. WIth the 100:1 ratio between the PE and the L1L2 routers (Area Edge routers), the bandwidth requirements drive us to petabit, multi-chassis routers. While these have been great fun to build, operators are not happy with this direction. Forklift upgrades are necessary with every silicon cycle. The complexity of the router has multiplied, the blast radius is off the charts, and the premiums charged for these devices are impressive. As a result, folks are trying to explore a ’scale-out’ approach. Rather than 2 huge area edge routers, they would much rather be able to scale out the area edge to be many routers.

If you pursue this design direction, the first thing you observe is that you can see is that you cannot afford to build a full-mesh of inter-area circuits across all of the area edge routers.

And if your network is a bit more geographically dispersed, you find that it becomes inefficient to have even a full mesh at the area level. This forces some areas to become transit.

Between these two forces, we are compelled to push transit traffic through the internals of an area.

> Natural Design
> 
> 
> 
> 


> Now for whatever reason some operators do not want to provision high-bandwidth transit links between their L2 routers in their L1 areas. This is critically important b/c otherwise you would simply use the above Natural Design. I'd like to better understand why that isn't just the answer here.


In this design, you’ve shown ten area edge routers.  Yes, you could provision a full mesh of links between them.  The issue remains one of scalability and uniformity. The number of ports per area edge router has to scale linearly with the size of area edge and the overall number of links used for this purpose is O(n^2).  Combine this with the fact that any non-uniformity in the traffic pattern and your full mesh ends up being congested and under-utilized simultaneously.  

All is not lost, however. Charles Clos to the rescue. By structuring each area as a Clos (or Benes) network (which my employer seems to insist on spelling ‘leaf-spine’), you avoid this. I assume I don’t need to go into details on this.  


> Area Proxy
> 
> First I'll look at area proxy. This seems a fairly simple idea, basically it's taking the now L1L2 areas and advertising them externally as a single LSP so the impact is very similar to if they were L1 only. This maps fairly closely to the Telco and Natural Design from above. Each L1 router in the Telco design would have 100 LSPs The L12 routers would have 100 L1 + 16 L2 LSP. In the Natural Design each L1 router has 100 L1 and each L12 router would have 100 L1 and 80 L2. With Area Proxy each router  has 100 L1 and 100 "Inner L2 LSPs" and 80 "Outer LSPs" + 8 "Outer L2 LSPs"


We’ve made some changes in the latest version of the draft.  In the current version, we require that each router in the area be L1L2.  However, only one LSP is advertised externally for each area.  Thus, each router will see 100 L1 LSPs, 100 L2 LSPs and 8 L2 Proxy LSPs.


> The key thing to note here is that if you double the number of areas you only add to the Outside LSP and Proxy count just as it would scale in the Natural Design, so going from 8 to 16 areas here adds 80 more "Outside LSPs" and 8 more L2 Proxy LSPs for a total of 276 L2 LSPs even though you've added 800 more routers to your network.


Doubling the number of areas would give you 16 L2 proxy LSPs, so you end up going from 208 LSPs to 216.  The key point here is that the database now scales linearly with the number of areas.

Demo time:

In a lab setup, we have an area of five routers.  We have three L2 routers.  The database on one of the pure L2 routers is just 5 entries (three L2, one proxy LSP, one pseudonode):

rtrmpls8>show isis data

IS-IS Instance: Amun VRF: default
  IS-IS Level 2 Link State Database
    LSPID                 Seq Num   Cksum  Life  IS Flags
    ip6.00-00             5         39442  1077  L2 <>
    ip7.00-00             6         40764  1077  L2 <>
    ip8.00-00             4         17533  1071  L2 <>
    ip8.06-00             1         42260  1071  L2 <>
    Proxy.00-00           1         13553  1087  L2 <>

On an inside node, the L1 data is 7 entries (including 2 pseudonodes), and the L2 database is 12 (including 3 pseudonodes):

rtrmpls4>show isis data

IS-IS Instance: Amun VRF: default
  IS-IS Level 1 Link State Database
    LSPID                 Seq Num   Cksum  Life  IS Flags
    ip1.00-00             6         57570  962   L2 <DefaultAtt>
    ip2.00-00             5         12641  963   L2 <DefaultAtt>
    ip2.06-00             1         26206  963   L2 <>
    ip3.00-00             5         30428  964   L2 <DefaultAtt>
    ip4.00-00             5         58405  962   L2 <DefaultAtt>
    ip4.07-00             1         35124  955   L2 <>
    ip5.00-00             5         55336  962   L2 <DefaultAtt>
  IS-IS Level 2 Link State Database
    LSPID                 Seq Num   Cksum  Life  IS Flags
    ip1.00-00             10        890    966   L2 <>
    ip2.00-00             7         4390   965   L2 <>
    ip2.06-00             1         61120  963   L2 <>
    ip3.00-00             7         43115  965   L2 <>
    ip4.00-00             7         64064  962   L2 <>
    ip4.07-00             1         6798   955   L2 <>
    ip5.00-00             9         34969  975   L2 <>
    ip6.00-00             5         39442  967   L2 <>
    ip7.00-00             6         40764  966   L2 <>
    ip8.00-00             4         17533  961   L2 <>
    ip8.06-00             1         42260  961   L2 <>
    Proxy.00-00           1         13553  975   L2 <>

Regards,
Tony