Re: [Lsr] WG Adoption Poll for “Using IS-IS Multi-Topology (MT) for Segment Routing based Virtual Transport Network” - draft-xie-lsr-isis-sr-vtn-mt-03

[Tony Przygienda <[e-mail]>]

I never saw more than 3-4 MT "slices" deployed, Gyan. Operational
complexity basically. Usual Ks or 10Ks prefixes in main instance and not
more than maybe 1K in others.

So practically speaking I would worry about operational procedures of such
a deployment first (OAM, connected topologies, provisioning [since both
sides of MT need matching configuration, as I said, on purpose from
experience operating such things then], validation). A controller makes a
lot sense here, not even necessarily as provisioning tool first but
visualization, OAM, operational analytics. Think basically that you're
getting yourself from "normal IGP" experience into something that will
resemble L3VPN more and you know the problems there pretty well ;-)

Also, planning of fast reroute strategy is good, what links/mix of
topologies you want to protect others and will you have enough capacity
once FRR kicks in. There are commercial tools available to "simulate" that
kind of stuff and AFAIR decent amount of theoretical work done on necessary
algorithms (and since the solution space starts to become real large real
fast something like an A/B test with ML could deliver good results [weighed
decision trees or something like this]. The difficult thing being of course
to establish a cost function as in "is one topology down due to overload &
e'one protected without loss better than e'one losing some packets better
or worse").

--- tony



On Mon, Mar 29, 2021 at 7:07 AM Gyan Mishra <[e-mail]> wrote:

> Hi  Tony
>
> One of the main concerns for operators as you have elaborated is the real
> world operators experiences aside from what you mentioned common use case
> of IPv6 as the first non zero MT, and vendor feedback on scalability and
> resource issues using many MT IPv4 and IPv6 separate instance and how well
> it scales or not and operational impacts with troubleshooting multiple RIB
> instances with common LSDB with discrete SPF instance per topology and
> control plane scale issues as the number of instances grows.
>
> Also complexity related to NOC troubleshooting and MTTR when you have
> let’s say 10 to 100 network slices for example, how complex do things get
> and is it manageable or not.
>
> As you have 12 bits of MT ID, 4096 instances is quite a lot.  Could you
> really ever scale to 4096 MT instances, probably not.
>
> What is a real world practical hard limit from your experiences of number
> of prefixes per MT RIB and maximum number of MT RIB instances.
>
> Also a cross comparison of MI non zero instance IID/ITID comparing 100
> ITID sub topologies to 100 MT RIB topologies and which scales better.
>
> Much appreciated your MT & MI experiences  and real world feedback.
>
>
> Kind Regards
>
> Gyan
>
> On Sat, Mar 27, 2021 at 6:59 AM Tony Przygienda <[e-mail]>
> wrote:
>
>> Having had a long history with RFC5120 (and with the concept before IETF
>> was even afoot really) and their deployment in its time I cannot resist
>> finally chiming in a bit ;-)
>>
>> Well, first, it seems there is agreement that the drafts state fairly
>> straightforward things but as informational probably nothing wrong with
>> having it captured and I'd agree with that.
>>
>> Second, as to realities of MT in such architecture some musings based on
>> experience;
>>
>> a) MT was best used when faced with the problem of "alien" topologies,
>> e.g. introduction of v6 into a network when a lot of boxes/links couldn't
>> do v6 yet (there was a good set of discussions in its time around MT
>> strictly per node or per link being best approach and for many reasons link
>> based architecture was chosen) or need for drastically different forwarding
>> decision deviating from simple SPF (RPF problems).
>> b) using underlay to "slice" is an expensive proposition of course. The
>> costliest resources in large networks is really ASIC space in the core and
>> if the MTs are properly separated, that will take ASIC state. in simplest
>> form e.g. topology being mirrored into per DSCP bits lookup or something
>> like this. Nothing wrong with that if one has the $ (or CNY ;-) to foot the
>> bill for that and slicing underlay gives of course the best quality
>> assurance compared to overlay approaches AFAIS.
>> c) MT (well, at least in ISIS ;-) has been built to scale ;-), originally
>> 8 bits were suggested and then I asked others "what's the biggest you can
>> imagine" to which answer was 1024 MT which needed of course a
>> multiplication by 4 to make sure stuff will still work in 25 years where we
>> are today ;-)
>> d) SPF is to an extent only limitation of MT. the tree itself can be
>> computed very efficiently today (and there are techniques like incremental
>> SPF which can drastically lessen the cost of a computation and one can go
>> to the extent of throwing the SPF computations to a beefy side-box even and
>> most extreme, ASIC [unbelievale or not, this has been done for trees up to
>> a certain space ;-)]), the limiting factor is more prefix attachement and
>> prefix download on massive changes (but there is no reason any LFA would
>> not work with MT BTW, even to the point where an MT can protect another MT
>> or MT use all links when protecting disregarding MT itself, I'm not aware
>> of implementations but the theoretical thinking/whiteboarding has been done
>> here long ago albeit one has to know a fair bit of math [metric sets
>> largely]) to not blow up on a mine ;-)
>> e) main limiting factor of MT deployment was IME operational
>> discontinuity of topologies that happens much more often than one thinks it
>> does. That's why MT negotiates on every link the topologies so continuity
>> is clearly provisioned and can be tracked. Tools to visualize, manage all
>> that are much harder to operate than one would think, I assume because
>> humans generally have poor intuition concerning topological spaces under
>> different metrics. Just look how confused we are when we try to strike
>> something in a glass of water ;-)
>> b) yes, TE can be advertised in topologies but here is bunch of things
>> that fall out of it
>> i) oversubscription is a fact of life in such an architecture and
>> discussions about this will start quickly
>> ii) except in case of massive under-subscription (but even then) the
>> correct scheduling of MTs will become an issue and possibly pre-emption in
>> HW when the timing becomes tight enough. DetNet is an interesting romping
>> ground of ideas for that right now
>> iii) Even with under-subscription interesting problems crop up like
>> parties that don't need much resources under normal circumstances but in
>> case of emergencies are willing to pay any amount to allocate resources
>> their need and pre-empt any other traffic.
>>
>> --- tony
>>
>>
>>
>>
>>
>>
>> On Sat, Mar 27, 2021 at 4:47 AM Gyan Mishra <[e-mail]>
>> wrote:
>>
>>>
>>> Hi Acee
>>>
>>> As an operator, I  support adoption of the draft and would like to
>>> provide answers to your questions.
>>>
>>> I would like to start by stating that as this is an informational
>>> document,  as nothing new is proposed other then the recommendation to use
>>> MT for VTN provisioning as a component of the 5G network slicing solutions,
>>> the benefit is that this concept can be used immediately as other NS
>>> features are still being developed.
>>>
>>> The solution for network slicing resource isolation is multi faceted
>>> involves Enhanced VPN+ provisioning, resource SIDs for provisioning
>>> underlay resources, SR-TE Per VPN or flow path steering to meet NS SLO
>>> requirements, as well as a method to isolate IGP resources for a VTN.
>>>
>>> MT is a component of the entire NS solution so there is really nothing
>>> to market as it’s not the only component used to provision the VTN.
>>>
>>> As their is a need for providing a viable means of provisioning IGP
>>> underlay resources VTN network slice and the ability to provide forwarding
>>> plane isolation via topological RIB without consuming tremendous control
>>> plane resources per instance as with MI.
>>>
>>> This draft does fill the gap of a means of forwarding plane isolation on
>>> shared infrastructure even though it does have scalability considerations.
>>>
>>> As other ideas of IGP forwarding plane isolation come about we are open
>>> to other solutions as well.
>>>
>>> As their are scalability concerns in section 5 that should be expanded,
>>> when MT should be used to support VTN and when should not. Agreed.
>>>
>>> I would deploy in a limited fashion taking into account the scalability
>>> concerns.
>>>
>>> Enhanced VPN  VPN+ scalability issue are described in detail in this
>>> draft below.  Lots of variables related to how many slices based on
>>> services which will eventually scale up but I think MT may suffice well in
>>> the beginning stages.
>>>
>>>
>>> https://tools.ietf.org/html/draft-dong-teas-enhanced-vpn-vtn-scalability-01
>>>
>>>
>>> There are many drafts and solutions in the works across many different
>>> WGs that are working on development of solution as to how network slicing
>>> and SLO can be realized  by operators for 5G services.
>>>
>>> Of these drafts below there are a number of Enhanced VPN Framework VPN+
>>>  related drafts that are critical to the provisioning of various components
>>> of network slicing.
>>>
>>> https://datatracker.ietf.org/doc/html/draft-ietf-teas-enhanced-vpn-07
>>>
>>>
>>> https://datatracker.ietf.org/doc/draft-dong-teas-enhanced-vpn-vtn-scalability/
>>>
>>> https://tools.ietf.org/html/draft-drake-bess-enhanced-vpn-06
>>>
>>> https://tools.ietf.org/html/draft-ietf-spring-resource-aware-segments-02
>>>
>>> Kind Regards
>>>
>>> Gyan
>>>
>>> On Thu, Mar 25, 2021 at 2:21 PM Acee Lindem (acee) <acee=
>>> [e-mail]> wrote:
>>>
>>>> Speaking as WG chair:
>>>>
>>>>
>>>>
>>>> There has been considerable support for this document. However, there
>>>> has also been objections to the document. The objections are either that
>>>> there is nothing to standardize given that all pieces exist and that the MT
>>>> isn’t a viable option for VTNs since it isn’t scalable.
>>>>
>>>>
>>>>
>>>> Since most of the draft’s support is from “friends and family”, I’d
>>>> like to know of the WG members who supported it, would you really want to
>>>> market it as a VTN solution? Those of you who operate networks, would you
>>>> actually consider deploying it?
>>>>
>>>>
>>>>
>>>> In any case, section 5 needs to be expanded on the scalability and
>>>> where using MTs to support VTNs would make sense and where it wouldn’t.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>> Acee
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From: *Lsr <[e-mail]> on behalf of "Acee Lindem (acee)"
>>>> <acee=[e-mail]>
>>>>
>>>>
>>>> *Date: *Tuesday, March 2, 2021 at 6:28 PM
>>>> *To: *"[e-mail]" <[e-mail]>
>>>> *Subject: *[Lsr] WG Adoption Poll for “Using IS-IS Multi-Topology (MT)
>>>> for Segment Routing based Virtual Transport Network” -
>>>> draft-xie-lsr-isis-sr-vtn-mt-03
>>>>
>>>>
>>>>
>>>> This information draft describes how MT could be used for VTN
>>>> segmentation. The authors have asked for WG adoption.
>>>>
>>>>
>>>>
>>>> This begins a three week LSR Working Group Adoption Poll for “Using
>>>> IS-IS Multi-Topology (MT) for Segment Routing based Virtual Transport
>>>> Network” - draft-xie-lsr-isis-sr-vtn-mt-03. I’m giving it three weeks due
>>>> to the IETF next week. Please register your support or objection on this
>>>> list prior to the end of the adoption poll on 3/24/2020.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Acee
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Lsr mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/lsr
>>>>
>>> --
>>>
>>> <http://www.verizon.com/>
>>>
>>> *Gyan Mishra*
>>>
>>> *Network Solutions A**rchitect *
>>>
>>> *Email [e-mail] <[e-mail]>*
>>>
>>>
>>>
>>> *M 301 502-1347*
>>>
>>> _______________________________________________
>>> Lsr mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/lsr
>>>
>> --
>
> <http://www.verizon.com/>
>
> *Gyan Mishra*
>
> *Network Solutions A**rchitect *
>
> *Email [e-mail] <[e-mail]>*
>
>
>
> *M 301 502-1347*
>
>