Re: [spring] Different MSDs for different traffic types on the same headend.

[Jeff Tantsura <[e-mail]>]

Pleasure, see inline
/*somewhat philosophical ;-)

Cheers,
Jeff
On Dec 17, 2019, 10:28 PM -0800, Gyan Mishra <[e-mail]>, wrote:
>
> Thanks Jeff!!
>
> Kind regards
>
> I have a personal Cisco VIRL server server which I just finished building out the SP topology with two SP SR domains.  Cisco XRv supports SRv6 in 6.6.1 so I have loaded the latest image 7.0.1 so hopefully have all features.  Plan to get it all up and running tomorrow.
>
> Question-  Since SR-MPLS uses an IPv4 data plane and since SRv6 uses IPv6 data plane is it possible to run SRv6 as an overlay both SR-MPLS and SRv6 together as two ships in the night.  Would save on having to go back and forth between simulations if they can both live together  so I can test both simultaneously.
[jeff] I’m not sure what you mean by overlay, just run ipv4+mpls for SR-MPLS and ipv6 for SRv6, perhaps different interfaces rather than dual stack
>
> In-line questions follow up
>
> > On Tue, Dec 17, 2019 at 8:41 PM Jeff Tantsura <[e-mail]> wrote:
> > > Gyan,
> > >
> > > I wrote this presentation ~3 years ago, so the numbers are not necessarily current.
> > >
> > > Please see inline, hope this helps you to better understand the technology behind the terminology.
> > > Let me know if anything is unclear.
> > >
> > > Cheers,
> > > Jeff
> > > On Dec 17, 2019, 3:45 PM -0800, Gyan Mishra <[e-mail]>, wrote:
> > > >
> > > > Hi Ketan
> > > >
> > > > Is this the artificial boundary below for SR-TE from the document provided by Jeff.
> > > >
> > > >  MSD supported by different HW/SW differs widely :
> > > > – Linux (kernel 4.10): 2 SID’s, some improvements recently (as of 4.11) – Low end off the shelf (merchant) silicon, e.g. BCM Trident2: 3-5 SID’s – High end off the shelf (merchant) silicon, e.g BCM Jericho1 : 4-7 SID’s – Vendor’ silicon, e.g. Juniper’ Trio, Nokia’ FP3: 4-10+ SID’s
> > > >
> > > > So if you are doing MPLS-TE w/o SR-TE policies reusing they MPLS data plane the label stack is identical to MPLS just with different label range for SRGB 16000-23999 for absolute or indexed label.
> > > [jeff]SRGB is either configurable or a static label range, it doesn’t have to be identical with SR domain, indexing provides the abstraction (and there’s life outside of Cisco ;-))
> > > > Label stack for SR-MPLS is identical to MPLS with single topmost label for label switching through the core and bottom of stack services label for L3 vpn services.
> > > [jeff] NH in the VPN route is recursively resolved to a LSP (or policy in modern lingo) that could have 1 or more SID’s associated with it (). In MPLS case, without TE, would be the label/N-SID of the egress PE, the SID stack would grow any time you need to use Adj-SID’s to deviate from SPT. Specifically to SR, egress PE can also associate a particular path (color) with the route and signal to the ingress PE what path to take in order to reach it (NH).
> >
> >     [gyan] So if you are running just SR-MPLS using SPF path and don’t deviate from that then with prefix/node SID which I believe is like the egress PE ibgp NH loopback FEC destination, that you have a SID label binding for your LSP to build topmost reusing the MPLS data plane.  In the case described there would be only 1 transport label.  Correct?
[jeff] yes, the only difference with LDP/RSVP would be the “globally significant label” (N-SID of egress PE), so there’s no swap (there is but to the same value)
> >
> > You would only have an Adj-SID if you are doing SR-TE.  Correct?  In that case if you had one Adj-SID in the middle of the path like a loose explicit route object ERO, then would still be only 1 transport label in the stack as if you were not doing SR-TE.  Or would it be 2 labels, not sure.
[jeff] if Adj-SID is locally significant (L flag not set) (default in most implementation) is must be evaluated only by the node that it is local to, which should result in 2 label construct (N-SID,Adj-SID), there are ways to optimize but this is the idea.
> >
> > So let’s say If your SR-TE path deviated from the spf along 10 hops, so then you would have 10 Adj-SIDs and so 10 labels in the stack.  So this is where MSD and BGP-LS IGP extensions come into play to signal back to the head end that the maximum SID depth has exceeded resulting in an error.  At that point the
> > SR-TE LSP would be broken and you would have to manually build policy with less strict hops.
[jeff] depends on implementation, it could as well be 20. No, this is other way around - (simplified version) BGP-LS distributes SR related data to the PCE and then PCE computes a path that meets the constrains, MSD being one of them, in other words - if PCE can’t compute a path to a destination that meets the constrains, it will not be instantiated
> >
> > I guess If you were using PCEP centralized controller that would signal MSD exceeded but I think you would still be in a broken LSP state.  The pcep controller would have to be sitting next to the head end for the SR-TE policy for the LSP to fix automatically.  Is that correct?
[jeff]see previous answer, if MSD (or any other) constrains can’t be met, the path is not going to be instantiated
PCEP is the protocol between PCE and PCC and has little to do with computation logic, PCE could compute a path but use for example RESTCONF to communicate with the head-end
> >
> > So for SR-TE due to the MSD issues and SID depth asic limitations even with the latest asic, you really have to deploy a pcep controller based architecture near the head end with SR-MPLS if using SR-TE.  If you are not using SR-TE then I believe there is really no benefit of going to SR-MPLS.
[jeff] you mean PCE controller. I disagree with you, reducing number of protocols in the network is a great goal (reduces complexity and number of interactions between them). Look at RSVP - there are many people using it but a only little part of them is actually doing TE, most use it for FRR and tactical TE.
To cite a great book ;-) - complexity doesn’t disappear, it moves to different places. In SR we have significantly reduced amount of  state and complexity associated with maintaining it in the network, but it is still there, it has just moved to a controller. In RSVP for example, you could do in-band  BW reservations, in SR it is “soft” controller has to do it and then network gets configured. Multicast works in IP because there’s enough state to compute loop-free MDT and so forth
> >
> > You agree.
> >
> > I guess you save on having to maintain ldp state on each node.  Thats  not a big deal or business justification to switch to SR-MPLS as for operators “MPLS” is their bread & butter they are use to so why change.
[jeff] No, I don’t agree -  the value is not in removing LDP state but reducing overall complexity, interactions between IGPs and LDP are complex and error prone, over the time resulted in 100’s of high severity bugs and broken networks (if you are old enough you should remember :))
> >
> > I think the big benefit for both SR-MPLS and SRv6 is individual coloring of vpn flows.  Cisco calls it per-VRF TE, but it’s ugly to configure and manage.
[jeff] well, complain to your account, building networking features is quite different from intuitive interfaces ;-) color is a piece of metadata associated with a route, on itself not directly related to SR, nothing prevents you of signaling a RSVP LSP or similar, there’s some work on generalization of colored path advertisement which I find very valuable (draft-chan-idr-bgp-lu2/  draft-szarecki-idr-bgp-lcu-traffic-steering)
> >
> > So SR-MPLS and SRv6 is the future solution for operators that need the per vpn coloring.
[jeff] different people have different requirements, our job here is to build technology that is flexible enough to address them at scale while manageable.
> >
> > Given the issues with SR-TE with SID depth asic limitations is it possible to use traditional RSVP TE.
[jeff] I don’t think you are right, while limitations are there, there are solvable, by a better silicon (or trading off capacity for functionality - looping/recycling packets), by a more clever computation, by brining some of the state back (take a look at draft-chunduri-lsr-isis-preferred-path-routing)
> >
> > I am going to try in my virl setup as that could be a good workaround for SR-TE hardware limitations until the hardware is up to snuff to handle very long explicit paths.
> >
> > I believe their is a draft with teas in the works for IP TE to use rsvp bandwidth signaling path and reserve messages with both SR-MPLS and SRv6.
[jeff] good engineering is always a right balance of trade-offs, extremities on either side (no state/all the state in a packet) (OpenFlow is a great example) are rigid and won’t survive when deployed at scale.
> >
> > > >
> > > > For clarity on nomenclature below please comment:
> > > >
> > > > Transport label - This is the SID label stack insertion at the ingress PE head end to the SR-MPLS domain performing SR-TE.  Each hop along the way hop the top SID label is popped until SL==0 at which time PHP default or UHP explicit null “pipe mode” is performed. (Similar to topmost in MPLS label switch world but now its a stack of labels  instead of a single label)
> > > [jeff] There’s no SL field in MPLS header, there’s however BoS bit that should be set on the most bottom label (in this case - VPN label), PHP is explicitly signaled by the egress PE (P/NP-flag in Prefix-SID)
> > > >
> > > > Once you get past the transport label now you are on common ground with M-BGP overlay services
> > > >
> > > > Services label- L3 vpn service SID signaled.
> > > > So with the service label that is signaled from ingress to egress PE for service.  Is this a single label like traditional L3 vpn aggregate label or more then one or many labels.
> > > [jeff] there’s no differences with any other VPNoverFoo, service labels are signaled in exactly same way independently of the transport and are locally significant (that’s why LxVPN’s are such a great technology)
> > > >
> > > > The services label my guess should be fixed and not have any vendor specific boundary constraints.
> > > [jeff] they are locally significant, in any implementation - label blocks are associated with particular functions and programmed in HW, so the service label would come out of the service range, which is significant to the device advertising it.
> > > In modern OS’s the ranges are configurable, in older ones, it is static and there’s usually a header file that looks like:
> > > #define MPLS_L3VPN_MIN_LABEL  0x40000
> > > #define MPLS_L3VPN_MAX_LABEL 0x85FFF
> > >
> > >
> > > > Gyan
> > > >
> > > > > On Tue, Dec 17, 2019 at 7:21 AM Ketan Talaulikar (ketant) <[e-mail]> wrote:
> > > > > > Hi Nat,
> > > > > >
> > > > > > The MSD framework enables us to define more/new MSD types. If there is a real use-case and requirement (as you express) and the necessary MSD type(s) can be formally defined then perhaps the WG can evaluate it.
> > > > > >
> > > > > > Thanks,
> > > > > > Ketan
> > > > > >
> > > > > > From: spring <[e-mail]> On Behalf Of Nat Kao
> > > > > > Sent: 17 December 2019 17:16
> > > > > > To: Robert Raszuk <[e-mail]>
> > > > > > Cc: SPRING WG <[e-mail]>; Nat Kao <[e-mail]>
> > > > > > Subject: Re: [spring] Different MSDs for different traffic types on the same headend.
> > > > > >
> > > > > > Hello, Robert.
> > > > > > Surely the current BMI-MSD definition is sufficient for platforms without artificial boundaries.
> > > > > > In this ideal case, maximum labels available for SR-TE policy can be inferred from BMI-MSD and VPN routes.
> > > > > >
> > > > > > However we have 3 different artificial boundaries across 3 different platforms now.
> > > > > > Hardcoding these boundaries might not scale well. (Especially different OS versions may behave differently.)
> > > > > > A standard mechanism to discover this boundary would greatly help constructing SR-TE policy candidate paths.
> > > > > >
> > > > > > Regards,
> > > > > > Nat.
> > > > > >
> > > > > >
> > > > > > On Tue, Dec 17, 2019 at 5:58 PM Robert Raszuk <[e-mail]> wrote:
> > > > > > > Hi Nat,
> > > > > > >
> > > > > > > I am having a bit of difficulty understanding reasoning and the way you are separating transport from service labels.
> > > > > > >
> > > > > > > The processing label limit usually comes from data plane capabilities of the platform namely LFIB or hardware below.
> > > > > > >
> > > > > > > Such layer is function agnostic and it does not matter what role the label serves.
> > > > > > >
> > > > > > > Label lookup results in pointer to a adj. rewrite on the outbound side. It really does not matter if the adjacent peer is your P router, segment endpoint or CE.
> > > > > > >
> > > > > > > Of course there are number of processing exceptions - for example concept of aggregate VPN label, additional header modifications etc ...
> > > > > > >
> > > > > > > Likely some platforms put such artificial boundary between transport and service hence your request. But if so IMHO this is more of an issue with specific platform or its marketing message then IETF :).
> > > > > > >
> > > > > > > Many thx,
> > > > > > > R.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Dec 17, 2019 at 6:06 AM Nat Kao <[e-mail]> wrote:
> > > > > > > > Hi, Jeff.
> > > > > > > > Consider a headend that can perform 1 of the following 2 modes(but not both):
> > > > > > > > 1) Plain IPv4: 6 transport labels + 0 service label => traffic can be steered into a 6-label SR-TE policy.
> > > > > > > > 2) Any type of VPN: 3 transport labels + 1~3 service labels => traffic cannot be steered into a 6-label SR-TE policy.
> > > > > > > > a) As defined in RFC8491, the BMI-MSD is 6 for this headend. Do we have a standardized way to signal the transport label depth in mode 2?
> > > > > > > >    Maybe in a different MSD type?
> > > > > > > > b) Since plain IPv4 and VPN routes can be steered into the same SR-TE policy, do we have a standardized headend behavior in this situation?
> > > > > > > >    (Should I open a new thread to discuss this? It seems not quite MSD-relative.)
> > > > > > > >
> > > > > > > > Thanks...
> > > > > > > > Nat
> > > > > > > > >
> > > > > > > > > On Tue, Dec 17, 2019 at 7:19 AM Jeff Tantsura <[e-mail]> wrote:
> > > > > > > > > > Gyan,
> > > > > > > > > >
> > > > > > > > > > MSD is only relevant for a device that either imposes the label stack (head-end) or manipulates it (BSID anchor). There are some other constrains when it comes to entropy labels and ERLD, please read the respective drafts.
> > > > > > > > > > In general, SID stack would grow when TE is in use (any time you need to use additional SID to deviate from SPT), another case is when additional SID’s are used for services on the nodes, other than the tail-end.
> > > > > > > > > > That’s why we've designed MSD to be very flexible to accommodate all the different use cases, it is upto computational logic to decide how to deal with different constrains (MSD types)
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > > Jeff
> > > > > > > > > >
> > > > > > > > > > P.S. you might want to see the NANOG MSD presentation I did some time ago.
> > > > > > > > > > https://pc.nanog.org/static/published/meetings/NANOG71/1424/20171004_Tantsura_The_Critical_Role_v1.pdf
> > > > > > > > > > On Dec 14, 2019, 11:59 PM -0800, Gyan Mishra <[e-mail]>, wrote:
> > > > > > > > > >
> > > > > > > > > > > Jeff
> > > > > > > > > > >
> > > > > > > > > > > With SR-MPLS with SR-TR let’s say if you use cSPF snd don’t have an ERO strict explicit path defined or is a loose path, then the for the cSPF would the transport labels be 1.  For loose would also be 1 also.  If the path were explicit defined to egress PE and was 7 hops from ingress to egress then transport would be 6.  And if L3 vpn service sid was signaled that would be 1 vpn label.
> > > > > > > > > > >
> > > > > > > > > > > Let me know if I have that right.
> > > > > > > > > > >
> > > > > > > > > > > In Nats scenario for IPv6 he has 3 vpnv6 labels.
> > > > > > > > > > >
> > > > > > > > > > > Why is that?
> > > > > > > > > > >
> > > > > > > > > > > With both SR-MPLS and SRv6 the L3 vpn AFI/SAFI MBGP services overlay single label sits on top off SR as if does today with MPLS so why 3 vpn labels.
> > > > > > > > > > >
> > > > > > > > > > > So with this draft with BGP-LS and BMI-MSD you can flood into the IGP the SID depth so all the nodes along the SR-TE path don’t go over the maximum which would result in an error.
> > > > > > > > > > >
> > > > > > > > > > > If you set your MTU high enough in the core like 9216, does that overcome the SID depth issues with SR-TE?
> > > > > > > > > > >
> > > > > > > > > > > Warm regards,
> > > > > > > > > > >
> > > > > > > > > > > Gyan
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Dec 14, 2019 at 2:43 AM Nat Kao <[e-mail]> wrote:
> > > > > > > > > > > > Hi, Jeff.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for the BMI-MSD reference. If I understand correctly:
> > > > > > > > > > > >
> > > > > > > > > > > > BMI-MSD = Transport Label Depth + Service Label Depth
> > > > > > > > > > > > Only former can be utilized by SR-TE policies.
> > > > > > > > > > > >
> > > > > > > > > > > > Currently do we have any method to determine the composition of BMI?
> > > > > > > > > > > > We need to know the transport label depth when doing service route per-destination steering.
> > > > > > > > > > > >
> > > > > > > > > > > > This problem arises when trying to steer a plain IPv4 route and a VPN service route into the same SR-TE policy that exceeds the transport label depth of the service route. I'm trying to figure out the standard behavior in this case since the headend we use currently produces some interesting results.
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > > Nat.
> > > > > > > > > > > >
> > > > > > > > > > > > On Sat, Dec 14, 2019 at 2:42 AM Jeff Tantsura <[e-mail]> wrote:
> > > > > > > > > > > > > Hi Nat,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Please read https://tools.ietf.org/html/rfc8491#section-5
> > > > > > > > > > > > > Currently defined MSD types are:
> > > > > > > > > > > > > 1: BMI
> > > > > > > > > > > > > 2: ERLD
> > > > > > > > > > > > >
> > > > > > > > > > > > > Specifically to BMI:
> > > > > > > > > > > > > Base MPLS Imposition MSD (BMI-MSD) signals the total number of MPLS labels that can be imposed, including all service/transport/special labels.
> > > > > > > > > > > > > The answer to your question is 6
> > > > > > > > > > > > >
> > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > Jeff
> > > > > > > > > > > > > On Dec 13, 2019, 3:42 AM -0800, Nat Kao <[e-mail]>, wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hello, SPRING WG.
> > > > > > > > > > > > > > How do we deal with an SR-TE policy headend with different MSDs for different types of traffic?
> > > > > > > > > > > > > > For example, a headend H can impose:
> > > > > > > > > > > > > > 6 transport labels for plain IPv4 packets;
> > > > > > > > > > > > > > 5 transport labels + 1 IPv6 ExpNull label for plain IPv6 packets;
> > > > > > > > > > > > > > 3 transport labels + 3 VPN  labels for VPN packets.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > a) For a plain IPv4 route R4 and a VPN route Rv both steered into the SR-TE policy P1 with SID list <S1, S2, S3, S4, S5>, what will H perform in this situation?
> > > > > > > > > > > > > > b) What is the MSD of H? 6, 5 or 3?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > Nat.
> > > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > > spring mailing list
> > > > > > > > > > > > > > [e-mail]
> > > > > > > > > > > > > > https://www.ietf.org/mailman/listinfo/spring
> > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > spring mailing list
> > > > > > > > > > > > [e-mail]
> > > > > > > > > > > > https://www.ietf.org/mailman/listinfo/spring
> > > > > > > > > > > --
> > > > > > > > > > > Gyan S. Mishra
> > > > > > > > > > > IT Network Engineering & Technology
> > > > > > > > > > > Verizon Communications Inc. (VZ)
> > > > > > > > > > > 13101 Columbia Pike FDC1 3rd Floor
> > > > > > > > > > > Silver Spring, MD 20904
> > > > > > > > > > > United States
> > > > > > > > > > > Phone: [phone]
> > > > > > > > > > > Email: [e-mail]
> > > > > > > > > > > www.linkedin.com/in/networking-technologies-consultant
> > > > > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > spring mailing list
> > > > > > > > [e-mail]
> > > > > > > > https://www.ietf.org/mailman/listinfo/spring
> > > > > > _______________________________________________
> > > > > > spring mailing list
> > > > > > [e-mail]
> > > > > > https://www.ietf.org/mailman/listinfo/spring
> > > > --
> > > > Gyan S. Mishra
> > > > IT Network Engineering & Technology
> > > > Verizon Communications Inc. (VZ)
> > > > 13101 Columbia Pike FDC1 3rd Floor
> > > > Silver Spring, MD 20904
> > > > United States
> > > > Phone: [phone]
> > > > Email: [e-mail]
> > > > www.linkedin.com/in/networking-technologies-consultant
> > > >
> > > > _______________________________________________
> > > > spring mailing list
> > > > [e-mail]
> > > > https://www.ietf.org/mailman/listinfo/spring
> --
> Gyan S. Mishra
> IT Network Engineering & Technology
> Verizon Communications Inc. (VZ)
> 13101 Columbia Pike FDC1 3rd Floor
> Silver Spring, MD 20904
> United States
> Phone: [phone]
> Email: [e-mail]
> www.linkedin.com/in/networking-technologies-consultant
>