Re: [spring] Different MSDs for different traffic types on the same headend.

[Gyan Mishra <[e-mail]>]

Thank you Jeff for taking the time for the detailed response!!

Cheers & Happy Holidays!!

Gyan

On Wed, Dec 18, 2019 at 7:13 PM Jeff Tantsura <[e-mail]>
wrote:

> Pleasure, see inline
> /*somewhat philosophical ;-)
>
> Cheers,
> Jeff
> On Dec 17, 2019, 10:28 PM -0800, Gyan Mishra <[e-mail]>,
> wrote:
>
>
> Thanks Jeff!!
>
> Kind regards
>
> I have a personal Cisco VIRL server server which I just finished building
> out the SP topology with two SP SR domains.  Cisco XRv supports SRv6 in
> 6.6.1 so I have loaded the latest image 7.0.1 so hopefully have all
> features.  Plan to get it all up and running tomorrow.
>
> Question-  Since SR-MPLS uses an IPv4 data plane and since SRv6 uses IPv6
> data plane is it possible to run SRv6 as an overlay both SR-MPLS and SRv6
> together as two ships in the night.  Would save on having to go back and
> forth between simulations if they can both live together  so I can test
> both simultaneously.
>
> [jeff] I’m not sure what you mean by overlay, just run ipv4+mpls for
> SR-MPLS and ipv6 for SRv6, perhaps different interfaces rather than dual
> stack
>
>
> In-line questions follow up
>
> On Tue, Dec 17, 2019 at 8:41 PM Jeff Tantsura <[e-mail]>
> wrote:
>
>> Gyan,
>>
>> I wrote this presentation ~3 years ago, so the numbers are not
>> necessarily current.
>>
>> Please see inline, hope this helps you to better understand the
>> technology behind the terminology.
>> Let me know if anything is unclear.
>>
>> Cheers,
>> Jeff
>> On Dec 17, 2019, 3:45 PM -0800, Gyan Mishra <[e-mail]>,
>> wrote:
>>
>>
>> Hi Ketan
>>
>> Is this the artificial boundary below for SR-TE from the document
>> provided by Jeff.
>>
>>  MSD supported by different HW/SW differs widely :
>> – Linux (kernel 4.10): 2 SID’s, some improvements recently (as of 4.11) –
>> Low end off the shelf (merchant) silicon, e.g. BCM Trident2: 3-5 SID’s –
>> High end off the shelf (merchant) silicon, e.g BCM Jericho1 : 4-7 SID’s –
>> Vendor’ silicon, e.g. Juniper’ Trio, Nokia’ FP3: 4-10+ SID’s
>>
>> So if you are doing MPLS-TE w/o SR-TE policies reusing they MPLS data
>> plane the label stack is identical to MPLS just with different label range
>> for SRGB 16000-23999 for absolute or indexed label.
>>
>> [jeff]SRGB is either configurable or a static label range, it doesn’t
>> have to be identical with SR domain, indexing provides the abstraction (and
>> there’s life outside of Cisco ;-))
>>
>> Label stack for SR-MPLS is identical to MPLS with single topmost label
>> for label switching through the core and bottom of stack services label for
>> L3 vpn services.
>>
>> [jeff] NH in the VPN route is recursively resolved to a LSP (or policy in
>> modern lingo) that could have 1 or more SID’s associated with it (). In
>> MPLS case, without TE, would be the label/N-SID of the egress PE, the SID
>> stack would grow any time you need to use Adj-SID’s to deviate from SPT.
>> Specifically to SR, egress PE can also associate a particular path (color)
>> with the route and signal to the ingress PE what path to take in order to
>> reach it (NH).
>>
>
>     [gyan] So if you are running just SR-MPLS using SPF path and don’t
> deviate from that then with prefix/node SID which I believe is like the
> egress PE ibgp NH loopback FEC destination, that you have a SID label
> binding for your LSP to build topmost reusing the MPLS data plane.  In the
> case described there would be only 1 transport label.  Correct?
>
> [jeff] yes, the only difference with LDP/RSVP would be the “globally
> significant label” (N-SID of egress PE), so there’s no swap (there is but
> to the same value)
>
>
> You would only have an Adj-SID if you are doing SR-TE.  Correct?  In that
> case if you had one Adj-SID in the middle of the path like a loose explicit
> route object ERO, then would still be only 1 transport label in the stack
> as if you were not doing SR-TE.  Or would it be 2 labels, not sure.
>
> [jeff] if Adj-SID is locally significant (L flag not set) (default in most
> implementation) is must be evaluated only by the node that it is local to,
> which should result in 2 label construct (N-SID,Adj-SID), there are ways to
> optimize but this is the idea.
>
>
> So let’s say If your SR-TE path deviated from the spf along 10 hops, so
> then you would have 10 Adj-SIDs and so 10 labels in the stack.  So this is
> where MSD and BGP-LS IGP extensions come into play to signal back to the
> head end that the maximum SID depth has exceeded resulting in an error.  At
> that point the
> SR-TE LSP would be broken and you would have to manually build policy with
> less strict hops.
>
> [jeff] depends on implementation, it could as well be 20. No, this is
> other way around - (simplified version) BGP-LS distributes SR related data
> to the PCE and then PCE computes a path that meets the constrains, MSD
> being one of them, in other words - if PCE can’t compute a path to a
> destination that meets the constrains, it will not be instantiated
>
>
> I guess If you were using PCEP centralized controller that would signal
> MSD exceeded but I think you would still be in a broken LSP state.  The
> pcep controller would have to be sitting next to the head end for the SR-TE
> policy for the LSP to fix automatically.  Is that correct?
>
> [jeff]see previous answer, if MSD (or any other) constrains can’t be met,
> the path is not going to be instantiated
> PCEP is the protocol between PCE and PCC and has little to do with
> computation logic, PCE could compute a path but use for example RESTCONF to
> communicate with the head-end
>
>
> So for SR-TE due to the MSD issues and SID depth asic limitations even
> with the latest asic, you really have to deploy a pcep controller based
> architecture near the head end with SR-MPLS if using SR-TE.  If you are not
> using SR-TE then I believe there is really no benefit of going to SR-MPLS.
>
>
> [jeff] you mean PCE controller. I disagree with you, reducing number of
> protocols in the network is a great goal (reduces complexity and number of
> interactions between them). Look at RSVP - there are many people using it
> but a only little part of them is actually doing TE, most use it for FRR
> and tactical TE.
> To cite a great book ;-) - complexity doesn’t disappear, it moves to
> different places. In SR we have significantly reduced amount of  state and
> complexity associated with maintaining it in the network, but it is still
> there, it has just moved to a controller. In RSVP for example, you could do
> in-band  BW reservations, in SR it is “soft” controller has to do it and
> then network gets configured. Multicast works in IP because there’s enough
> state to compute loop-free MDT and so forth
>
>
> You agree.
>
> I guess you save on having to maintain ldp state on each node.  Thats  not
> a big deal or business justification to switch to SR-MPLS as for operators
> “MPLS” is their bread & butter they are use to so why change.
>
> [jeff] No, I don’t agree -  the value is not in removing LDP state but
> reducing overall complexity, interactions between IGPs and LDP are complex
> and error prone, over the time resulted in 100’s of high severity bugs and
> broken networks (if you are old enough you should remember :))
>
>
> I think the big benefit for both SR-MPLS and SRv6 is individual coloring
> of vpn flows.  Cisco calls it per-VRF TE, but it’s ugly to configure and
> manage.
>
> [jeff] well, complain to your account, building networking features is
> quite different from intuitive interfaces ;-) color is a piece of metadata
> associated with a route, on itself not directly related to SR, nothing
> prevents you of signaling a RSVP LSP or similar, there’s some work on
> generalization of colored path advertisement which I find very valuable
> (draft-chan-idr-bgp-lu2/  draft-szarecki-idr-bgp-lcu-traffic-steering)
>
>
> So SR-MPLS and SRv6 is the future solution for operators that need the per
> vpn coloring.
>
> [jeff] different people have different requirements, our job here is to
> build technology that is flexible enough to address them at scale while
> manageable.
>
>
> Given the issues with SR-TE with SID depth asic limitations is it possible
> to use traditional RSVP TE.
>
> [jeff] I don’t think you are right, while limitations are there, there are
> solvable, by a better silicon (or trading off capacity for functionality -
> looping/recycling packets), by a more clever computation, by brining some
> of the state back (take a look
> at draft-chunduri-lsr-isis-preferred-path-routing)
>
>
> I am going to try in my virl setup as that could be a good workaround for
> SR-TE hardware limitations until the hardware is up to snuff to handle very
> long explicit paths.
>
> I believe their is a draft with teas in the works for IP TE to use rsvp
> bandwidth signaling path and reserve messages with both SR-MPLS and SRv6.
>
> [jeff] good engineering is always a right balance of trade-offs,
> extremities on either side (no state/all the state in a packet) (OpenFlow
> is a great example) are rigid and won’t survive when deployed at scale.
>
>
>
>> For clarity on nomenclature below please comment:
>>
>> Transport label - This is the SID label stack insertion at the ingress PE
>> head end to the SR-MPLS domain performing SR-TE.  Each hop along the way
>> hop the top SID label is popped until SL==0 at which time PHP default or
>> UHP explicit null “pipe mode” is performed. (Similar to topmost in MPLS
>> label switch world but now its a stack of labels  instead of a single
>> label)
>>
>> [jeff] There’s no SL field in MPLS header, there’s however BoS bit that
>> should be set on the most bottom label (in this case - VPN label), PHP is
>> explicitly signaled by the egress PE (P/NP-flag in Prefix-SID)
>>
>>
>> Once you get past the transport label now you are on common ground with
>> M-BGP overlay services
>>
>> Services label- L3 vpn service SID signaled.
>> So with the service label that is signaled from ingress to egress PE for
>> service.  Is this a single label like traditional L3 vpn aggregate label or
>> more then one or many labels.
>>
>> [jeff] there’s no differences with any other VPNoverFoo, service labels
>> are signaled in exactly same way independently of the transport and are
>> locally significant (that’s why LxVPN’s are such a great technology)
>>
>>
>> The services label my guess should be fixed and not have any vendor
>> specific boundary constraints.
>>
>> [jeff] they are locally significant, in any implementation - label blocks
>> are associated with particular functions and programmed in HW, so the
>> service label would come out of the service range, which is significant to
>> the device advertising it.
>> In modern OS’s the ranges are configurable, in older ones, it is static
>> and there’s usually a header file that looks like:
>> #define MPLS_L3VPN_MIN_LABEL  0x40000
>> #define MPLS_L3VPN_MAX_LABEL 0x85FFF
>>
>>
>> Gyan
>>
>> On Tue, Dec 17, 2019 at 7:21 AM Ketan Talaulikar (ketant) <
>> [e-mail]> wrote:
>>
>>> Hi Nat,
>>>
>>>
>>>
>>> The MSD framework enables us to define more/new MSD types. If there is a
>>> real use-case and requirement (as you express) and the necessary MSD
>>> type(s) can be formally defined then perhaps the WG can evaluate it.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Ketan
>>>
>>>
>>>
>>> *From:* spring <[e-mail]> *On Behalf Of* Nat Kao
>>> *Sent:* 17 December 2019 17:16
>>> *To:* Robert Raszuk <[e-mail]>
>>> *Cc:* SPRING WG <[e-mail]>; Nat Kao <[e-mail]>
>>> *Subject:* Re: [spring] Different MSDs for different traffic types on
>>> the same headend.
>>>
>>>
>>>
>>> Hello, Robert.
>>>
>>> Surely the current BMI-MSD definition is sufficient for platforms
>>> without artificial boundaries.
>>>
>>> In this ideal case, maximum labels available for SR-TE policy can be
>>> inferred from BMI-MSD and VPN routes.
>>>
>>>
>>> However we have 3 different artificial boundaries across 3 different
>>> platforms now.
>>>
>>> Hardcoding these boundaries might not scale well. (Especially different
>>> OS versions may behave differently.)
>>>
>>> A standard mechanism to discover this boundary would greatly help
>>> constructing SR-TE policy candidate paths.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Nat.
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Dec 17, 2019 at 5:58 PM Robert Raszuk <[e-mail]> wrote:
>>>
>>> Hi Nat,
>>>
>>>
>>>
>>> I am having a bit of difficulty understanding reasoning and the way you
>>> are separating transport from service labels.
>>>
>>>
>>>
>>> The processing label limit usually comes from data plane capabilities of
>>> the platform namely LFIB or hardware below.
>>>
>>>
>>>
>>> Such layer is function agnostic and it does not matter what role the
>>> label serves.
>>>
>>>
>>>
>>> Label lookup results in pointer to a adj. rewrite on the outbound side.
>>> It really does not matter if the adjacent peer is your P router, segment
>>> endpoint or CE.
>>>
>>>
>>>
>>> Of course there are number of processing exceptions - for example
>>> concept of aggregate VPN label, additional header modifications etc ...
>>>
>>>
>>>
>>> Likely some platforms put such artificial boundary between transport and
>>> service hence your request. But if so IMHO this is more of an issue with
>>> specific platform or its marketing message then IETF :).
>>>
>>>
>>>
>>> Many thx,
>>> R.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Dec 17, 2019 at 6:06 AM Nat Kao <[e-mail]> wrote:
>>>
>>> Hi, Jeff.
>>>
>>> Consider a headend that can perform 1 of the following 2 modes(but not
>>> both):
>>>
>>> 1) Plain IPv4: 6 transport labels + 0 service label => traffic can be
>>> steered into a 6-label SR-TE policy.
>>>
>>> 2) Any type of VPN: 3 transport labels + 1~3 service labels => traffic
>>> cannot be steered into a 6-label SR-TE policy.
>>>
>>> a) As defined in RFC8491, the BMI-MSD is 6 for this headend. Do we have
>>> a standardized way to signal the transport label depth in mode 2?
>>>
>>>    Maybe in a different MSD type?
>>>
>>> b) Since plain IPv4 and VPN routes can be steered into the same SR-TE
>>> policy, do we have a standardized headend behavior in this situation?
>>>
>>>    (Should I open a new thread to discuss this? It seems not quite
>>> MSD-relative.)
>>>
>>>
>>>
>>> Thanks...
>>>
>>> Nat
>>>
>>>
>>>
>>> On Tue, Dec 17, 2019 at 7:19 AM Jeff Tantsura <[e-mail]>
>>> wrote:
>>>
>>> Gyan,
>>>
>>>
>>>
>>> MSD is only relevant for a device that either imposes the label stack
>>> (head-end) or manipulates it (BSID anchor). There are some other constrains
>>> when it comes to entropy labels and ERLD, please read the respective drafts.
>>>
>>> In general, SID stack would grow when TE is in use (any time you need to
>>> use additional SID to deviate from SPT), another case is when additional
>>> SID’s are used for services on the nodes, other than the tail-end.
>>>
>>> That’s why we've designed MSD to be very flexible to accommodate all the
>>> different use cases, it is upto computational logic to decide how to deal
>>> with different constrains (MSD types)
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Jeff
>>>
>>>
>>>
>>> P.S. you might want to see the NANOG MSD presentation I did some time
>>> ago.
>>>
>>>
>>> https://pc.nanog.org/static/published/meetings/NANOG71/1424/20171004_Tantsura_The_Critical_Role_v1.pdf
>>>
>>> On Dec 14, 2019, 11:59 PM -0800, Gyan Mishra <[e-mail]>,
>>> wrote:
>>>
>>> Jeff
>>>
>>>
>>>
>>> With SR-MPLS with SR-TR let’s say if you use cSPF snd don’t have an ERO
>>> strict explicit path defined or is a loose path, then the for the cSPF
>>> would the transport labels be 1.  For loose would also be 1 also.  If the
>>> path were explicit defined to egress PE and was 7 hops from ingress to
>>> egress then transport would be 6.  And if L3 vpn service sid was signaled
>>> that would be 1 vpn label.
>>>
>>>
>>>
>>> Let me know if I have that right.
>>>
>>>
>>>
>>> In Nats scenario for IPv6 he has 3 vpnv6 labels.
>>>
>>>
>>>
>>> Why is that?
>>>
>>>
>>>
>>> With both SR-MPLS and SRv6 the L3 vpn AFI/SAFI MBGP services overlay
>>> single label sits on top off SR as if does today with MPLS so why 3 vpn
>>> labels.
>>>
>>>
>>>
>>> So with this draft with BGP-LS and BMI-MSD you can flood into the IGP
>>> the SID depth so all the nodes along the SR-TE path don’t go over the
>>> maximum which would result in an error.
>>>
>>>
>>>
>>> If you set your MTU high enough in the core like 9216, does that
>>> overcome the SID depth issues with SR-TE?
>>>
>>>
>>>
>>> Warm regards,
>>>
>>>
>>>
>>> Gyan
>>>
>>>
>>>
>>> On Sat, Dec 14, 2019 at 2:43 AM Nat Kao <[e-mail]> wrote:
>>>
>>> Hi, Jeff.
>>>
>>>
>>>
>>> Thanks for the BMI-MSD reference. If I understand correctly:
>>>
>>>
>>>
>>> BMI-MSD = Transport Label Depth + Service Label Depth
>>>
>>> Only former can be utilized by SR-TE policies.
>>>
>>>
>>>
>>> Currently do we have any method to determine the composition of BMI?
>>>
>>> We need to know the transport label depth when doing service route
>>> per-destination steering.
>>>
>>>
>>>
>>> This problem arises when trying to steer a plain IPv4 route and a VPN
>>> service route into the same SR-TE policy that exceeds the transport label
>>> depth of the service route. I'm trying to figure out the standard behavior
>>> in this case since the headend we use currently produces some interesting
>>> results.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Nat.
>>>
>>>
>>>
>>> On Sat, Dec 14, 2019 at 2:42 AM Jeff Tantsura <[e-mail]
>>> <[e-mail]>> wrote:
>>>
>>> Hi Nat,
>>>
>>>
>>>
>>> Please read https://tools.ietf.org/html/rfc8491#section-5
>>>
>>> Currently defined MSD types are:
>>>
>>> 1: BMI
>>>
>>> 2: ERLD
>>>
>>>
>>>
>>> Specifically to BMI:
>>>
>>> Base MPLS Imposition MSD (BMI-MSD) signals the total number of
>>> MPLS labels that can be imposed, including all
>>> service/transport/special labels.
>>>
>>> The answer to your question is 6
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Jeff
>>>
>>> On Dec 13, 2019, 3:42 AM -0800, Nat Kao <[e-mail]>, wrote:
>>>
>>> Hello, SPRING WG.
>>>
>>> How do we deal with an SR-TE policy headend with different MSDs for
>>> different types of traffic?
>>>
>>> For example, a headend H can impose:
>>> 6 transport labels for plain IPv4 packets;
>>> 5 transport labels + 1 IPv6 ExpNull label for plain IPv6 packets;
>>>
>>> 3 transport labels + 3 VPN  labels for VPN packets.
>>>
>>>
>>>
>>> a) For a plain IPv4 route R4 and a VPN route Rv both steered into the
>>> SR-TE policy P1 with SID list <S1, S2, S3, S4, S5>, what will H perform in
>>> this situation?
>>> b) What is the MSD of H? 6, 5 or 3?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Nat.
>>>
>>> _______________________________________________
>>> spring mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/spring
>>>
>>> _______________________________________________
>>> spring mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/spring
>>>
>>> --
>>>
>>> Gyan S. Mishra
>>>
>>> IT Network Engineering & Technology
>>>
>>> Verizon Communications Inc. (VZ)
>>>
>>> 13101 Columbia Pike
>>> <https://www..google.com/maps/search/13101+Columbia+Pike?entry=gmail&source=g>
>>> FDC1 3rd Floor
>>>
>>> Silver Spring, MD 20904
>>>
>>> United States
>>>
>>> Phone: [phone]
>>>
>>> Email: [e-mail]
>>>
>>> www.linkedin.com/in/networking-technologies-consultant
>>>
>>>
>>>
>>> _______________________________________________
>>> spring mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/spring
>>>
>>> _______________________________________________
>>> spring mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/spring
>>>
>> --
>>
>> Gyan S. Mishra
>>
>> IT Network Engineering & Technology
>>
>> Verizon Communications Inc. (VZ)
>>
>> 13101 Columbia Pike
>> <https://www.google.com/maps/search/13101+Columbia+Pike?entry=gmail&source=g>
>> FDC1 3rd Floor
>>
>> Silver Spring, MD 20904
>>
>> United States
>>
>> Phone: [phone]
>>
>> Email: [e-mail]
>>
>> www.linkedin.com/in/networking-technologies-consultant
>>
>> _______________________________________________
>> spring mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/spring
>>
>> --
>
> Gyan S. Mishra
>
> IT Network Engineering & Technology
>
> Verizon Communications Inc. (VZ)
>
> 13101 Columbia Pike
> <https://www.google.com/maps/search/13101+Columbia+Pike?entry=gmail&source=g>
> FDC1 3rd Floor
>
> Silver Spring, MD 20904
>
> United States
>
> Phone: [phone]
>
> Email: [e-mail]
>
> www.linkedin.com/in/networking-technologies-consultant
>
> --

Gyan S. Mishra

IT Network Engineering & Technology

Verizon Communications Inc. (VZ)

13101 Columbia Pike FDC1 3rd Floor

Silver Spring, MD 20904

United States

Phone: [phone]

Email: [e-mail]

www.linkedin.com/in/networking-technologies-consultant