Re: [Rift] RIFT Open Source Update Presentation (RIFT WG meeting IETF-108)

[Bruno Rijsman <[e-mail]>]

Hi Robert,

Thanks for the observations and questions!

See answers below.

— Bruno

> On Jul 25, 2020, at 8:56 PM, Robert Raszuk <[e-mail]> wrote:
> 
> Hi Bruno,
> 
> This is excellent ! I wish everyone would prepare such video ahead of time like you did. 
> 
> One observation and few questions ... 
> 
> * First let me observe that concept of negative routing in the control plane is not new. As example - the proposals on that have been written many years ago for BGP. Perhaps it is unfortunate that vendor did not choose to pursue it in public at that time :) But it did pursue patents and implementation for negative routing over low power and lossy networks.
> 

I would be very interested in the negative routing proposal that was written for BGP (or for any other protocol). Do you have any public references? Did it also include an automatic mechanism for triggering negative disaggregation, or did it require manual configuration of the negative routes to be advertised? (Since you mention the vendor did not pursue this in public, I would understand if you don't have a reference.)

Either way, I certainly did not intend to imply that negative disaggregation has never been done before. In technology, as in art, almost everything has been done before. It's just that the publicly available implementations of BGP, ISIS, OSPF, etc. currently don't include the concept of negative disaggregation, or at least not as far as I know.

Even within the narrow scope of RIFT, I don't claim to have contributed any original thoughts on the topic of negative disaggregation -- the vast majority of the credit for that goes to Pascal Thubert (it is perhaps no coincidence that he works on low power and lossy networks for a major network equipment vendor, perhaps the same one that you allude to).

> * In your talk you describe the behaviour in rest of the network in regards to negative routing. But how will leaf-1 in pod-1 receive negative routes if the real trigger for them is based on control plane messages traveling over horizontal links between super spines ? 


Leaf-1-1 in pod-1 does not rely on negative disaggregation to avoid plane-1.

Instead it relies on the fact that RIFT originates a south-bound default route at the top-of-fabric nodes, and then lower nodes in the hierarchy only propagate the south-bound default route if they have received a default route advertisement from at least one parent.

This is a little bit of a simplification, see section 4.2.3.8in version 12 of the RIFT draft for details, and there is also an interesting interaction with ZTP.

See APPENDIX below for details.


> 
> * How do you stabilize/control control plane and data plane churn during negative routing being triggered by link flapping (say one down and one flapping in your example between pod 1 and plane 1) and no native link dampening in place ? 
> 

TIEs (= RIFT Link State Packets) for negative disaggregate prefixes are subject to the same link-flap considerations that apply to normal node TIEs or normal positive prefix TIE.

Hence, all the usual mechanisms that protect link-state protocols from excessive link flapping apply, including but not limited to:

* Limiting the rate at which TIEs are flooded over adjacencies. In RIFT there are transmit queues for TIEs, and a distinction is made between fast transmits (for the initial transmission) and slow re-transmits (for retransmits).

* Limiting the rate at which SPF runs are conducted.

* Various flow-control mechanisms (including sequence numbers and the `you_are_sending_too_quickly` field in LIEs) to detect that RIFT packets are being dropped and adjusting the flooding rates accordingly. (I have not yet implemented this in my code -- see also PS below).

> * Is the load balancing spread based on the configured link bandwidth for the interfaces ? If so it does not account for brownouts where fabric in any network element itself or linecard partially fails and while say BFD or RIFT still goes through the actual bandwidth for the data plane is severely reduced. Any plans for this little enhancement ? :) 
> 

In my code, the load balancing is indeed based on the configured interface speed.

That said, it is an implementation choice, and there is nothing in the RIFT draft that prevents an implementation from doing something more sophisticated, for example using bandwidth based on actual telemetry measurements. I currently have no plans to do that -- see also PS below.

> Thank you,
> R.
> 
> PS. The FSM performance is a fantastic addition. I wish all code would have such CLI :) But while we are at this is there some analysis how would python-rift performance compare with say c-rift when dealing with say 1M to 10M routes fabric ? Case of no overlay and flat routing of /32s and /128s all over the cluster. 
> 

The RIFT-Python code was originally started with the following goals in mind:

* Check whether it was possible to create an interoperability implementation of RIFT based purely on the text in the RIFT specification, and where necessary, improve the quality of the RIFT specification.

* Be a reference implementation of RIFT.

It was never a goal of the RIFT-Python project to support 1000s of routes or millions of routes. The code is optimized for clarity and readability. It is not optimized for performance: it is not multi-threaded and Python is not a logical choice for performance.

That said, I expect that RIFT-Python would make a fine host-based RIFT router since the RIFT performance demand on leaf routers is very modest, even in very large topologies.



APPENDIX

Let's go through the concrete example shown on slide 12 in the presentation deck.

Before the links are broken, the sitation is as follows:


Node super-1-1 originates a south-bound default route because it is top-of-fabric:

super-1-1> show node
Node:
+-------------------------------------------+-------------------------------------------------------------+
| Name                                      | super-1-1                                                   |
+-------------------------------------------+-------------------------------------------------------------+
| Originate IPv4 Default Route              | True                                                        |
| Reason for Originating IPv4 Default Route | All other nodes at my level have no north-bound adjacencies |
| Originate IPv6 Default Route              | True                                                        |
| Reason for Originating IPv6 Default Route | All other nodes at my level have no north-bound adjacencies |
+-------------------------------------------+-------------------------------------------------------------+

Node spine-1-1 originates a south-bound default route (as do spine-1-2 and spine-1-3) because it received a default advertisement from the superspine and computed a default route to install in its RIB:

spine-1-1> show node
Node:
+-------------------------------------------+----------------------------------------------+
| Name                                      | spine-1-1                                    |
+-------------------------------------------+----------------------------------------------+
| Originate IPv4 Default Route              | True                                         |
| Reason for Originating IPv4 Default Route | This node has north-bound IPv4 default route |
| Originate IPv6 Default Route              | True                                         |
| Reason for Originating IPv6 Default Route | This node has north-bound IPv6 default route |
+-------------------------------------------+----------------------------------------------+

Hence, leaf-1-1 has north-bound default routes that ECMP the traffic over all three planes (for the sake of space I only show IPv4 routes):

leaf-1-1> show route
IPv4 Routes:
+-----------+-----------+----------+-----------------+----------+----------+
| Prefix    | Owner     | Next-hop | Next-hop        | Next-hop | Next-hop |
|           |           | Type     | Interface       | Address  | Weight   |
+-----------+-----------+----------+-----------------+----------+----------+
| 0.0.0.0/0 | North SPF | Positive | veth-1001a-101a | 99.0.2.2 | 33       |
|           |           | Positive | veth-1001b-102a | 99.0.4.2 | 33       |
|           |           | Positive | veth-1001c-103a | 99.0.6.2 | 33       |
+-----------+-----------+----------+-----------------+----------+----------+

After we break links spine-1-1 <-> super-1-1 and spine-1-1 <-> super-1-2, we would expect the following to happen:

1. Super-1-1 still advertises a south-bound default route.
2. But spine-1-1 doesn't receives the default advertisement from any superspine (since all links are broken) and hence does not propage the default advertisement:
3. Hence, leaf-1-1 now has a north-bound default route that only ECMPs over plane-2 and plane-3, but avoids plane-1:

leaf-1-1> show route
IPv4 Routes:
+-----------+-----------+----------+-----------------+----------+----------+
| Prefix    | Owner     | Next-hop | Next-hop        | Next-hop | Next-hop |
|           |           | Type     | Interface       | Address  | Weight   |
+-----------+-----------+----------+-----------------+----------+----------+
| 0.0.0.0/0 | North SPF | Positive | veth-1001b-102a | 99.0.4.2 | 50       |
|           |           | Positive | veth-1001c-103a | 99.0.6.2 | 50       |
+-----------+-----------+----------+-----------------+----------+----------+

However, before we even get there something else quite surprising happens:

1. When spine-1-1 loses both north-bound adjacencies, it is no longer able to auto-derive it's level using ZTP.
2. This causes spine-1-1 to shut down all of its adjacencies including the south-bound ones to the leaves.
3. This has the same (good) result: leaf-1-1 now has a north-bound default route that only ECMPs over plane-2 and plane-3, but avoids plane-1.

This was discussed on the RIFT mailing list on 23-April-2020 (see https://mailarchive.ietf.org/arch/msg/rift/zGLnispMv630LWq0ALRgGs7dLJI/)


> 
> On Sat, 25 Jul 2020 at 18:44, Bruno Rijsman <[e-mail] <mailto:[e-mail]>> wrote:
> Because IETF-108 is a pure virtual meeting, I went through the trouble of pre-recording my "RIFT Open Source Update" presentation scheduled for the RIFT working group meeting on  Wednesday.
> 
> Enjoy the "Sneak preview" ;-)
> 
> Video: https://www.youtube.com/watch?v=qhiiTDPuku0 <https://www.youtube.com/watch?v=qhiiTDPuku0>
> 
> Slides: https://bit.ly/rift-open-source-update-ietf-108 <https://bit.ly/rift-open-source-update-ietf-108>
> 
> — Bruno Rijsman
> _______________________________________________
> RIFT mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/rift <https://www.ietf.org/mailman/listinfo/rift>