Re: [Rift] [RIFT] spine-tof links break => spine-leaf adjacencies go down?

[Tony Przygienda <[e-mail]>]

Absolutely intendend and expected behavior ;-) You can't figure out
topology from spine down basically without a ToF. ToFs are the only source
of truth where north is and if that isn't around no'one knows which way is
up ...

It's actually easy to put some code in that watches for the condition on a
node & complains that there are no valid offers.

Alternative would be to "store" offers and use them even if a node went
away. Think through that, it's impossible to converge the protocol
correctly then without having something like flooding of offers with seqnr#
and so on ...

--- tony

On Thu, Apr 23, 2020 at 11:13 AM Bruno Rijsman <[e-mail]>
wrote:

> While testing negative disaggregation with Mariano, I discovered an
> "interesting" behavior in RIFT and I would like to check whether this
> behavior is expected or not. I am not saying the behavior is wrong, I was
> just surprised to see it and wanted to check whether this was intended when
> the protocol was designed.
>
> Consider the following topology:
>
>  TOF1     TOF2
>    \       /
>     \     /
>      \   /
>      SPINE
>      /   \
>     /     \
>    /       \
>  LEAF1  LEAF2
>
> All adjacencies are 3way.
>
> The TOFs have top-of-fabric level 24.
>
> The leaves have level 0.
>
> The spine uses ZTP to derive a level, so it ends up with level 23.
>
> Now we break all links between the spine and its TOFs:
>
>  TOF1     TOF2
>    \       /
>     X     X   <<< These adj go down (as expected)
>      \   /
>      SPINE
>      /   \
>     /     \   <<< Surprise: these adj also go down
>    /       \
>  LEAF1  LEAF2
>
> At this point the spine node does not have any Valid Offered Levels (VOLs)
> because (see [1]):
> (1) The levels from the TOFs are not valid because the adjacencies are down
> (2) The levels from leaves are never accepted as valid offers
>
> Hence, the derived level for the spine becomes "undefined".
>
> Once the level of the spine becomes undefined, it stops accepting LIE
> messages from the leaves because a node whose level is undefined never
> accepts LIEs (see [2] below).
>
> Since the spine stops accepting LIEs from the leaves, the old LIEs time
> out and the adjacencies go down.
>
> *My question is: it is normal and expected that breaking the links from
> the spine to the TOFs *also* causes the adjacencies between th spine and
> the leaves to go down?*
>
> [1] On page [86] of draft-11 it says:
>
>  Valid Offered Level (VOL): A neighbor’s level received on a valid
>  LIE (i.e. passing all checks for adjacency formation while
>  disregarding all clauses involving level values) persisting for
>  the duration of the holdtime interval on the LIE. *Observe that*
> * offers from nodes offering level value of 0 do not constitute VOLs*
> * (since no valid DERIVED_LEVEL can be obtained from those and*
> * consequently ‘not_a_ztp_offer‘ MUST be ignored)*. Offers from LIEs
>  with ‘not_a_ztp_offer‘ being true are not VOLs either. If a node
>  maintains parallel adjacencies to the neighbor, VOL on each
>  adjacency is considered as equivalent, i.e. the newest VOL from
>  any such adjacency updates the VOL received from the same node.
>
> [2] On page [45] of draft -11 it says:
>
>  4. PROCESS_LIE:
>  1. if lie has wrong major version OR our own system ID or
>  invalid system ID then CLEANUP else
>  2. if lie has non matching MTUs then CLEANUP, PUSH
>  UpdateZTPOffer, PUSH MTUMismatch else
>  3. if PoD rules do not allow adjacency forming then CLEANUP,
>  PUSH PODMismatch, PUSH MTUMismatch else
>  4. if lie has undefined level OR *my level is undefined* OR this
>  node is leaf and remote level lower than HAT OR (lie’s level
>  is not leaf AND its difference is more than one from my
>  level) then CLEANUP, PUSH UpdateZTPOffer, PUSH
>  UnacceptableHeader else
>
>