Re: [Rift] Why negative disaggregation cannot replace positive disaggregation in all cases.

[Tony Przygienda <[e-mail]>]

chairs, if we start a monthly again I think we need interim framework to be
in line here ...

-- tony

On Mon, May 18, 2020 at 7:25 AM Bruno Rijsman <[e-mail]>
wrote:

> Hi Pascal,
>
> Yes, we can do a monthly.
>
> For now, here is the issue that I ran into that makes me believe that
> negative disaggregation cannot replace positive disaggregation in all cases
> (specifically not for the “normal” case of non-tof routers).
>
> Here is a simplified summary of how positive disaggregation works (see
> section 4.2.5.1 for details):
>
> If this router R1 has a "direct route" for prefix P, and that route P has
> a set of next-hop {NH}, and there is some other router R2 at the same level
> as R1 that does not have NH as a south-adjacency for each NH in {NH}, then
> this router R1 must positively disaggregate prefix P: this router R1 must
> "attract" the traffic so that it doesn't go to the other router R2 (who
> cannot delivery the traffic to P).
>
> Now let's try to the the reverse thing with negative disaggregation:
>
> If some other router R2 at the same level as R1, and R2 has a "direct
> route" for prefix P, and that route P has some set of next-hops {NH}, and
> this router R1 does not have NH as a south-adjacency for each NH in {NH},
> then this router R1 must negatively disaggregate prefix P: this router R1
> must "repel" the traffic so that it goes to R2 instead of R1.
>
> The problem that this requires router R1 to “ now” what the set of routes
> is that router R2 has, and to know the next-hops for those routes. But R1
> cannot know/calculate the "direct routes" that R2 has because it doesn't
> know which north-TIEs R2 has in its database. So, I don’t think we can do
> negative disaggregation here...
>
> Am I correct or am I missing something?
>
> — Bruno
>
>
> On May 18, 2020, at 12:28 AM, Antoni Przygienda <[e-mail]> wrote:
>
> Agreed. Let’s set up a monthly
>
>
>    - Tony
>
>
> *From: *"Pascal Thubert (pthubert)" <pthubert=[e-mail]>
> *Date: *Sunday, May 17, 2020 at 10:15 PM
> *To: *Bruno Rijsman <[e-mail]>
> *Cc: *Antoni Przygienda <[e-mail]>, "[e-mail]" <[e-mail]>,
> Tony Przygienda <[e-mail]>
> *Subject: *Re: [Rift] Negative disaggregation feature guide
>
> *[External Email. Be cautious of content]*
>
> This is quite interesting, Bruno.
>
>
> Does that mean that there are issues running negative in stand alone? I’d
> be really interested if we have a monthly or something one of these days.
> Also there’s the section on disaggregation in the applicability draft that
> we may enrich
> Many thanks for all your efforts!
>
> Pascal
>
>
> Le 18 mai 2020 à 01:02, Bruno Rijsman <[e-mail]> a écrit :
>
> Hi Tony,
>
> I ran into various complications and a road-block while working on the
> “disaggregation negative-only” knob in RIFT-python, so I will be ripping
> that knob out of my code.
>
> Instead I will stick to the three commandments as handed down on the stone
> tablet by the wise RIFT gods from above :-)
>
>
> On May 14, 2020, at 12:26 PM, Antoni Przygienda <
> prz=[e-mail]> wrote:
>
> In a probably more detailed way with a little stone tablet I suggest 😉
>
>
>    1. You SHALL negatively disaggregate only if you’re ToF and have
>    horizontal links (ring)
>    2. You SHALL propagate transitively negative disaggregation
>    3. You SHALL use positive if you’re not ToF or have no horizontal
>
>
> Of course, any vendor, even open sourced one can add any flavor of knobs
> that makes their dish unique even if it breaks the spec strictly speaking …
> So did I 😉
>
> --- tony
>
> *From: *Tony Przygienda <[e-mail]>
> *Date: *Thursday, May 14, 2020 at 11:12 AM
> *To: *Bruno Rijsman <[e-mail]>
> *Cc: *"Pascal Thubert (pthubert)" <[e-mail]>, "[e-mail]" <
> [e-mail]>, Melchior Aelmans <[e-mail]>, Christian Graf <
> [e-mail]>, Oliver Steudler <[e-mail]>, Olivier
> Vandezande <[e-mail]>, Antoni Przygienda <[e-mail]>,
> "[e-mail]" <[e-mail]>, Jeff Tantsura <
> [e-mail]>, Zhaohui Zhang <[e-mail]>, Zhaohui
> Zhang <[e-mail]>, Jeffrey Zhang <[e-mail]>
> *Subject: *Re: Negative disaggregation feature guide
>
> *[External Email. Be cautious of content]*
>
>  my thoughts:
>
> negative is seriously more complex to  implement and understand
> operationally and only needed on multi-plane fabrics, taht's why positive
> is KISS ;-) Also, negative always forces you to ring the top of the fabric
> which in single plane design is an unnecessary requirement.
>
> I foresee vendors not implementing negative for simplicity, very small
> footprint fabrics.
>
> the discussion about capability advertisement is interesting, we can just
> add one bit on the node capabilities element, make it optional and default
> true.
>
> Of course an implementation taht understands that all involved nodes
> understand negative is free to use negative instead of positive then.
>
> -- tony
>
> On Thu, May 14, 2020 at 11:05 AM Bruno Rijsman <[e-mail]>
> wrote:
>
> Broadening a discussion on negative disaggregation to the RIFT mailing
> list.
>
> Now that we have implemented negative disaggregation in RIFT-Python, and
> now that we are getting some operational experience with it, the following
> is becoming more and more evident:
>
> It seems to us (Bruno and Pascal, for now) that once negative
> disaggregation is implemented, there is really no need for positive
> disaggregation any more.
>
> Every use case that can be solved by positive disaggregation can (as far
> as we can currently tell) also be solved by negative disaggregation.
>
> Furthermore, negative disaggregation solves the problems in a far simpler
> and more elegant way: there are fewer advertising nodes and fewer
> advertised TIEs involved, and we don’t have the synchronization issue that
> positive disaggregation has that potentially causes incast problems.
>
> Thus, at the very least, it makes sense to recommend that in a given
> fabric we use either only negative disaggregation only or positive
> disaggregation only, with negative disaggregation being the default.
>
> If positive and negative disaggregation are enabled simultaneously in one
> and the same fabric, everything still works fine as far as we can see. But
> there are some “interesting” interactions that make things unnecessarily
> complex and potentially fragile.
>
> If support for negative disaggregation is mandatory, and if our assessment
> that negative disaggregation can solve all use cases is correct, then we
> could go one bold step further and completely remove positive
> disaggregation from the specification.
>
> Thoughts?
>
> — Bruno
>
>
>
> On May 14, 2020, at 10:48 AM, Pascal Thubert (pthubert) <
> [e-mail]> wrote:
>
> Hello Bruno:
>
> I like your negative-only default because it is probably the safest. Let
> me elaborate..
>
> partitioning saves a lot more ports per ToF node than the ring cost so it
> does not matter.
>
> If you enable negative, there’s no point in doing any positive at all, is
> there? So what do we try to save by combining? In your example, I guess
> that super 1 2 had to retract the positive disag and then do negative
> instead when the link to spine 1 1 fell. Was that not a bit complex and
> error prone?
>
> Also I’m not clear how we can always decide that dynamically and I do not
> believe that we specified that. So I understand that it’s more like a use
> case thingy, like a configuration that would be adapted to the use case.
>
> => do not configure both at the same time use either positive or negative
> disag.
>
> We do not try to do positive disag transitively. So if there is a need for
> transitive, you have to use negative.  In a multiplane case, there’s
> usually (as in your picture) one ToP per plane in each PoD, and a first
> link failure (say Spine 1 1 to leaf 1 1) can already cause a fallen leaf:
> leaf 1 1 is no more reachable within plane 1. This tells you that all
> leaves in the other PoDs must avoid that plane, and that’s a leaf decision;
> which tells you that you need to recurse transitively down. This indicates
> that negative must be activated in a highly partitioned (meaning low
> redundancy) multiplane.
>
> Say we try to use positive disaggregation transitively in a ToF that is
> partitioned like in planes, you’d run a risk that none of the ToF nodes
> that can still reach the leaf can see (through south reflection) the ToF
> node that cannot so they do not know they need to disag. So you end up
> needing the same ring as for negative, the southern reflection become
> mostly useless and the benefits of positive are gone.
>
> Partitionned ToF => negative
>
> I trust that we can use positive in a highly redundant and symmetrical
> non-partitioned ToF. If no one can reach the leaf anymore (like it’s dead)
> we have no issue since it should not be disaggregated, just use the
> discards route at any ToF node. If you have enough redundancy and no
> partition,  there will always be nodes that can both reach a leaf and
> discover that a peer does not.
>
> => Positive is indicated for a very specific use case, ideally 2 levels
> and fully meshed. Maybe that’s enough deployments to justify the feature in
> the code.
>
> So the question is really that blurry zone between highly redundant single
> plane and highly partitioned multiplane what should one do? I becomes a
> risk/benefit judgement… If negative is implemented and the ring is there,
> I’d use it, better safe than sorry.
>
> Take care
>
> Pascal
>
>
>
> *From:* Bruno Rijsman <[e-mail]>
> *Sent:* jeudi 14 mai 2020 17:36
> *To:* Pascal Thubert (pthubert) <[e-mail]>
> *Cc:* Melchior Aelmans <[e-mail]>; Christian Graf <
> [e-mail]>; Oliver Steudler <[e-mail]>; Olivier
> Vandezande <[e-mail]>; Tony Przygienda <[e-mail]>;
> Antoni Przygienda <prz=[e-mail]>;
> [e-mail]; Jeff Tantsura <[e-mail]>;
> Jeffrey (Zhaohui) Zhang <[e-mail]>; Jeffrey (Zhaohui) Zhang <
> zzhang=[e-mail] <zzhang=40juniper.net@dmarc..ietf.org>>;
> Jeffrey Zhang <[e-mail]>
> *Subject:* Re: Negative disaggregation feature guide
>
>
>
> On May 14, 2020, at 8:53 AM, Pascal Thubert (pthubert) <[e-mail]>
> wrote:
>
> This does not show because on your first breakage you use the positive
> disag. If you stick to the logic you used at the beginning, that is pick an
> example where positive applies but use negative, then on your first
> breakage you’d show that the negative does not need to go transitively to
> the leaf, because the spine nodes still have solutions. Is that a lot of
> work to change?
>
>
> This is exactly what I was thinking as well.
>
> Maybe I need to go through the current specification with a fine-toothed
> comb again, but my (possibly incorrect) understanding is that the current
> version of the specification is that we positive disaggregation is used
> (MUST be used? SHOULD be used?) in the “first failure” scenario.
>
> Personally, I see no reason why we would not be able to use negative
> disaggregation instead of positive disaggregation in the first failure
> scenario as well.
>
> In the context of the tutorial, this would have the advantage of clearly
> demonstrating the propagation logic more clearly.
>
> But more importantly, it seems to me that once negative disaggregation is
> implemented, there is really no reason to use positive disaggregation
> anywhere. It would be “cleaner” (less signaling) and “better” (no incast)
> to simply use negative disaggregation everywhere.
>
> So…. I was thinking to add a configuration knob to my code
> “positive-only”, “negative-only”, “positive-and-negative”, with
> “negative-only” being the default? (Seems like a waste to have coded up the
> positive disaggregation).
>
> What does everyone think of that?
>
> Should we go even further and simply remove positive disaggregation from
> the spec?
>
> Should we have negative disaggregation as a capability in the capability
> negotiation? (Seems non-sensical, given it is something that the whole
> fabric needs to support or not.)
>
> Should we take this to the RIFT mailing list?
>
> — Bruno
>
>
>
> _______________________________________________
> RIFT mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/rift
> <https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Frift__%3B!!NEt6yMaO-gk!TmsyrEr6P-LonGpbG3vC0JFQ_i21Ign6Y6BE9wvWBuAioJWe1MkV9Q-fUBH1Rg%24&data=02%7C01%7C%7C4db24006111d4624c40708d7faf4bcef%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637253801557732691&sdata=R8MLL0Q8aTmhr4xq%2BGTTTf2IgKLWcDoBKQSz2APcGsI%3D&reserved=0>
>
>
> _______________________________________________
> RIFT mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/rift
>
>
> Juniper Business Use Only
>
>
>