Re: [Anima] recursive system dependencies (Was: Re: New Version Notification for draft-trossen-rtgwg-impact-of-dlts-00.txt)

[Jon Crowcroft <[e-mail]>]

the cambridge distributed system was an integrated programmable architecture where routers
(called ring-ring bridges), nameservers, bootservers, fileservers, were all treated in a
coherent sand consistent fashion - no-one thought there was anything odd about this - we even
had an internet protocol (the universe data gram protocol) which allowed interworking with non
cambridge networks (indeed, we connected several universities in the UK with satellite and
later land lines)...

ref
Needham, Roger Michael; Herbert, Andrew J. (1983). The Cambridge Distributed Computing System. Addison Wesley.
also coverd in 
https://dl.acm.org/doi/10.1145/6041.6074


i have to admit that one upgrade in the mid 80s, one of my colleagues forgot about
dependencies, and managed to require the nameserver to allow the bootserver to boot - this was
tricky after power outages given the nameserver booted from the bootserver(s) - luckily, we
could rollback a version, reboot everything, fix the bug, and roll forward again

(this wasn't as idiotic as it might seem, since the nameserver was also an integral part of
how some (stateful) routing was done...so booting from a bootserver on a remote ring required
a path.....hmmmmi think i wrote a paper about this back then...)

i can't get to exciting by people thinking routers (and switches and NICs, and AI
accelerators) aren't just yet more computers....

however, there are specialisations (and aforementioned dependencies) which may require extreme
optimisations, which render some devices rather inappropriate for some kinds of
computations...

like running a renderfarm on a bunch of P4 programmable boxes might be a bit daft...


> On 10-Mar-22 21:52, Toerless Eckert wrote:
> 
>     [adding anima]
> 
>     One should not be surprised to see a lot of outages to be related to
>     loss of connectivity and/or control due to non-understood circular 
> dependencies.
> 
> 
> This problem is as old as distributed computing. I remember the difficulty
> of restarting the CERN computer centre after a power outage back in the
> 1980s, when nobody knew the correct restart sequence and the dependencies
> turned out to be different each time. While having great hope for
> autonomic networking, I don't think we have a magic solution for this.
> See draft-ciavaglia-anima-coordination from 2016, and add in the case of a
> general outage. Probably ANIMA needs to restart work on that topic.
> 
>    Brian
> 
> 
>     This is certainly true for several of the big outages reported, such 
> as
>     some complex mutual automation control failure in Google a few years 
> back,
>     and then the typical "routing broke, and we couldn't remotely access 
> the
>     node we needed to fix... because routing broke". That issue of course
>     is resolveable by ACP/RFC8994 and lists a documented example of that 
> issue
>     as investigated by FCC because the network was running 911 services 
> and
>     was thus subject to regulation. When big OTT networks have outages, 
> you
>     will of course almost never see a good root cause analysis publically 
> because
>     there is no regulation that forces them to. Maybe a bit like how the 
> banking
>     sector i think also has managed to keep a lot of its security issues 
> under wraps.
> 
>     We do also have in ANIMA participants interested in having those 
> dependency
>     chains modelled, which would allow to discover loops that could 
> result in blockage,
>     and to manage service instantiation in the order necessary to resolve 
> dependencies
>     when they are resolvable. However:
> 
>     I always look with marvel and dismay at those dependency resolution 
> systems
>     found in linux distributions to (compile and) install packages. And 
> that sounds
>     so simple that one should expect that it should not result in 
> dependency resolution
>     failures when there are no actual circular ones, but in practice 
> those issues occur
>     repeatedly, and one has to take really unnecessary brute-force 
> resolution
>     measures even though in detailled analysis one could see a limited 
> dependency
>     resolution chain, which the platform package management system just 
> couldn't figure
>     out. Admittedly, this is more of a problem with more flexible systems 
> such as
>     gentoo where you can have multiple versions of packages and also 
> differently
>     parameterized packages(through parameterized compilation). But those 
> are exactly
>     the minimum complexity requirements i would expect in an actual 
> network services
>     (ever seen a netork service without per-deployment parameters ?).
>     So i fear that the more we're automating networks with SDN, the more 
> we will run
>     in the buildup of system dependencies that can only be resolved 
> automatically with
>       "please reboot the continent wide network... now!".
> 
>     Or else we could go back building simpler networks. Maybe bottom up 
> like we
>     used to do instead of orchestrating a random complexity zoo like we 
> do like to do
>     now in the times of automation...
> 
>     Cheers
>          Toerless
> 
>     On Thu, Mar 10, 2022 at 06:49:07AM +0000, Dirk Trossen wrote:
> 
>         Have you seen anything like this and what do you think are the 
> consequences?
>         [DOT] What the consequences of this may be is difficult to judge 
> IMO. Maybe an SDN-like situation, i.e., still having an IP-based control 
> overlay which then manages the remaining IP service plane? Another 
> possible direction to look into is using ANIMA concepts to bootstrap (and 
> maintain) a DLT-like control overlay. But this is also why we are asking 
> the community for possible network innovations that may help improve on 
> the efficacy of DLTs. One approach we looked into, as an example but this 
> still needs more work, is 'L3-based service routing' as a form of 
> semantic routing (hence the hint to the semantic routing discussions in 
> the draft). Here, we interpret the DLT as a service provided across the 
> network. I could see this being provided as a baseline/bootstrap service 
> through which to manage the (service routing) data that is being enriched 
> over time by other services. But again, we were hoping to find more folks 
> in the community who have thought deeper along those lines; we are
> 
>       ju
> 
>           st at the start of doing so.
> 
>         Thanks,
>         Adrian
> 
>         -----Original Message-----
>         From: rtgwg <[e-mail]> On Behalf Of Dirk Trossen
>         Sent: 02 March 2022 13:16
>         To: Dirk Trossen <dirk.trossen=[e-mail]>; 
> [e-mail]; [e-mail]
>         Subject: RE: New Version Notification for 
> draft-trossen-rtgwg-impact-of-dlts-00.txt
> 
>         All,
> 
>         We have posted an updated to the draft below at 
> https://datatracker.ietf.org/doc/draft-trossen-rtgwg-impact-of-dlts/.
> 
>         We now welcome Mike McBride and Xinxin Fan as additional 
> co-authors and have also added language into the introduction to clarify 
> our initial insights being limited to PoW -based DLT systems, which lead 
> to some characteristic communication patterns and associated 
> inefficiencies compared to, say, PoS-based systems.
> 
>         Please provide any comments you may have on the draft and/or its 
> insights, including any interests in contributing in its expansion (e.g., 
> to other DLT systems).
> 
>         Best,
> 
>         Dirk
> 
>         -----Original Message-----
>         From: routing-discussion 
> [mailto:[e-mail]] On Behalf Of Dirk Trossen
>         Sent: 14 February 2022 16:16
>         To: [e-mail]; [e-mail]
>         Subject: FW: New Version Notification for 
> draft-trossen-rtgwg-impact-of-dlts-00.txt
> 
>         All,
> 
>         We posted the draft below to continue a piece of work initiated 
> in the Industry IoT Consortium (IIC) on understanding the "Impact of DLTs 
> on provider networks", leading to the whitepaper at 
> https://www.iiconsortium.org/pdf/2022-01-10-Impact-of-Distributed-Ledgers-on
>         -Provider-Networks.pdf
> 
>         With this draft, we solicit feedback from the wider IETF 
> community on our insights regarding DLTs with the desire to broaden our 
> findings with the expertise we can find here but also to capture possible 
> network and routing innovations that may improve on the impacts we have 
> identified.
> 
>         If you have any comments or would like to contribute to this 
> work, please do let us know, either on the list or directly to the 
> authors.
> 
>         Best,
> 
>         Dirk (on behalf of the authors)
> 
>         -----Original Message-----
>         From: [e-mail] [mailto:[e-mail]]
>         Sent: 14 February 2022 16:10
>         To: David Guzman <[e-mail]>; Dirk Trossen 
> <[e-mail]>
>         Subject: New Version Notification for
>         draft-trossen-rtgwg-impact-of-dlts-00.txt
> 
> 
>         A new version of I-D, draft-trossen-rtgwg-impact-of-dlts-00.txt
>         has been successfully submitted by Dirk Trossen and posted to the 
> IETF
>         repository.
> 
>         Name:           draft-trossen-rtgwg-impact-of-dlts
>         Revision:       00
>         Title:          Impact of DLTs on Provider Networks
>         Document date:  2022-02-14
>         Group:          Individual Submission
>         Pages:          16
>         URL:
>         
> https://www.ietf.org/archive/id/draft-trossen-rtgwg-impact-of-dlts-00.txt
>         Status:
>         
> https://datatracker.ietf.org/doc/draft-trossen-rtgwg-impact-of-dlts/
>         Htmlized:
>         
> https://datatracker.ietf.org/doc/html/draft-trossen-rtgwg-impact-of-dlts
> 
> 
>         Abstract:
>             This document discusses the impact of distributed ledger 
> technologies
>             being realized over IP-based provider networks.  The focus 
> here lies
>             on the impact that the DLT communication patterns have on 
> efficiency
>             of resource usage in the underlying networks.  We provide 
> initial
>             insights into experimental results to quantify this impact in 
> terms
>             of inefficient and wasted communication, aligned along 
> challenges
>             that the DLT realization over IP networks faces.
> 
>             This document is intended to outline this impact but also
>             opportunities for network innovations to improve on the 
> identified
>             impact as well as the overall service quality.  While this 
> document
>             does not promote specific solutions that capture those 
> opportunities,
>             it invites the wider community working on DLT and network 
> solutions
>             alike to contribute to the insights in this document to aid 
> future
>             research and development into possible solution concepts and
>             technologies.
> 
>             The findings presented here have first been reported within 
> the
>             similarly titled whitepaper released by the Industry IoT 
> Consortium
>             [IIC_whitepaper].
> 
> 
> 
> 
>         The IETF Secretariat
> 
> 
>         _______________________________________________
>         routing-discussion mailing list
>         [e-mail]
>         https://www.ietf.org/mailman/listinfo/routing-discussion
> 
>         _______________________________________________
>         rtgwg mailing list
>         [e-mail]
>         https://www.ietf.org/mailman/listinfo/rtgwg
> 
>         _______________________________________________
>         rtgwg mailing list
>         [e-mail]
>         https://www.ietf.org/mailman/listinfo/rtgwg
> 
> 
> 
> _______________________________________________
> routing-discussion mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/routing-discussion
>