Re: [Anima] Object store and PubSub model

[Alexander Clemm <[e-mail]>]

I can think of several use cases where such a capability would be useful.  

One such use case is documented in https://tools.ietf.org/html/draft-irtf-nmrg-autonomic-sla-violation-detection-10.  This concerns a use case to coordinate service level measurements among nodes in a network.  To come to better autonomic decisions, nodes need more information, specifically data of observations (in that case) made by other nodes. I would expect this to be a very common design pattern for many distributed autonomic applications, running in Autonomic Services Agents that are not "isolated" to their containing node, but that share information in order to each individually be able to make better decisions.  

Having pub-sub infrastructure will facilitate the development of such applications.  

--- Alex

-----Original Message-----
From: Anima [mailto:[e-mail]] On Behalf Of Toerless Eckert
Sent: Wednesday, July 05, 2017 4:48 PM
To: Artur Hecker <[e-mail]>
Cc: [e-mail]
Subject: Re: [Anima] Object store and PubSub model

Artur: pubsub and object store seem to be (to me) two hot buzzwords, but i am not aware (ENOCYCLES) what the IETF is currently doing about those. It would certainly be interesting for the ANIMA-WG to see proposals of work in the area with good motivation why this work would be particularily be suited to ANIMA. [this does not mean that we'd have time in the WG slot in Prague though, Sheng is managing the time there ;-)]

I think its overall a very wide field, and i'd primarily look for precise use case examples and specific justifications why something is best done in ANIMA. 

Example:

a) Justification: existing requirement in ANIMA, pubsub/object-store to the rescue:

   We need in ANIMA intent distribution, that would be a use-case. If there is anything
   existing from IETF or other bodies that would allow to do this, (whether or not
   that is some pubsub bus or object store), then that would be highly worth to bring up
   in ANIMA. Main issue is that we have no clear agreement on what intent is. IMHO it
   could be along what the current (not WG item) intent draft says, but maybe we need to
   even call it differently (not intent). See of course also grasp-information-distribution
   (which is more a cateogrization of requirements than a solution).

b) Justification: build it autonomously:

   There are a lot of pubsub buses today, but i am not aware of any bus where the distribution
   infra of those buses is autonomously set up. Except when they simple flooding buseses.
   All the other buses i know seem to have some manually provisioned infra like brokers or
   the like. So IMHO any proposal to autonmously create a bus that is better than
   flooding would IMHO a great idea for ANIMA.
   
   But i am also not sure if there is a realistic option to autonomously create a good
   distributed bus infra. In between "everything is centralized" and "flooding&storage
   everywhere" there are so many aspects of optimiation that its IMHO almost impossible
   to figure out a generic solution that almost automatically adopts itself to the best
   shape pending on a particular use-case. And i am saying this with just boring
   information distribution background called multicast. Not to mention CCN/ICN.

Cheers
    Toerless

On Thu, Jun 29, 2017 at 03:28:08PM +0000, Artur Hecker wrote:
> Dear Brian, all
> 
> 
> Thanks a lot for the explanations. Please find my answers below:
> 
> -----Original Message-----
> From: Brian E Carpenter [mailto:[e-mail]]
> Sent: 28 June 2017 22:35
> To: Artur Hecker; [e-mail]
> Subject: Re: [Anima] Object store and PubSub model
> 
> > So far, I don't believe we've discussed distributed storage as such. 
> > It would be a natural part of the discussion of intent. We have 
> > tended to assume that intent originates centrally, so it needs a 
> > replication model rather than distributed storage in the general 
> > sense. The simplest replication model is of course flooding, which we already have for relatively small data objects. For example, GRASP as it exists today could flood an objective whose value is the URL of the latest intent for the whole network.
> 
> [[AH] ] Ok, very clear, thanks. Would you welcome such a discussion e.g. for the re-chartering of ANIMA?
> 
> We believe that network-wide storage could be of high value for autonomous networking, just as routing is. The reasons for this are, among others:
> 
> c) given the existing ANIMA descriptions, I believe that limited 
> storage is presumed available on all nodes (i.e. at least to store 
> that URL in your example);
> b) the APIs for data object storage are well-established and compact 
> (put(), get());
> a) the information a node might need to store is not necessarily limited to the scope of that single node. Therefore, an efficient capability to access network-wide information (e.g. configuration data, network-wide intents, network-wide averages or aggregates of data) would really be of a value. If every application has to redo it, we run the risk of repetitive wheel reinventions - or flooding. Essentially, as you just hinted, if we do not provide these means (either by a mandatory ASA or over the ANIMA API), then every developer will have to revert to flooding at least for the first message. While this might be assumed rare for intents, I am not sure we can claim this for every individual application scenario.
> 
> 
> > I would not advocate developing a distributed storage model specific 
> > to Anima. Rather, if we decided it was
>  > necessary, IMHO we should adopt an existing solution, since this is not a trivial problem.
> 
> [[AH] ] Agreed, and it was not intended. If there is a perceived consensus of the value of this, we would rather see how to integrate/adopt a suitable solution.
> 
>  
> >> 2. In a similar vein, does a publish/subscribe information 
> >> distribution model, built upon an underlying storage, look 
> >> promising and relevant to the group? Can GRASP support that or will 
> >> it require extensions? The pub/sub is, in our opinion, a more general information distribution model than what we saw in the current work stream. Yet, before submitting a draft that details such a model, we wanted to kindly ask for your opinion on that.
> 
> > Have you studied draft-liu-anima-grasp-distribution?
> [[AH] ] Yes, that was where our motivation came from. It seems to be based on the flooding/replication model. Which is fine for some scenarios, but which also has known limitations. The idea was to allow for a more general model, considered to be better for scalability, etc.
> 
> > But first, what sort of autonomic use cases would require distributed storage or a pub/sub solution?
> 
> [[AH] ] Any example under a) above or our original example of ECA type of intent, which would require to "ripen" before being executed. The exact condition could be something that an individual node might not know, e.g. "close the blinds when the temperature in the building exceeds 25 C (=77F)".
> 
> 
> 
> Regards
> artur
> 
> _______________________________________________
> Anima mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/anima

--
---
[e-mail]

_______________________________________________
Anima mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/anima