[Anima] (shepherd) review of draft-carpenter-anima-asa-guidelines-09

[Toerless Eckert <[e-mail]>]

Dear draft authors:

Thanks for this work. Please find below an initial shepherd review of the
draft. I have no technical concerns with the document bcause i think this is
a very explorative area where this document is a great first stake in the ground.

All comments/suggestions therefore are either

a) text quality
b) Marketing (text could be more positive about how easy this is and at
   best equally complex as other pre-existing and less autonomic solutions)

Cheers
    Toerless

> Network Working Group                                    B. E. Carpenter
> Internet-Draft                                         Univ. of Auckland
> Intended status: Informational                              L. Ciavaglia
> Expires: 26 January 2021                                           Nokia
>                                                                 S. Jiang
>                                             Huawei Technologies Co., Ltd
>                                                                P. Peloso
>                                                                    Nokia
>                                                             25 July 2020
> 
> 
>                 Guidelines for Autonomic Service Agents
>                 draft-carpenter-anima-asa-guidelines-09
> 
> Abstract
> 
>    This document proposes guidelines for the design of Autonomic Service
>    Agents for autonomic networks, as a contribution to describing an
>    autonomic ecosystem.  It is based on the Autonomic Network
>    Infrastructure outlined in the ANIMA reference model, using the
>    Autonomic Control Plane and the Generic Autonomic Signaling Protocol.
> 
> Status of This Memo
> 
>    This Internet-Draft is submitted in full conformance with the
>    provisions of BCP 78 and BCP 79.
> 
>    Internet-Drafts are working documents of the Internet Engineering
>    Task Force (IETF).  Note that other groups may also distribute
>    working documents as Internet-Drafts.  The list of current Internet-
>    Drafts is at https://datatracker.ietf.org/drafts/current/.
> 
>    Internet-Drafts are draft documents valid for a maximum of six months
>    and may be updated, replaced, or obsoleted by other documents at any
>    time.  It is inappropriate to use Internet-Drafts as reference
>    material or to cite them other than as "work in progress."
> 
>    This Internet-Draft will expire on 26 January 2021.
> 
> Copyright Notice
> 
>    Copyright (c) 2020 IETF Trust and the persons identified as the
>    document authors.  All rights reserved.
> 
>    This document is subject to BCP 78 and the IETF Trust's Legal
>    Provisions Relating to IETF Documents (https://trustee.ietf.org/
>    license-info) in effect on the date of publication of this document.
>    Please review these documents carefully, as they describe your rights
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 1]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    and restrictions with respect to this document.  Code Components
>    extracted from this document must include Simplified BSD License text
>    as described in Section 4.e of the Trust Legal Provisions and are
>    provided without warranty as described in the Simplified BSD License.
> 
> Table of Contents
> 
>    1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
>    2.  Logical Structure of an Autonomic Service Agent . . . . . . .   4
>    3.  Interaction with the Autonomic Networking Infrastructure  . .   5
>      3.1.  Interaction with the security mechanisms  . . . . . . . .   5
>      3.2.  Interaction with the Autonomic Control Plane  . . . . . .   5
>      3.3.  Interaction with GRASP and its API  . . . . . . . . . . .   6
>      3.4.  Interaction with policy mechanism . . . . . . . . . . . .   7
>    4.  Interaction with Non-Autonomic Components . . . . . . . . . .   7
>    5.  Design of GRASP Objectives  . . . . . . . . . . . . . . . . .   8
>    6.  Life Cycle  . . . . . . . . . . . . . . . . . . . . . . . . .   9
>      6.1.  Installation phase  . . . . . . . . . . . . . . . . . . .   9
>        6.1.1.  Installation phase inputs and outputs . . . . . . . .  10
>      6.2.  Instantiation phase . . . . . . . . . . . . . . . . . . .  11
>        6.2.1.  Operator's goal . . . . . . . . . . . . . . . . . . .  11
>        6.2.2.  Instantiation phase inputs and outputs  . . . . . . .  12
>        6.2.3.  Instantiation phase requirements  . . . . . . . . . .  12
>      6.3.  Operation phase . . . . . . . . . . . . . . . . . . . . .  13
>    7.  Coordination between Autonomic Functions  . . . . . . . . . .  14
>    8.  Coordination with Traditional Management Functions  . . . . .  14
>    9.  Robustness  . . . . . . . . . . . . . . . . . . . . . . . . .  14
>    10. Security Considerations . . . . . . . . . . . . . . . . . . .  15
>    11. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  16
>    12. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  16
>    13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  16
>      13.1.  Normative References . . . . . . . . . . . . . . . . . .  16
>      13.2.  Informative References . . . . . . . . . . . . . . . . .  17
>    Appendix A.  Change log [RFC Editor: Please remove] . . . . . . .  19
>    Appendix B.  Example Logic Flows  . . . . . . . . . . . . . . . .  20
>    Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  25
> 
> 1.  Introduction
> 
>    This document proposes guidelines for the design of Autonomic Service
>    Agents (ASAs) in the context of an Autonomic Network (AN) based on
>    the Autonomic Network Infrastructure (ANI) outlined in the ANIMA
>    reference model [I-D.ietf-anima-reference-model].  This
>    infrastructure makes use of the Autonomic Control Plane (ACP)
>    [I-D.ietf-anima-autonomic-control-plane] and the Generic Autonomic
>    Signaling Protocol (GRASP) [I-D.ietf-anima-grasp].  This document is
>    a contribution to the description of an autonomic ecosystem,
>    recognizing that a deployable autonomic network needs more than just
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 2]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    ACP and GRASP implementations.  It must achieve management goals that
>    a Network Operations Center (NOC) cannot achieve manually, including

"cannot achieve" == must have is always a great way to upsell ASA,
but cost reduction or increasement of reliability (faster or easier
automated recovery), aka: improvement of operational metrics is also IMHO
significant and might be worth to mention here. Also maybe easier understood/
accepted without going into examples here in the intro.

>    at least a library of ASAs and corresponding GRASP objective
>    definitions.  There must also be tools to deploy and oversee ASAs,
>    and integration with existing operational mechanisms [RFC8368].
>    However, this document focuses on the design of ASAs, with some
>    reference to implementation and operational aspects.
> 
>    There is a considerable literature about autonomic agents with a
>    variety of proposals about how they should be characterized.  Some
>    examples are [DeMola06], [Huebscher08], [Movahedi12] and [GANA13].
>    However, for the present document, the basic definitions and goals
>    for autonomic networking given in [RFC7575] apply . According to RFC
>    7575, an Autonomic Service Agent is "An agent implemented on an
>    autonomic node that implements an autonomic function, either in part
>    (in the case of a distributed function) or whole."
> 
>    ASAs must be distinguished from other forms of software component.
>    They are components of network or service management; they do not in
>    themselves provide services.  For example, the services envisaged for
>    network function virtualisation [RFC8568] or for service function
>    chaining [RFC7665] might be managed by an ASA rather than by
>    traditional configuration tools.

I think a lot of operators will have a hard time figuring out what a practical
instance of an ASA could be. Maybe it would help to give here or later in the
document a reasonable example, at least this is always what i think of:

AFAIK, all router vendors have some for of automation scripting language
that allow to write scripts running locally on routers. For example if
you look up vendor C, H and J, you will find they have Tcl automation
scripting. I never know where to find vendor N documentation. Maybe under
A or T (Laurent i hope would know).

In any case, the way how i would explain an ASA is simply this:

Consider you could write Tcl scripts that can noy only locally monitor/configure
router functions/services, but that can also easily communicate with peer
scripts on neighboring or remote routers because the ACP will already
build a secure control connection even before any other config is performed
on the router, and it provides via GRASP local and remote peer discovery
and standardized, easily extensible negotiation mechanisms for these
scripts to coordinate with each other.

Maybe such an example explanation would help to make these ASA we want to build
look a lot more short-term relevant than what many casual observers of ANIMA
may think - given our new and not well operator understood terminology.

>    The reference model [I-D.ietf-anima-reference-model] expands this by
>    adding that an ASA is "a process that makes use of the features
>    provided by the ANI to achieve its own goals, usually including
>    interaction with other ASAs via the GRASP protocol
>    [I-D.ietf-anima-grasp] or otherwise.  Of course it also interacts
>    with the specific targets of its function, using any suitable
>    mechanism.  Unless its function is very simple, the ASA will need to
>    handle overlapping asynchronous operations.  This will require either
>    a multi-threaded implementation, or a logically equivalent event loop
>    structure.  It may therefore be a quite complex piece of software in
>    its own right, forming part of the application layer above the ANI."
> 
>    There will certainly be very simple ASAs that manage a single
>    objective in a straightforward way and do not need asynchronous
>    operations.  In such a case, many aspects of the current document do
>    not apply.  However, in general a basic property of an ASA is that it
>    is a relatively complex software component that will in many cases
>    control and monitor simpler entities in the same host or elsewhere.
>    For example, a device controller that manages tens or hundreds of
>    simple devices might contain a single ASA.

Sorry, i know i will sound even more like a cheap? ASA marketing person
than above notion of ASA = tcl-script+ACP(GRASP), but let me contend that
there will likely be a lot of really useful ASA that are really simple.
So i would love if the text does not scare off possible short term
adopters just with language.

One typical type of example i like to use is:

How do you automatically configure and secure your routing protocol across
a domain ? Well, with ACP+ASA you can have a simple agent that finds
neighboring routers from the same domain on each LAN via GRASP on the
ACP, then configures the IGP on that interface and even negotiates an IGP
security key across the GRASP connection to the neighbor.

>    The remainder of this document offers guidance on the design of such
>    ASAs.
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 3]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 2.  Logical Structure of an Autonomic Service Agent
> 
>    As mentioned above, all but the simplest ASAs will need to suport
>    asynchronous operations.  Not all programming environments explicitly
>    support multi-threading.  In that case, an 'event loop' style of
>    implementation should be adopted, in which case each thread would be
>    implemented as an event handler called in turn by the main loop.  For
>    this, the GRASP API (Section 3.3) must provide non-blocking calls.
>    If necessary, the GRASP session identifier will be used to
>    distinguish simultaneous operations.
> 
>    A typical ASA will have a main thread that performs various initial
>    housekeeping actions such as:
> 
>    *  Obtain authorization credentials.
> 
>    *  Register the ASA with GRASP.
> 
>    *  Acquire relevant policy parameters.
> 
>    *  Define data structures for relevant GRASP objectives.
> 
>    *  Register with GRASP those objectives that it will actively manage.
> 
>    *  Launch a self-monitoring thread.
> 
>    *  Enter its main loop.
> 
>    The logic of the main loop will depend on the details of the
>    autonomic function concerned.  Whenever asynchronous operations are
>    required, extra threads will be launched, or events added to the
>    event loop.  Examples include:
> 
>    *  Repeatedly flood an objective to the AN, so that any ASA can
>       receive the objective's latest value.
> 
>    *  Accept incoming synchronization requests for an objective managed
>       by this ASA.
> 
>    *  Accept incoming negotiation requests for an objective managed by
>       this ASA, and then conduct the resulting negotiation with the
>       counterpart ASA.
> 
>    *  Manage subsidiary non-autonomic devices directly.
> 
>    These threads or events should all either exit after their job is
>    done, or enter a wait state for new work, to avoid blocking others
>    unnecessarily.
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 4]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    According to the degree of parallelism needed by the application,
>    some of these threads or events might be launched in multiple
>    instances.  In particular, if negotiation sessions with other ASAs
>    are expected to be long or to involve wait states, the ASA designer
>    might allow for multiple simultaneous negotiating threads, with
>    appropriate use of queues and locks to maintain consistency.
> 
>    The main loop itself could act as the initiator of synchronization
>    requests or negotiation requests, when the ASA needs data or
>    resources from other ASAs.  In particular, the main loop should watch
>    for changes in policy parameters that affect its operation.  It
>    should also do whatever is required to avoid unnecessary resource
>    consumption, such as including an arbitrary wait time in each cycle
>    of the main loop.
> 
>    The self-monitoring thread is of considerable importance.  Autonomic
>    service agents must never fail.  To a large extent this depends on
>    careful coding and testing, with no unhandled error returns or
>    exceptions, but if there is nevertheless some sort of failure, the
>    self-monitoring thread should detect it, fix it if possible, and in
>    the worst case restart the entire ASA.

This is a great overview of the functional aspects an ASA can consider.
If anything, am worried about the "should" used in the text, e.g.: for event-loop,
because there is also the whole other religion of callback driven
parallelism (and there may be other religions too.. i am not religious).

You don't need to change anything for me, but if you are
worried about future feedback from other similar feedback, then i would
suggest to rephrase to "nonwithstanding other options, an ASA may use (..an
event loop)", describe that the text outlines one known to work approach
instead of making (strict) recommendation.

>    Appendix B presents some example logic flows in informal pseudocode.
> 
> 3.  Interaction with the Autonomic Networking Infrastructure
> 
> 3.1.  Interaction with the security mechanisms
> 
>    An ASA by definition runs in an autonomic node.  Before any normal
>    ASAs are started, such nodes must be bootstrapped into the autonomic
>    network's secure key infrastructure in accordance with
>    [I-D.ietf-anima-bootstrapping-keyinfra].  This key infrastructure
>    will be used to secure the ACP (next section) and may be used by ASAs
>    to set up additional secure interactions with their peers, if needed.

At the risk of BRSKI authors killing me ;-), but again with my "ASA sales person
on", i would like to note that the real dependency to make ASA easy to
build and functional is ACP and GRASP over it. If you may not have BRSKI
but need to stage devices with crypto material manually, you will still
get all the ASA benefits afters. Aka: faster initial adoption is perfectly
possible with just ACP(GRASP)

>    Note that the secure bootstrap process itself may include special-
>    purpose ASAs that run in a constrained insecure mode.
> 
> 3.2.  Interaction with the Autonomic Control Plane
> 
>    In a normal autonomic network, ASAs will run as users of the ACP,

clients intead of users ? 

>    which will provide a fully secured network environment for all
>    communication with other ASAs, in most cases mediated by GRASP (next
>    section).
> 
>    Note that the ACP formation process itself may include special-
>    purpose ASAs that run in a constrained insecure mode.
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 5]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 3.3.  Interaction with GRASP and its API
> 
>    GRASP [I-D.ietf-anima-grasp] is expected to run as a separate process
>    with its API [I-D.ietf-anima-grasp-api] available in user space.
>    Thus ASAs may operate without special privilege, unless they need it
>    for other reasons.  The ASA's view of GRASP is built around GRASP
>    objectives (Section 5), defined as data structures containing
>    administrative information such as the objective's unique name, and
>    its current value.  The format and size of the value is not
>    restricted by the protocol, except that it must be possible to
>    serialise it for transmission in CBOR [RFC7049], which is no
>    restriction at all in practice.

Same comment as above "expected to" could be replaced with something
"without excluding other options, in known to work setups, GRASP can run as"..

(again, not for me, but just in case other reviewers especially IESG later
 on want to make those arguments that there are other options... e.g.:
 primarily to avoid having to add a lot more other text for other options
 unless the authors feel its valuable)
> 
>    The GRASP API should offer the following features:
> 
>    *  Registration functions, so that an ASA can register itself and the
>       objectives that it manages.
> 
>    *  A discovery function, by which an ASA can discover other ASAs
>       supporting a given objective.
> 
>    *  A negotiation request function, by which an ASA can start
>       negotiation of an objective with a counterpart ASA.  With this,
>       there is a corresponding listening function for an ASA that wishes
>       to respond to negotiation requests, and a set of functions to
>       support negotiating steps.
> 
>    *  A synchronization function, by which an ASA can request the
>       current value of an objective from a counterpart ASA.  With this,
>       there is a corresponding listening function for an ASA that wishes
>       to respond to synchronization requests.
> 
>    *  A flood function, by which an ASA can cause the current value of
>       an objective to be flooded throughout the AN so that any ASA can
>       receive it.

Some simple example explaining the difference between negotiation and
synchronization would be helpful for easier reading.

>    For further details and some additional housekeeping functions, see
>    [I-D.ietf-anima-grasp-api].
> 
>    This API is intended to support the various interactions expected
>    between most ASAs, such as the interactions outlined in Section 2.
>    However, if ASAs require additional communication between themselves,
>    they can do so using any desired protocol.

Given how we're defining ASA as the automation layer and not the underlying
service layer (which may have already protocols), i wonder what a good
example for another protocol would be. I guess if there is some pre-defined
automation protocol. A simple example would be great.

>     One option is to use
>    GRASP discovery and synchronization as a rendez-vous mechanism
>    between two ASAs, passing communication parameters such as a TCP port
>    number via GRASP.

>     As noted above, either the ACP or in special cases
>    the autonomic key infrastructure will be used to secure such
>    communications.

That sentence is difficult to understand ANIMA language. Maybe more explanatory:
when ASA want to use protocols (including GRASP) outside the ACP to other
ACP peers, they can still automatically secure such connections using the ACP
certificate, for example by using TLS. 

> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 6]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 3.4.  Interaction with policy mechanism
> 
>    At the time of writing, the policy (or "Intent") mechanism for the
>    ANI is undefined and is regarded as a research topic.  It is expected
>    to operate by an information distribution mechanism (e.g.
>    [I-D.liu-anima-grasp-distribution]) that can reach all autonomic
>    nodes, and therefore every ASA.  However, each ASA must be capable of
>    operating "out of the box" in the absence of locally defined policy,
>    so every ASA implementation must include carefully chosen default
>    values and settings for all policy parameters.

There are probably newer Intent references in NMRG now, please look up/add,
Laurent should know.
> 
> 4.  Interaction with Non-Autonomic Components
> 
>    An ASA, to have any external effects, must also interact with non-
>    autonomic components of the node where it is installed.  For example,
>    an ASA whose purpose is to manage a resource must interact with that
>    resource.  An ASA whose purpose is to manage an entity that is
>    already managed by local software must interact with that software.
>    For example, if such management is performed by NETCONF [RFC6241],
>    the ASA must interact directly with the NETCONF server in the same
>    node.

insert aragraph break

>      This is stating the obvious, and the details are specific to
>    each case, but it has an important security implication.  The ASA
>    might act as a loophole by which the managed entity could penetrate
>    the security boundary of the ANI.  The ASA must be designed to avoid
>    such loopholes, and should if possible operate in an unprivileged
>    mode.

Would suggest moving this paragrap into security consideration. First
sentence also reads a bit too colloquial (ey dude, security, you know, sure, yeah! ;-)

>    In an environment where systems are virtualized and specialized using
>    techniques such as network function virtualization or network
>    slicing, there will be a design choice whether ASAs are deployed once
>    per physical node or once per virtual context.  A related issue is
>    whether the ANI as a whole is deployed once on a physical network, or
>    whether several virtual ANIs are deployed.  This aspect needs to be
>    considered by the ASA designer.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 7]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 5.  Design of GRASP Objectives
> 
>    The general rules for the format of GRASP Objective options, their
>    names, and IANA registration are given in [I-D.ietf-anima-grasp].
>    Additionally that document discusses various general considerations
>    for the design of objectives, which are not repeated here.  However,
>    we emphasize that the GRASP protocol does not provide transactional
>    integrity.  In other words, if an ASA is capable of overlapping
>    several negotiations for a given objective, then the ASA itself must
>    use suitable locking techniques to avoid interference between these
>    negotiations.  For example, if an ASA is allocating part of a shared
>    resource to other ASAs, it needs to ensure that the same part of the
>    resource is not allocated twice.  This might impact the design of the
>    objective as well as the logic flow of the ASA.

(Sales person hat on ;-). This sounds unnecessarily GRASP bashing. Unless
someone builds a much higher layer, resource aware protocol, everything
you say equally applies to other protocols. Eg.: HTTP as it is used
billion times on the planet. The problem you describe is simply solved
at a higher layer, like with how you design REST services. Whether they
would run on top of HTTP or GRASP. Maybe some rewording to explain this
would make this sound more positive ("like HTTP, GRASP is not itself resource
aware and therefore coordination is subject to the service definition,
for example by employing REST for the service").

>    In particular, if 'dry run' mode is defined for the objective, its
>    specification, and every implementation, must consider what state
>    needs to be saved following a dry run negotiation, such that a
>    subsequent live negotiation can be expected to succeed.  It must be
>    clear how long this state is kept, and what happens if the live
>    negotiation occurs after this state is deleted.  An ASA that requests
>    a dry run negotiation must take account of the possibility that a
>    successful dry run is followed by a failed live negotiation.  Because
>    of these complexities, the dry run mechanism should only be supported
>    by objectives and ASAs where there is a significant benefit from it.

The start of this paragraph reads as if 'dry run' was defined in before
somewhere, but it is not that makes this paragraph confusing to read.
Maybe rephrase "'dry run' is one useful mode of operation that objectives
may support. If an objectives does support such a dry run, then ..."

> 
>    The actual value field of an objective is limited by the GRASP
>    protocol definition to any data structure that can be expressed in
>    Concise Binary Object Representation (CBOR) [RFC7049].  For some
>    objectives, a single data item will suffice; for example an integer,
>    a floating point number or a UTF-8 string.  For more complex cases, a
>    simple tuple structure such as [item1, item2, item3] could be used.

>    Nothing prevents using other formats such as JSON, but this requires
>    the ASA to be capable of parsing and generating JSON.

(Sales person) this sentence derails readers from a good path explaining
GRASP. If you want to mention JSON here, i would recommend to just say that
CBOR is a compacyt format derived from JSON so it inherits most of JSONs
flexible complex data structure options.

>    The formats
>    acceptable by the GRASP API will limit the options in practice.  A
>    fallback solution is for the API to accept and deliver the value
>    field in raw CBOR, with the ASA itself encoding and decoding it via a
>    CBOR library.

"...fallback solution..." sound again difficult and bad.

Why not simply write that the very basic expectation is that ASA MUST have
access to a CBOR library to en/decode the objective field ? Thats not a
job for the GRASP API. Its like ntoh/hton for decoding/encoding
the address parameters for socket library API calls. Also not part really
prt of the socket API (although its typically bundled, just like a CBOR
library would be.).

>    Note that a mapping from YANG to CBOR is defined by
>    [I-D.ietf-core-yang-cbor].  Subject to the size limit defined for
>    GRASP messages, nothing prevents objectives using YANG in this way.

... So GRASP can be used as an alternative to NETCONF ?!
(would sound like a great sales pitch if thats what you where tinking
 and would also write it out explicitly ;-)

> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 8]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 6.  Life Cycle
> 
>    Autonomic functions could be permanent, in the sense that ASAs are
>    shipped as part of a product and persist throughout the product's
>    life.  However, a more likely situation is that ASAs need to be
>    installed or updated dynamically, because of new requirements or
>    bugs.  Because continuity of service is fundamental to autonomic
>    networking, the process of seamlessly replacing a running instance of
>    an ASA with a new version needs to be part of the ASA's design.
> 
>    The implication of service continuity on the design of ASAs can be
>    illustrated along the three main phases of the ASA life-cycle, namely
>    Installation, Instantiation and Operation.
> 
> 
>                      +--------------+
>    Undeployed ------>|              |------> Undeployed
>                      |  Installed   |
>                  +-->|              |---+
>         Mandate  |   +--------------+   | Receives a
>       is revoked |   +--------------+   |  Mandate
>                  +---|              |<--+
>                      | Instantiated |
>                  +-->|              |---+
>              set |   +--------------+   | set
>             down |   +--------------+   | up
>                  +---|              |<--+
>                      |  Operational |
>                      |              |
>                      +--------------+
> 
> 
>              Figure 1: Life cycle of an Autonomic Service Agent
> 
> 6.1.  Installation phase
> 
>    Before being able to instantiate and run ASAs, the operator must
                                                                 ^^^^
I am going to stop point out that rewording these mandates with something
like "nonwithstanding other options, one working design to support the requirements is..."
can eliminate future DISCUSS ;-)

>    first provision the infrastructure with the sets of ASA software
>    corresponding to its needs and objectives.  The provisioning of the
>    infrastructure is realized in the installation phase and consists in
>    installing (or checking the availability of) the pieces of software
>    of the different ASA classes in a set of Installation Hosts.

I was confused if my understnding of "installed" is the same as that of
the authors. Reading further below, i now know it is, but it might be
good to add a text aside here explaining how "installed" means that installation
means the ASA will be available on the installation host, but not instantiated(running).

Also: class introduced without being explained.

Aka: or else you can also have a terminology section.

Installation Hosts seems to be duplicate to "node". If not, then introduce
and explain difference of terms in text.

>    There are 3 properties applicable to the installation of ASAs:
> 
>    The dynamic installation property  allows installing an ASA on
>       demand, on any hosts compatible with the ASA.
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021                [Page 9]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    The decoupling property  allows controlling resources of a NE from a
>       remote ASA, i.e. an ASA installed on a host machine different from
>       the resources' NE.

NE used without being introduced. If same as node, eliminate.

>    The multiplicity property  allows controlling multiple sets of
>       resources from a single ASA.
> 
>    These three properties are very important in the context of the
>    installation phase as their variations condition how the ASA class
>    could be installed on the infrastructure.

This text seems to rely on text further below to really make sense, so
it would be good to make this forward dependency explicit in the text so the reader
is not left hanging trying to figure out if/how this makes sense without reading
further, eg.: before listing the properties
writing something like: there are three properties related to ASA that
influence how ASA can be installed. The following subsections explains
them.

.... list of properties.

> 
> 6.1.1.  Installation phase inputs and outputs
> 
>    Inputs are:
> 
>    [ASA class of type_x]  that specifies which classes ASAs to install,

Is "that ..." for all these bullet points good english ? Sounds better
to me if these "that" where eliminated.

type_x used without being introduced/explained.
> 
>    [Installation_target_Infrastructure]  that specifies the candidate
>       Installation Hosts,
> 
>    [ASA class placement function, e.g. under which criteria/
>    constraints as defined by the operator]  that specifies how the
>       installation phase shall meet the operator's needs and objectives
>       for the provision of the infrastructure.  In the coupled mode, the
>       placement function is not necessary, whereas in the decoupled
>       mode, the placement function is mandatory, even though it can be
>       as simple as an explicit list of Installation hosts.

coupled/decoupled used without being introduced.
> 
>    The main output of the installation phase is an up-to-date directory
>    of installed ASAs which corresponds to [list of ASA classes]
>    installed on [list of installation Hosts].  This output is also
>    useful for the coordination function and corresponds to the static
>    interaction map (see next section).

Brian Carpenter would say "replace "next section" with explicit xref)" ;-).

>    The condition to validate in order to pass to next phase is to ensure
>    that [list of ASA classes] are well installed on [list of
>    installation Hosts].  The state of the ASA at the end of the
>    installation phase is: installed. (not instantiated).  The following
>    commands or messages are foreseen: install(list of ASA classes,
>    Installation_target_Infrastructure, ASA class placement function),
>    and un-install (list of ASA classes).

Better? "A minimum set of commands to support installation coud be: ..."
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 10]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 6.2.  Instantiation phase
> 
>    Once the ASAs are installed on the appropriate hosts in the network,
>    these ASA may start to operate.  From the operator viewpoint, an
>    operating ASA means the ASA manages the network resources as per the
>    objectives given.  At the ASA local level, operating means executing
>    their control loop/algorithm.
> 
>    But right before that, there are two things to take into
>    consideration.  First, there is a difference between 1. having a
>    piece of code available to run on a host and 2. having an agent based

host == node ??

>    on this piece of code running inside the host.  Second, in a coupled
>    case, determining which resources are controlled by an ASA is
>    straightforward (the determination is embedded), in a decoupled mode
>    determining this is a bit more complex (hence a starting agent will
>    have to either discover or be taught it).
> 
>    The instantiation phase of an ASA covers both these aspects: starting
>    the agent piece of code (when this does not start automatically) and
>    determining which resources have to be controlled (when this is not
>    obvious).
> 
> 6.2.1.  Operator's goal
> 
>    Through this phase, the operator wants to control its autonomic
>    network in two things:
> 
>    1  determine the scope of autonomic functions by instructing which of
>       the network resources have to be managed by which autonomic
>       function (and more precisely which class e.g. 1. version X or
>       version Y or 2. provider A or provider B),

This requires more explanation before i understand the examples.
Whats a class of an autonomic function ? Do i imagine it right that
a single ASA code on a node could be instantiated multiple times with
different parameters ? But is version a parameter or is this two
different versions of the same ASA ? etc. pp.

> 
>    2  determine how the autonomic functions are organized by instructing
>       which ASAs have to interact with which other ASAs (or more
>       precisely which set of network resources have to be handled as an
>       autonomous group by their managing ASAs).

Hard to read. Is this meant to say something like

"autonomic functions are instantiated
by instantiating a set of ASA across one or nodes and instructing them
accordingly about the other ASA in the set as necessary."

If not, maybe rephrase to make it easier to read...


>    Additionally in this phase, the operator may want to set objectives
>    to autonomic functions, by configuring the ASAs technical objectives.

Are these GRASP objectives or other objectives ? If the document wants
to use the term objective outside of GRASP i highly suggest to change all
appropriate occurances of "objective" with "GRASP objective", so a standalone
occurance of "objective" will not be mixed up with a "GRASP objective".

You could use a different word for non-GRASP, such as "goal" for the non-GRASP
occurrances, or maybe best, all occurrances of "objective" are prefixed: Operator
 objective, AF objective, ASA objective, GRASP objective.

(e.g.: if there iss no difference between operator goal and operator objective,
 its less confusing to just use objective with distinguishing prefixes instead
 of mixing goal and objective).

>    The operator's goal can be summarized in an instruction to the ANIMA
>    ecosystem matching the following pattern:
> 
>       [ASA of type_x instances] ready to control
>       [Instantiation_target_Infrastructure] with
>       [Instantiation_target_parameters]
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 11]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 6.2.2.  Instantiation phase inputs and outputs
> 
>    Inputs are:
> 
>    [ASA of type_x instances]  that specifies which are the ASAs to be
>       targeted (and more precisely which class e.g. 1. version X or
>       version Y or 2. provider A or provider B),

Is there a difference between targeting and instantiation ? If yes, explain
targeting, else eliminate surplus word targeted.

Same confusion about version and provider as above. Maybe version and
provider are two words used without being explained ?
> 
>    [Instantiation_target_Infrastructure]  that specifies which are the
>       resources to be managed by the autonomic function, this can be the
>       whole network or a subset of it like a domain a technology segment
>       or even a specific list of resources,
> 
>    [Instantiation_target_parameters]  that specifies which are the
>       technical objectives to be set to ASAs (e.g. an optimization
>       target)
> 
>    Outputs are:
> 
>    [Set of ASAs - Resources relations]  describing which resources are
>       managed by which ASA instances, this is not a formal message, but
>       a resulting configuration of a set of ASAs,

Ending section with a , something missing ?
> 
> 6.2.3.  Instantiation phase requirements
> 
>    The instructions described in section 4.2 could be either:
> 
>    sent to a targeted ASA  In which case, the receiving Agent will have
>       to manage the specified list of
>       [Instantiation_target_Infrastructure], with the
>       [Instantiation_target_parameters].

Why no brackets [send to a targeted ASA] ? Or some other bracketing {}
just to make sure one doesn't read it as a sentence gibberish because these
handing lists in RFC format aren't really well isolating the tag from the text.

>    broadcast to all ASAs  In which case, the ASAs would collectively
>       determine from the list which Agent(s) would handle which
>       [Instantiation_target_Infrastructure], with the
>       [Instantiation_target_parameters].
> 
>    This set of instructions can be materialized through a message that
>    is named an Instance Mandate (description TBD).

Given how Instance Mandate can materialize stuff, i figure it is a 3D printer
 or a Star Trek replicator ?  (kidding: "materialized" reads like a strange word here).

>    The conclusion of this instantiation phase is a ready to operate ASA
>    (or interacting set of ASAs), then this (or those) ASA(s) can
>    describe themselves by depicting which are the resources they manage
>    and what this means in terms of metrics being monitored and in terms
>    of actions that can be executed (like modifying the parameters
>    values).  A message conveying such a self description is named an
>    Instance Manifest (description TBD).

very dense. Maybe rephrase in a more explanatory way. Given how you are
mentioning AF in before, i am missing AF being mentioned in this conclusion.

> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 12]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    Though the operator may well use such a self-description "per se",

s/self-description/Instance Manifest/ ?!

>    the final goal of such a description is to be shared with other ANIMA
>    entities like:

Maybe stronger: The Instance Manifest does not only serve informational
purposes such as acknowledgement of successfull instantion to the
operator, but is also necessary for further autonomous operations:
> 
>    *  the coordination entities (see [I-D.ciavaglia-anima-coordination])
> 
>    *  collaborative entities in the purpose of establishing knowledge
>       exchanges (some ASAs may produce knowledge or even monitor metrics
>       that other ASAs cannot make by themselves why those would be
>       useful for their execution)
> 
> 6.3.  Operation phase
> 
>    Note: This section is to be further developed in future revisions of
>    the document, especially the implications on the design of ASAs.

Ok... i will skip this section until i see a version where this note is removed.

>    During the Operation phase, the operator can:
> 
>       Activate/Deactivate ASA: meaning enabling those to execute their
>       autonomic loop or not.
> 
>       Modify ASAs targets: meaning setting them different objectives.
> 
>       Modify ASAs managed resources: by updating the instance mandate
>       which would specify different set of resources to manage (only
>       applicable to decouples ASAs).
> 
>    During the Operation phase, running ASAs can interact the one with
>    the other:
> 
>       in order to exchange knowledge (e.g. an ASA providing traffic
>       predictions to load balancing ASA)
> 
>       in order to collaboratively reach an objective (e.g.  ASAs
>       pertaining to the same autonomic function targeted to manage a
>       network domain, these ASA will collaborate - in the case of a load
>       balancing one, by modifying the links metrics according to the
>       neighboring resources loads)
> 
>    During the Operation phase, running ASAs are expected to apply
>    coordination schemes
> 
>       then execute their control loop under coordination supervision/
>       instructions
> 
>    The ASA life-cycle is discussed in more detail in "A Day in the Life
>    of an Autonomic Function" [I-D.peloso-anima-autonomic-function].
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 13]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
> 7.  Coordination between Autonomic Functions
> 
>    Some autonomic functions will be completely independent of each
>    other.  However, others are at risk of interfering with each other -
>    for example, two different optimization functions might both attempt
>    to modify the same underlying parameter in different ways.  In a
>    complete system, a method is needed of identifying ASAs that might
>    interfere with each other and coordinating their actions when
>    necessary.  This issue is considered in "Autonomic Functions
>    Coordination" [I-D.ciavaglia-anima-coordination].
> 
> 8.  Coordination with Traditional Management Functions
> 
>    Some ASAs will have functions that overlap with existing
>    configuration tools and network management mechanisms such as command
>    line interfaces, DHCP, DHCPv6, SNMP, NETCONF, RESTCONF and YANG-based
>    solutions.

(sales person hat):
Maybe add that this is not a novel problem for ASA/AF, but is already
 a pre-existing problem when more than one source of e.g. configuration
exists: Two operators not coordinating with each other operators and
SDN controllers. Likewise existing solutions to such coordination 
rqiuremenet in pre-ASA cases can often be extended to include ASA.

>    Each ASA designer will need to consider this issue and
>    how to avoid clashes and inconsistencies.  Some specific
>    considerations for interaction with OAM tools are given in [RFC8368].
>    As another example, [I-D.ietf-anima-prefix-management] describes how
>    autonomic management of IPv6 prefixes can interact with prefix
>    delegation via DHCPv6.  The description of a GRASP objective and of
>    an ASA using it should include a discussion of any such interactions.
> 
>    A related aspect is that management functions often include a data
>    model, quite likely to be expressed in a formal notation such as
>    YANG.  This aspect should not be an afterthought in the design of an
>    ASA.  To the contrary, the design of the ASA and of its GRASP
>    objectives should match the data model; as noted above, YANG
>    serialized as CBOR may be used directly as the value of a GRASP
>    objective.

Paragraph is in duplication with earlier mentioning of YANG. Either
replce "noted above" with explicit xref or move all text from "above" here
into this section. Data-modelling of the interface is otherwise also a good
key point to justify its own subsection in the right place even if its
just one..two paragraphs - just so it shows up in the TOC.

> 
> 9.  Robustness
> 
>    It is of great importance that all components of an autonomic system
>    are highly robust.  In principle they must never fail.  This section
>    lists various aspects of robustness that ASA designers should
>    consider.
> 
>    1.  If despite all precautions, an ASA does encounter a fatal error,
>        it should in any case restart automatically and try again.  To
>        mitigate a hard loop in case of persistent failure, a suitable
>        pause should be inserted before such a restart.  The length of
>        the pause depends on the use case.
> 
>    2.  If a newly received or calculated value for a parameter falls out
>        of bounds, the corresponding parameter should be either left
>        unchanged or restored to a safe value.
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 14]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    3.  If a GRASP synchronization or negotiation session fails for any
>        reason, it may be repeated after a suitable pause.  The length of
>        the pause depends on the use case.
> 
>    4.  If a session fails repeatedly, the ASA should consider that its
>        peer has failed, and cause GRASP to flush its discovery cache and
>        repeat peer discovery.
> 
>    5.  In any case, it may be prudent to repeat discovery periodically,
>        depending on the use case.
> 
>    6.  Any received GRASP message should be checked.  If it is wrongly
>        formatted, it should be ignored.  Within a unicast session, an
>        Invalid message (M_INVALID) may be sent.  This function may be
>        provided by the GRASP implementation itself.
> 
>    7.  Any received GRASP objective should be checked.  If it is wrongly
>        formatted, it should be ignored.  Within a negotiation session, a
>        Negotiation End message (M_END) with a Decline option (O_DECLINE)
>        should be sent.  An ASA may log such events for diagnostic
>        purposes.

Unlike 6., checking of the objective can not be performed by the GRASP
implementation but must be done by the ASA (GRASP client) ?!
> 
>    8.  If an ASA receives either an Invalid message (M_INVALID) or a
>        Negotiation End message (M_END) with a Decline option
>        (O_DECLINE), one possible reason is that the peer ASA does not
>        support a new feature of either GRASP or of the objective in
>        question.  In such a case the ASA may choose to repeat the
>        operation concerned without using that new feature.
> 
>    9.  All other possible exceptions should be handled in an orderly
>        way.  There should be no such thing as an unhandled exception
>        (but see point 1 above).

So.. i think one fundamental consideration worth to note is this:

ASA are not services but they automate services. This has a fundamental
impact on how to design robustness for ASA. In general: When ASA
observe a particular state [1] of operations of the services/resources they
control, they typically aim to improve this state to e.g. a better state [2].
Ideally, ASA are built so that they can ensure that any error they encounter
can still lead to returning to [1] instead of a state [3] which is worse
than [1]. One example instance of this principle is "make-before-break"
used in reconfiguration of routing protocols in manual operations. This
principle of operations can accordingly be coded into the operation of ASA.
The dry-run option mentioned (xref) is another tool towards this ASA design
goal "test-before-make".

> 10.  Security Considerations
> 
>    ASAs are intended to run in an environment that is protected by the
>    Autonomic Control Plane [I-D.ietf-anima-autonomic-control-plane],

ACP provides keying material for mutual authentication between ASA as well
as confidential communication channels for messages between ASA..

>    admission to which depends on an initial secure bootstrap process

s/depends.../can leverage.../ ?!

>    [I-D.ietf-anima-bootstrapping-keyinfra].  In some deployments, a
>    secure partition of the link layer might be used instead
>    [I-D.carpenter-anima-l2acp-scenarios].  However, this does not
>    relieve ASAs of responsibility for security.  In particular, when
>    ASAs configure or manage network elements outside the ACP, they must

whats a network element ? funtion/services on the ASA node or another node ?!

>    use secure techniques and carefully validate any incoming
>    information.  As noted above, this will apply in particular when an
>    ASA interacts with a management component such as a NETCONF server.
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 15]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    As appropriate to their specific functions, ASAs should take account
>    of relevant privacy considerations [RFC6973].

Privacy is not security. If asked for i would always put this into a separate
section as a sentence by itself. (my 2c, i do understand how you just
want to fend of (ridiculuous) DISCUSSes...)

>    Authorization of ASAs is a subject for future study.  At present,
>    ASAs are trusted by virtue of being installed on a node that has
>    successfully joined the ACP.  In the general case, a node may have
>    mutltiple roles and a role may use multiple ASAs, each using multiple
>    GRASP objectives.  Additional mechanisms for the authorization of
>    nodes and ASAs to manipulate specific GRASP objectives could be
>    designed.

Authorization comes out of left field. Some explanatory sentence might
help. e.g.:

Like in most SDN controller designs, this document assumes ASA are authorized
to perform any node configuration/monitoring action it needs to perform for
its function. Like in SDN management solutions, interfaces between ASA and
the router configuration/monitoring services of the node can be subject to
authentication that provides more fine grained authorization for specific
such services. These additional authentication parameters could be
passed to ASA during their instantiation phase.
> 
> 11.  IANA Considerations
> 
>    This document makes no request of the IANA.
> 
> 
> 12.  Acknowledgements
> 
>    Useful comments were received from Michael Behringer Toerless Eckert,
>    Alex Galis, Bing Liu, Michael Richardson, and other members of the
>    ANIMA WG.
>
> 13.  References
> 
> 13.1.  Normative References

Can an informational doc have normative references ? I think not.

>    [I-D.ietf-anima-autonomic-control-plane]
>               Eckert, T., Behringer, M., and S. Bjarnason, "An Autonomic
>               Control Plane (ACP)", Work in Progress, Internet-Draft,
>               draft-ietf-anima-autonomic-control-plane-27, 2 July 2020,
>               <https://tools.ietf.org/html/draft-ietf-anima-autonomic-
>               control-plane-27>.
> 
>    [I-D.ietf-anima-bootstrapping-keyinfra]
>               Pritikin, M., Richardson, M., Eckert, T., Behringer, M.,
>               and K. Watsen, "Bootstrapping Remote Secure Key
>               Infrastructures (BRSKI)", Work in Progress, Internet-
>               Draft, draft-ietf-anima-bootstrapping-keyinfra-41, 8 April
>               2020, <https://tools.ietf.org/html/draft-ietf-anima-
>               bootstrapping-keyinfra-41>.
> 
>    [I-D.ietf-anima-grasp]
>               Bormann, C., Carpenter, B., and B. Liu, "A Generic
>               Autonomic Signaling Protocol (GRASP)", Work in Progress,
>               Internet-Draft, draft-ietf-anima-grasp-15, 13 July 2017,
>               <https://tools.ietf.org/html/draft-ietf-anima-grasp-15>.
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 16]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    [RFC7049]  Bormann, C. and P. Hoffman, "Concise Binary Object
>               Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049,
>               October 2013, <https://www.rfc-editor.org/info/rfc7049>.
> 
> 13.2.  Informative References
> 
>    [DeMola06] De Mola, F. and R. Quitadamo, "An Agent Model for Future
>               Autonomic Communications", Proceedings of the 7th WOA 2006
>               Workshop From Objects to Agents 51-59, September 2006.
> 
>    [GANA13]   "Autonomic network engineering for the self-managing
>               Future Internet (AFI): GANA Architectural Reference Model
>               for Autonomic Networking, Cognitive Networking and Self-
>               Management.", April 2013,
>               <http://www.etsi.org/deliver/etsi_gs/
>               AFI/001_099/002/01.01.01_60/gs_afi002v010101p.pdf>.
> 
>    [Huebscher08]
>               Huebscher, M. C. and J. A. McCann, "A survey of autonomic
>               computing--degrees, models, and applications", ACM
>               Computing Surveys (CSUR) Volume 40 Issue 3 DOI:
>               10.1145/1380584.1380585, August 2008.
> 
>    [I-D.carpenter-anima-l2acp-scenarios]
>               Carpenter, B. and B. Liu, "Scenarios and Requirements for
>               Layer 2 Autonomic Control Planes", Work in Progress,
>               Internet-Draft, draft-carpenter-anima-l2acp-scenarios-02,
>               8 April 2020, <https://tools.ietf.org/html/draft-
>               carpenter-anima-l2acp-scenarios-02>.
> 
>    [I-D.ciavaglia-anima-coordination]
>               Ciavaglia, L. and P. Peloso, "Autonomic Functions
>               Coordination", Work in Progress, Internet-Draft, draft-
>               ciavaglia-anima-coordination-01, 21 March 2016,
>               <https://tools.ietf.org/html/draft-ciavaglia-anima-
>               coordination-01>.
> 
>    [I-D.ietf-anima-grasp-api]
>               Carpenter, B., Liu, B., Wang, W., and X. Gong, "Generic
>               Autonomic Signaling Protocol Application Program Interface
>               (GRASP API)", Work in Progress, Internet-Draft, draft-
>               ietf-anima-grasp-api-06, 12 June 2020,
>               <https://tools.ietf.org/html/draft-ietf-anima-grasp-api-
>               06>.
> 
>    [I-D.ietf-anima-prefix-management]
>               Jiang, S., Du, Z., Carpenter, B., and Q. Sun, "Autonomic
>               IPv6 Edge Prefix Management in Large-scale Networks", Work
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 17]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>               in Progress, Internet-Draft, draft-ietf-anima-prefix-
>               management-07, 18 December 2017,
>               <https://tools.ietf.org/html/draft-ietf-anima-prefix-
>               management-07>.
> 
>    [I-D.ietf-anima-reference-model]
>               Behringer, M., Carpenter, B., Eckert, T., Ciavaglia, L.,
>               and J. Nobre, "A Reference Model for Autonomic
>               Networking", Work in Progress, Internet-Draft, draft-ietf-
>               anima-reference-model-10, 22 November 2018,
>               <https://tools.ietf.org/html/draft-ietf-anima-reference-
>               model-10>.
> 
>    [I-D.ietf-core-yang-cbor]
>               Veillette, M., Petrov, I., and A. Pelov, "CBOR Encoding of
>               Data Modeled with YANG", Work in Progress, Internet-Draft,
>               draft-ietf-core-yang-cbor-13, 4 July 2020,
>               <https://tools.ietf.org/html/draft-ietf-core-yang-cbor-
>               13>.
> 
>    [I-D.liu-anima-grasp-distribution]
>               Liu, B., Xiao, X., Hecker, A., Jiang, S., and Z.
>               Despotovic, "Information Distribution in Autonomic
>               Networking", Work in Progress, Internet-Draft, draft-liu-
>               anima-grasp-distribution-13, 12 December 2019,
>               <https://tools.ietf.org/html/draft-liu-anima-grasp-
>               distribution-13>.
> 
>    [I-D.peloso-anima-autonomic-function]
>               Pierre, P. and L. Ciavaglia, "A Day in the Life of an
>               Autonomic Function", Work in Progress, Internet-Draft,
>               draft-peloso-anima-autonomic-function-01, 21 March 2016,
>               <https://tools.ietf.org/html/draft-peloso-anima-autonomic-
>               function-01>.
> 
>    [Movahedi12]
>               Movahedi, Z., Ayari, M., Langar, R., and G. Pujolle, "A
>               Survey of Autonomic Network Architectures and Evaluation
>               Criteria", IEEE Communications Surveys & Tutorials Volume:
>               14 , Issue: 2 DOI: 10.1109/SURV.2011.042711.00078,
>               Page(s): 464 - 490, 2012.
> 
>    [RFC6241]  Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed.,
>               and A. Bierman, Ed., "Network Configuration Protocol
>               (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011,
>               <https://www.rfc-editor.org/info/rfc6241>.
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 18]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    [RFC6973]  Cooper, A., Tschofenig, H., Aboba, B., Peterson, J.,
>               Morris, J., Hansen, M., and R. Smith, "Privacy
>               Considerations for Internet Protocols", RFC 6973,
>               DOI 10.17487/RFC6973, July 2013,
>               <https://www.rfc-editor.org/info/rfc6973>.
> 
>    [RFC7575]  Behringer, M., Pritikin, M., Bjarnason, S., Clemm, A.,
>               Carpenter, B., Jiang, S., and L. Ciavaglia, "Autonomic
>               Networking: Definitions and Design Goals", RFC 7575,
>               DOI 10.17487/RFC7575, June 2015,
>               <https://www.rfc-editor.org/info/rfc7575>.
> 
>    [RFC7665]  Halpern, J., Ed. and C. Pignataro, Ed., "Service Function
>               Chaining (SFC) Architecture", RFC 7665,
>               DOI 10.17487/RFC7665, October 2015,
>               <https://www.rfc-editor.org/info/rfc7665>.
> 
>    [RFC8368]  Eckert, T., Ed. and M. Behringer, "Using an Autonomic
>               Control Plane for Stable Connectivity of Network
>               Operations, Administration, and Maintenance (OAM)",
>               RFC 8368, DOI 10.17487/RFC8368, May 2018,
>               <https://www.rfc-editor.org/info/rfc8368>.
> 
>    [RFC8568]  Bernardos, CJ., Rahman, A., Zuniga, JC., Contreras, LM.,
>               Aranda, P., and P. Lynch, "Network Virtualization Research
>               Challenges", RFC 8568, DOI 10.17487/RFC8568, April 2019,
>               <https://www.rfc-editor.org/info/rfc8568>.
> 
> Appendix A.  Change log [RFC Editor: Please remove]
> 
>    draft-carpenter-anima-asa-guidelines-09, 2020-07-25:
> 
>    *  Additional text on future authorization.
>    *  Editorial fixes
> 
>    draft-carpenter-anima-asa-guidelines-08, 2020-01-10:
> 
>    *  Introduced notion of autonomic ecosystem.
>    *  Minor technical clarifications.
>    *  Converted to v3 format.
> 
>    draft-carpenter-anima-asa-guidelines-07, 2019-07-17:
> 
>    *  Improved explanation of threading vs event-loop
>    *  Other editorial improvements.
> 
>    draft-carpenter-anima-asa-guidelines-06, 2018-01-07:
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 19]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    *  Expanded and improved example logic flow.
>    *  Editorial corrections.
> 
>    draft-carpenter-anima-asa-guidelines-05, 2018-06-30:
> 
>    *  Added section on relationshp with non-autonomic components.
>    *  Editorial corrections.
> 
>    draft-carpenter-anima-asa-guidelines-04, 2018-03-03:
> 
>    *  Added note about simple ASAs.
>    *  Added note about NFV/SFC services.
>    *  Improved text about threading v event loop model
>    *  Added section about coordination with traditional tools.
>    *  Added appendix with example logic flow.
> 
>    draft-carpenter-anima-asa-guidelines-03, 2017-10-25:
> 
>    *  Added details on life cycle.
>    *  Added details on robustness.
>    *  Added co-authors.
> 
>    draft-carpenter-anima-asa-guidelines-02, 2017-07-01:
> 
>    *  Expanded description of event-loop case.
>    *  Added note about 'dry run' mode.
> 
>    draft-carpenter-anima-asa-guidelines-01, 2017-01-06:
> 
>    *  More sections filled in.
> 
>    draft-carpenter-anima-asa-guidelines-00, 2016-09-30:
> 
>    *  Initial version
> 
> Appendix B.  Example Logic Flows

[too late today, will review another time]

> 
>    This appendix describes generic logic flows for an Autonomic Service
>    Agent (ASA) for resource management.  Note that these are
>    illustrative examples, and in no sense requirements.  As long as the
>    rules of GRASP are followed, a real implementation could be
>    different.  The reader is assumed to be familiar with GRASP
>    [I-D.ietf-anima-grasp] and its conceptual API
>    [I-D.ietf-anima-grasp-api].
> 
>    A complete autonomic function for a resource would consist of a
>    number of instances of the ASA placed at relevant points in a
>    network.  Specific details will of course depend on the resource
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 20]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    concerned.  One example is IP address prefix management, as specified
>    in [I-D.ietf-anima-prefix-management].  In this case, an instance of
>    the ASA would exist in each delegating router.
> 
>    An underlying assumption is that there is an initial source of the
>    resource in question, referred to here as an origin ASA.  The other
>    ASAs, known as delegators, obtain supplies of the resource from the
>    origin, and then delegate quantities of the resource to consumers
>    that request it, and recover it when no longer needed.
> 
>    Another assumption is there is a set of network wide policy
>    parameters, which the origin will provide to the delegators.  These
>    parameters will control how the delegators decide how much resource
>    to provide to consumers.  Thus the ASA logic has two operating modes:
>    origin and delegator.  When running as an origin, it starts by
>    obtaining a quantity of the resource from the NOC, and it acts as a
>    source of policy parameters, via both GRASP flooding and GRASP
>    synchronization.  (In some scenarios, flooding or synchronization
>    alone might be sufficient, but this example includes both.)
> 
>    When running as a delegator, it starts with an empty resource pool,
>    it acquires the policy parameters by GRASP synchronization, and it
>    delegates quantities of the resource to consumers that request it.
>    Both as an origin and as a delegator, when its pool is low it seeks
>    quantities of the resource by requesting GRASP negotiation with peer
>    ASAs.  When its pool is sufficient, it hands out resource to peer
>    ASAs in response to negotiation requests.  Thus, over time, the
>    initial resource pool held by the origin will be shared among all the
>    delegators according to demand.
> 
>    In theory a network could include any number of origins and any
>    number of delegators, with the only condition being that each
>    origin's initial resource pool is unique.  A realistic scenario is to
>    have exactly one origin and as many delegators as you like.  A
>    scenario with no origin is useless.
> 
>    An implementation requirement is that resource pools are kept in
>    stable storage.  Otherwise, if a delegator exits for any reason, all
>    the resources it has obtained or delegated are lost.  If an origin
>    exits, its entire spare pool is lost.  The logic for using stable
>    storage and for crash recovery is not included in the pseudocode
>    below.
> 
>    The description below does not implement GRASP's 'dry run' function.
>    That would require temporarily marking any resource handed out in a
>    dry run negotiation as reserved, until either the peer obtains it in
>    a live run, or a suitable timeout expires.
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 21]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    The main data structures used in each instance of the ASA are:
> 
>    *  The resource_pool, for example an ordered list of available
>       resources.  Depending on the nature of the resource, units of
>       resource are split when appropriate, and a background garbage
>       collector recombines split resources if they are returned to the
>       pool.
> 
>    *  The delegated_list, where a delegator stores the resources it has
>       given to consumers routers.
> 
>    Possible main logic flows are below, using a threaded implementation
>    model.  The transformation to an event loop model should be apparent
>    - each thread would correspond to one event in the event loop.
> 
>    The GRASP objectives are as follows:
> 
>    *  ["EX1.Resource", flags, loop_count, value] where the value depends
>       on the resource concerned, but will typically include its size and
>       identification.
> 
>    *  ["EX1.Params", flags, loop_count, value] where the value will be,
>       for example, a JSON object defining the applicable parameters.
> 
>    In the outline logic flows below, these objectives are represented
>    simply by their names.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 22]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    <CODE BEGINS>
> 
>    MAIN PROGRAM:
> 
>    Create empty resource_pool (and an associated lock)
>    Create empty delegated_list
>    Determine whether to act as origin
>    if origin:
>        Obtain initial resource_pool contents from NOC
>        Obtain value of EX1.Params from NOC
>    Register ASA with GRASP
>    Register GRASP objectives EX1.Resource and EX1.Params
>    if origin:
>        Start FLOODER thread to flood EX1.Params
>        Start SYNCHRONIZER listener for EX1.Params
>    Start MAIN_NEGOTIATOR thread for EX1.Resource
>    if not origin:
>        Obtain value of EX1.Params from GRASP flood or synchronization
>        Start DELEGATOR thread
>    Start GARBAGE_COLLECTOR thread
>    do forever:
>        good_peer = none
>        if resource_pool is low:
>            Calculate amount A of resource needed
>            Discover peers using GRASP M_DISCOVER / M_RESPONSE
>            if good_peer in peers:
>                peer = good_peer
>            else:
>                peer =  #any choice among peers
>                grasp.request_negotiate("EX1.Resource", peer)
>                i.e., send M_REQ_NEG
>                Wait for response (M_NEGOTIATE, M_END or M_WAIT)
>                if OK:
>                    if offered amount of resource sufficient:
>                        Send M_END + O_ACCEPT #negotiation succeeded
>                        Add resource to pool
>                        good_peer = peer
>                    else:
>                        Send M_END + O_DECLINE #negotiation failed
>        sleep() #sleep time depends on application scenario
> 
>    MAIN_NEGOTIATOR thread:
> 
>    do forever:
>        grasp.listen_negotiate("EX1.Resource")
>        i.e., wait for M_REQ_NEG
>        Start a separate new NEGOTIATOR thread for requested amount A
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 23]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    NEGOTIATOR thread:
> 
>    Request resource amount A from resource_pool
>    if not OK:
>        while not OK and A > Amin:
>            A = A-1
>            Request resource amount A from resource_pool
>    if OK:
>        Offer resource amount A to peer by GRASP M_NEGOTIATE
>        if received M_END + O_ACCEPT:
>            #negotiation succeeded
>        elif received M_END + O_DECLINE or other error:
>            #negotiation failed
>    else:
>        Send M_END + O_DECLINE #negotiation failed
> 
>    DELEGATOR thread:
> 
>    do forever:
>        Wait for request or release for resource amount A
>        if request:
>            Get resource amount A from resource_pool
>            if OK:
>                Delegate resource to consumer
>                Record in delegated_list
>            else:
>                Signal failure to consumer
>                Signal main thread that resource_pool is low
>        else:
>            Delete resource from delegated_list
>            Return resource amount A to resource_pool
> 
>    SYNCHRONIZER thread:
> 
>    do forever:
>        Wait for  M_REQ_SYN message for EX1.Params
>        Reply with M_SYNCH message for EX1.Params
> 
> 
>    FLOODER thread:
> 
>    do forever:
>        Send M_FLOOD message for EX1.Params
>        sleep() #sleep time depends on application scenario
> 
> 
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 24]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    GARBAGE_COLLECTOR thread:
> 
>    do forever:
>        Search resource_pool for adjacent resources
>        Merge adjacent resources
>        sleep() #sleep time depends on application scenario
> 
>    <CODE ENDS>
> 
> Authors' Addresses
> 
>    Brian Carpenter
>    School of Computer Science
>    University of Auckland
>    PB 92019
>    Auckland 1142
>    New Zealand
> 
>    Email: [e-mail]
> 
> 
>    Laurent Ciavaglia
>    Nokia
>    Villarceaux
>    91460 Nozay
>    France
> 
>    Email: [e-mail]
> 
> 
>    Sheng Jiang
>    Huawei Technologies Co., Ltd
>    Q14 Huawei Campus
>    156 Beiqing Road
>    Hai-Dian District
>    Beijing
>    100095
>    China
> 
>    Email: [e-mail]
> 
> 
>    Pierre Peloso
>    Nokia
>    Villarceaux
>    91460 Nozay
>    France
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 25]
> 
> Internet-Draft               ASA Guidelines                    July 2020
> 
> 
>    Email: [e-mail]
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Carpenter, et al.        Expires 26 January 2021               [Page 26]