Re: [nmrg] Review of draft-irtf-nmrg-autonomic-network-definitions-04

[Michael Behringer <[e-mail]>]

Kostas,

First of all thanks for your very thorough review of the document, this 
is really appreciated. Sorry for the late reply, I'm working in 
"batches" on each draft. Inline more comments.

On 27/10/2014 19:58, Kostas Pentikousis wrote:
> Dear authors, all,
>
> I reviewed draft-irtf-nmrg-autonomic-network-definitions-04. Overall, the draft is easy-to-read, but it feels more like "original" work than the RG documents I'm familiar with. As I mentioned in the Toronto meeting and on the list, the draft lacks the appropriate list of "citations and references to relevant research publications". I personally do not consider sufficient the three references mentioned in passing ("There is a
> large literature, including several useful overview papers (e.g., [Samaan], [Movahedi], and [Dobson])"). Further, several assumptions remain unstated (could be fixed by adding a new section) and certain inconsistencies persist in this revision (see below).

We made a conscious choice to not go into the details of existing work, 
because 1) there is a vast range of papers on the subject, and it would 
blow the format of an IETF document to go through all of those, and 2) 
there are existing papers that compare and analyse the existing work 
already. Our intention is to cite those overview papers, but consciously 
not to repeat such work here.

I'll address the inconsistencies of course, below.

> In summary: If this draft is to become the reference RFC on Autonomic Networking, more work is needed in the RG.
>
>
> General comments:
> -----------------
>
> The draft includes some normative-resembling language, which may be misleading to uninitiated readers:

I'll look through those sentences in detail, but generally, a reader 
with MINIMAL knowledge of the IETF will understand that this is not 
normative, because 1) it's an informational draft, 2) it doesn't 
reference RFC2119, and 3) none of the "MUST" and other keywords is in 
uppercase.
However, I agree we want to avoid misunderstandings, so I'll still look 
through those sentences.
> * "For this reason we suggest here to NOT generally disable autonomic functions if they encounter unexpected conditions ..."
>
> * "Autonomic functions must be able to adapt their behavior ... "
>
> * "Intent must not be used to convey low-level commands or concepts ..."
>
> * "an autonomic function must understand the full life cycle of the device it runs on ...
>
> * "Warnings however should be issued if node specific overrides may conflict with autonomic behaviour."
>
> * "Information about the network should be collected and aggregated by the network itself, presented in consolidated fashion to the administrator."
>
> * "Autonomic network events should concern the autonomic network as a whole, not individual systems in isolation. ..." (and the rest of the paragraph)
>
> * "apply specific algorithms to determine what should be reported."

Up to here, I do not see a possible misunderstanding, and I really don't 
see how this could be mistaken for normative language in an 
informational document. I also don't quite see what you would like us to 
do? Not use the words "should" and "must"?

> * "All the components around the "autonomic service agents" should be common components, such that the autonomic service agents do not have to replicate common tasks individually." Doesn't this read like an implementation suggestion?

The intention here is to explain why standardisation is actually 
necessary. There should be common components that take on tasks that 
many components will need to do. I don't see this as an implementation 
suggestion, but as an architectural consideration. How would you suggest 
to change it?

> * "autonomic functions in the context of this framework should therefore operate at the IP layer."
>
> I guess it would be better to rephrase the above sentences to avoid future misunderstandings.
>
> Also, have we reached consensus in NMRG that "The goal of the work on autonomic networking in the IETF is therefore not just to create autonomic functions, but to define a common infrastructure that autonomic functions can use." And, why is an NMRG draft indicating what the IETF goal is?

I believe this sentence reflects discussions we had in NMRG meetings and 
outside, and yes, this is sort of a statement of direction for the work 
in the IETF. I believe this is in fact what the IRTF aims to do: 
Recommend to the IETF how to handle topics that are mature in research 
but not yet in implementations.

> Section-specific comments:
> --------------------------
>
> - Abstract: The sentence "This document applies the concepts of autonomic systems to a network, ..." does not reflect the draft contents. There is very little that is being _applied_ to, as no references to earlier work are given for the concepts defined in the draft (as per Sec. 2). As mentioned above, the text reads more like original work undertaken for this draft.

Very good point, you are right, the abstract doesn't reflect the 
document content very well any more. I've changed it significantly to 
reflect the current content better.

> - Section 1: The draft makes an implicit assumption which should become explicit. Namely, the draft adopts a node-centric approach ("Autonomic Networking aims at putting the intelligence of today's operations back into algorithms at the node level ...". This assumption should be stated explicitly. Mind you, not everyone in the research community may agree that AN should be defined at the node level, instead of at the system level. I'd would be interested in other folks' opinion/pointers in earlier literature on this. Note that "node" is defined in Sec. 2 by what it does (i.e. it "employs exclusively autonomic functions").

This is indeed a very important point, and I THOUGHT we had brought it 
out well, but obviously NOT. So thanks for raising this, this MUST be 
clear! So, I'll be clearer about this.

I agree that you could define it either way (node level or system 
level). We just need to be clear what we're working on here.

Personal opinion: The system level definition is perfectly acceptable, 
but probably doesn't require additional standardisation in the IETF. You 
could use existing protocols to implement feedback loops between central 
elements and network elements. Essentially, I would argue the SDN 
approach has the potential to go this way.

In this work we want to concentrate on node level autonomy. This does 
NOT mean that it's the only correct way, it's just setting the scope. As 
said, I'd expect system level autonomy to come from other groups.

But, you are right, this needs to be made clearer. Working on it.

> The following sentence implies a certain dichotomy: "a mixed approach, where some functions are autonomic and others are centrally mangaged", which may be true if you have the node-centric assumption in mind. A RoutFlow-like network runs OSPF "centrally" but I would argue that it retains the autonomic properties of the protocol at the system level ("Autonomic Domain" in terms of this draft). This dichotomy appears later in the text as well, e.g. "which functions should be centralised or follow an autonomic approach."

This is really a question of positioning though, isn't it? With your 
comments above I have clarified that

       <t>This work specifically focuses on node level autonomic 
functions. It
       focuses on intelligence of algorithms at the node level, to minimize
       dependency on human administrators and central management 
systems. </t>

If I re-read the above text, to me that means that here we focus on node 
level autonomy a dn centrally managed is out of scope for this work. 
Then, the above makes sense I think. Would you agree?

> Regarding related work, the text indicates that in "the present document we focus on concepts and definitions that seem sufficiently mature to become the basis for interoperable specifications in the near future." Since no references are given, _who_ has determined that the "concepts and definitions" in this draft are "mature". The authors? NMRG? ... Was there a workshop to tackle this point? Perhaps adding a reference would resolve this in a speedy manner. Ditto for the ending sentence of this section: "This document provides the definitions and design goals for Autonomic Networking." In which context?

We have had quite a few discussions in NMRG meetings, and on the list 
about this point, and I believe the definitions and concepts in the doc 
represent a consensus. Can you be specific on where you believe we do 
not have consensus?

All we're trying to say with this sentence: In academic research, 
autonomics goes much further, we're really trying to capture the 
fundamentals only.

To your last question: Context is the work in the IETF and IRTF. (I'll 
add this to the text)

Put bluntly: If a future document in the IETF agrees with these 
definitions, it may reference this document. If they want to make their 
own definitions, fine. Nobody is obliged to use this document. Needless 
to say, of course we hope it to be usefuls, and would be seriously 
concerned if it wasn't used in the future.

> Another assumption is that the draft aims for a migration environment ("such specifications will need to co-exist with traditional methods of network configuration and management, rather than realising an exclusively autonomic system with all the properties that it would require."). A distinct section entitled "Assumptions" would serve the casual reader well.

Hmmm... Somewhat sceptical about re-arranging the entire doc at this 
point. Let me think about this.

> - Section 2: Since no references are provided, some text should be added to explicitly state that these definitions are specified by _this_ draft. Alternatively, of course, refs could be added.

Well, if you use external references, you add them. If you don't add 
references, it's the definition of this draft.  Isn't that obvious? 
Frankly, I think this is redundant. But, to avoid a lengthy discussion, 
I'll add "We make the following definitions:" OK?

> It's not crystal clear what is the relationship between an Autonomic Network and a Autonomic Domain. Based on their current definitions, it appears to be the lack of (the same) intent. Please clarify.

An autonomic network may have one or severals autonomic domains. Added, 
thanks.

> The concept of an operating region for an autonomic node/network/domain is missing in the definitions.

Can you explain what you mean by "operating region"? We don't use that 
term currently.

> - Section 3: Do we have consensus in NMRG that "The original design goals of autonomic systems as described in [Kephart] also apply to Autonomic Networks" (as-is)? I heard different opinions, but I'm curious what others in the RG think.

You're implying that they don't, without saying why. Rather than leaving 
us all guessing, can you please be explicit where you think we deviate?

> In Sec. 3.1:
>
> "Functions do not require to be configured," -- by whom? Do you mean a network operator?

We mean: "Functions do not require to be configured, neither by an 
administrator nor a management system."
(changed)

> "secure themselves against potential attacks." -- shouldn't this be a bit scoped?

I see this as a high level design goal. Specifics shouldn't be in 
"design goals", but I do agree that when we get to the detailled 
specifications, we need to go into much more detail. For this section, I 
think we can leave this high level statement. Would you agree?

> Proposed text:
>
> OLD: "determine ways to optimise their behaviour."
> NEW: "determine ways to optimise their behavior against a set of well-defined goals".

Changed. Thanks.

> In Sec. 3.2:
>
> "Conflict resolution between autonomic default behaviour and intent on one side ..." -- This may be confusing. Why doesn't _autonomic default behavior_ include intent? Please clarify.
>
> Given the implicit
(now after your comments more explicit ;-)
> node-centric approach adopted in this draft (see above) the following sentence is a bit contradictory, and I think inconsistent with the rest of the text: "Generally, autonomic mechanisms define a network wide behaviour, whereas the alternative methods are typically on a node by node basis."

I don't understand your point here. Let me give you an example to 
illustrate our thinking:
- default behaviour: a network uses ULA to address itself.
- intent overrides this with "use prefix x instead".
Both are network wide.
Between autonomic default (1st line) and intent (2nd line) clearly, 
intent overrides (maybe should make this explicit?)
Both can be overridden locally by someone configuring (any) other 
address on a particular box. In which case that overrides any autonomic 
approach (above lines).

In the light of such an example, what is not clear about the above text? 
Please help me understand where  we're confusing.

> Then, "Node based management concepts take a higher priority over autonomic methods." But don't we refer to Autonomic Nodes?

The point is that in practice you can also configure a node, in which 
case you need to say what takes precedence. You need to help me out 
here, to me this is completely obvious.

> As the draft indicates, network operations tend to be manual/configuration-oriented as opposed to adopting well-defined algorithms for steering operation. Is operating an atomic plant less critical or less complex than operating a network? Some retrospective/discussion on earlier efforts in the autonomic networking practice would be valuable here.

Can you suggest some examples?
My fear is: Yes, we could add a lot of history, examples, discussion. 
But the document would quickly grow to 50+ pages if we're not careful. 
Frankly, I'd prefer to stick to the essentials and keep the document to 
the point.

> In sec. 3.4: does the following maxim "If a problem can be solved in a distributed manner, it should not be centralised." belong to this draft?

I think so. It clarifies what we're up to with this draft. This doesn't 
mean you can't do system autonomic (SDN like) approaches, and they make 
perfect sense for many tasks. But this draft isn't written for that 
world. And I think we want to be clear about that.

And I say that exactly BECAUSE I get drawn into these debates all the 
time: "But also a controller based system can be autonomic". True, but 
this is system level, and this doc is just not focusing on that. There 
is no value statement here, btw. Both work. We choose to describe here 
the distributed approach. (And I hope we're making that clear).

> In Sec. 3.6: the goal "optimize my network for energy efficiency" is not well-defined. Abstraction may it be, but it has to be measurable somehow. See proposed text above.

It's an example. We're not defining here how to do it, nor how to 
measure it. Reading your comment I was about to define it, but frankly, 
I think this is a rat hole - because as soon as you write "overall power 
consumption has to be minimised" then questions on redundancy and 
reduced availability will pop up, with related exceptions, 
optimisations, etc. Please: Let's leave it as an example, to avoid 
ratholing, ok? ;-)

Alternatively: Do you have a better example?

> Also, it's not clear how this sentence "The administrator should not even be exposed to the version of the IP protocol running in the network." is related with "energy optimization".

It isn't :-)  I split those into two paragraphs, thanks for calling this 
out! :-)

> In Sec. 3.9: Please add a couple of references/examples to better document the sentence "For example, layer 2 switching today is already relatively autonomic in many environments; routing functions can be autonomic."

Done. Thanks.

> - Section 4: In general this section needs further editing. First, the opening sentence ("This section identifies various items that are explicitly not design goals for autonomic networks, ...") is quite vague. For whom is this _not_ a design goal? NMRG? IRTF? IETF? Researchers? Practitioners? Engineers? Network operators? Protocol designers/implementers? In general, it is not clear whose anxieties does this section address by explicitly declaring non-goals. Second, are the following sentences really needed?

It's in the context of IETF/IRFT (added), and I added some text there to 
clarify "They address some commonly expressed concerns from network 
administrators and architects."

> * "(They should become more like doctors than hospital orderlies.)"
> * "Truck rolls will not be eliminated when faulty equipment needs to be replaced."
> * "Senior management might fear loss of control of an autonomic network."
>
> In general, is Section 4 needed in the first place? Isn't it better to define the goals clearly so that listing an arbitrary set of non-goals becomes redundant?

Well, to be honest, I see those concerns raised a lot, and I think it 
adds value to address them here.

> - Section 5: wrt Fig. 1, it's not clear why the "Autonomic Control Plane" is outside the "Standard Operating System Functions". Such an illustration somehow indicates that ACP is an add-on, as opposed to being part and parcel of an autonomic node.

That's a good point. Long term you're of course right. We should have 
said "Current standard OS functions".

> Editorial/Nits:
> -----
>
> In the beginning of Sec. 2, a sentence such as "This document uses the following terms:" would be nice to add.

Done. (Already from your previous comments ;-)

> s/mangaged./managed.
>
> s/self- managing,/self-managing,
>
> s/take place."/take place.
>
> s/outside scope/outside the scope
>
> respected&#8206; ???
>
> s/(Section 3.5,/ (Section 3.5)

All done, thanks!

> Finally, there's no text adhering to RFC 5743, Sec. 2.1. It would be good to add in the next draft revision.

I believe that will get added once there is consensus in the RG.

Thanks for your detailed review again, and sorry for taking a long time 
to work on it! Maybe, if we're still not in agreement on some issues, we 
should discuss on the phone - might be easier.

Michael
>
>
> Best regards,
>
> Kostas
>
>
> .
>