Re: [OPSAWG] (final) WGLC on draft-ietf-opsawg-operations-and-management

["Adrian Farrel" <[e-mail]>]

Hi,

Dan asked me to have a look at this which I do primarily with a routing area 
view on the world.

I'm sorry that I haven't been following this I-D very closely and apologise 
if you have already discussed and reached conclusion on some of the points I 
make.

I am largely supportive of this work (see 
draft-ietf-pce-manageability-requirements-06.txt), but I do have some 
issues. I would be comforted to know that the I-D had had extensive review 
from within the Ops Area without relying just on my comments!

I would also like to understand how the recommendations parts of the draft 
will be interpreted by the Ops ADs as they perform IESG review of I-Ds from 
other areas. Depending on the answer (which I would like to see encapsulated 
in the document!) I might have other harsher comments about the content.

More details below.

Cheers,
Adrian

===
Title is...
 Guidelines for Considering Operations and Management of New Protocols
...but the Abstract immediately includes...
   New protocols or protocol extensions
===
Obviously needs the new IPR boilerplate.
===
Abstract line 3
s/protocol/protocols/
===
I am somewhat worried by "recommend appropriate standard management
protocols ... to address the relevant operations and management
needs."

I understand the theory behind this guidance in terms of making
equipment that will interwork with management systems, but I
question its practicality given the vast array of management
protocols available. And actually, I am not sure that this has a
direct impact on the success of or development of a protocol.

The introduction says that this "is similar to a WG considering
which security threats are relevant to their protocol, and then
recommending appropriate standard security protocols to mitigate
the relevant threats." I dispute this. The security considerations
normally make modifications to the protocol under development to
make it secure, or mandate security features in other protocols
that the protocol under development uses. But where there is a
need for an external protocol, this is not usually mandated. For
example, where key exchange is needed the text will usually say
something like "may use some form of secure key exchange protocol
(such as the Internet Key Exchange protocol v2 (IKE) [RFC4306]."
This is far looser than a recommendation of a specific protocol.

I believe that in practice operators want to go on using the
management protocols that they already have deployed. If, for
example, I have deployed routers that I manage using SNMP, I will
not want to deploy a new routing protocol on those routers if that
protocol is strongly recommended to be managed using TL1.

Since data modeling is usually tied closely to the protocol used
for management, it follows from this that developing specific
data formats (SNMP MIB modules, XML schema, etc.) might also not
be so very practical. (Arguably we have found this out with MIB
modules over the last 10 years, but still churn them out.)

What *would* be beneficial, however, is an abstract data model
for management of the protocol. There are many ways of
representing the data (XML, BNF, ASN.1, graphical representation,
etc.) and the Ops Area could usefully pick one and standardise
it. From there we would have a solid data set that all
implementations could make available and which separate documents
could easily map to suitable formats for use by specific
management protocols. Smart implementers would be able to support
multiple management protocols with ease.

You do come to this to some extent in section 4.1, and the text
there seems to be at odds with what you have said in the
Introduction.

===
Section 1

Can you expand "WG" on first usage?

===
Section 1

   We want to avoid seeming to impose a solution by putting in place a
   strict terminology - for example implying that a formal data model,
   or even using a management protocol is mandatory.

<grin mode=wry>

You want to avoid seeming to impose it?
So, you want to impose it, but you don't want anyone to notice?

</grin>

===
Section 1

   Making a Management Considerations section a mandatory publication
   requirement for IETF documents is the responsibility of the IESG, or
   specific area directors, or working groups, and this document avoids
   recommending any mandatory publication requirements.

I think you may need to go a step further to get this through the
IESG. It would not be funny to see this paragraph as written, but
to get a DISCUSS on an I-D from an Ops AD because the I-D did not
contain a Management Considerations section.

===
Introduction

   We provide some objective criteria to promote interoperability

<grin mode=laconic>

There seems to be only one author. Are you royalty?
Perhaps... "This document provides..."

</grin>

===
Section 1 final line

s/??/?"/

===

2.1.  IETF Management Framework

   For years the IETF has stressed the use of the IETF Standard
   Management Framework and MIB modules [RFC2578] for managing new
   protocols.  The IETF designed these to permit multiple protocols to
   utilize the MIB data [RFC1052], but it became a common

s/became/is/

===

3.1.  Operations Model

   o  what are the typical user interfaces - Command line (CLI) or
      graphical user interface (GUI)?

I am not sure that this last item is particularly relevant for a
large number of the protocols designed in the IETF. We might say
that the question is valid for a protocol such as FTP, but is it
relevant for OSPF? Yes, there is a configuration interface to a
device that runs OSPF, but it is surely nothing to do with the
protocol design how that interface works. Indeed, you might
reduce the question to: are the configuration interfaces local or
remote? But I think you will find that the answer to the question
is "both" as in the subsequent paragraph.

===

3.1.  Operations Model

   Auto-configuration and default parameters might be
   possible for some new protocols.

I would make a much bigger noise about default parameters. There is
a key question about how the protocol can operate "out of the box".
If implementers are free to select their own defaults, the protocol
needs to operate well with any choice of values. If there are
sensible defaults, these need to be stated.

Indeed, section 3.2 seems to say this. Perhaps a forward reference?

===

3.1.  Operations Model

   There may be a need to support a human interface, e.g., for
   troubleshooting, and a programmatic interface, e.g., for automated
   monitoring and root cause analysis.  It might be important that the
   internal method routines used by the application programming
   interfaces and the human interfaces should be the same to ensure that
   data exchanged between these two interfaces is always consistent.
   Mixing methods leads to inconsistency, so identifying consistent
   methods of retrieving information is relevant.

This is really sailing too close to the implementation. People
must be free to construct sub-standard and unmarketable
implementations. The IETF is not responsible for making all
implementations successful. The job is to make it possible for
all implementations to be successful given core competences in the
developers.

I think the furthest a protocol spec RFC could go would be to say
what information must be available to an operator at a human
interface for troubleshooting

===

3.2.  Installation and Initial Setup

In view of the reasons you give why defaults might become out-
dated, you might point out that sometimes it is not advisable to
have defaults. In these cases explicit configuration is required.

===

3.2.  Installation and Initial Setup

Is the text at the top of page 8 supposed to be a bullet list?

===

3.4.  Requirements on Other Protocols and Functional Components

   These considerations should generally remain illustrative to avoid
   creating restrictions or dependencies, or potentially impacting the
   behavior of existing protocols, or restricting the extensibility of
   other protocols, or assuming other protocols will not be extended in
   certain ways.

Well, this is quite interesting.

Suppose you decide to utilise a particular feature of a transport
protocol. You are creating a hard requirement that that feature is
implemented in all implementations of the set {your new protocol,
the transport protocol}. But more importantly, perhaps, you are
requiring that new versions of the transport protocol do not
deprecate the feature you require.

In short, I don't think you should be too nervous about creating
restrictions or dependencies. If they exist, they must be stated.

===

3.2.  Installation and Initial Setup

   For example, the design of Resource ReSerVation Protocol (RSVP)

I think this example is not well expressed.

The problem in RSVP was that the IP packet was addressed to the
ultimate RSVP-capable destination, but the RSVP-capable routers
along the path needed to intercept and process the Path message.
This they could only do by "deep packet inspection" or by the
router alert option.

I wonder whether this example adds significantly to the I-D.

===

4.  Management Considerations

   NETCONF addresses this in a generic manner by

Add citation.

===

4.  Management Considerations

   However, debugging on-the-wire interactions is a protocol
   issue: it is enormously helpful if a protocol has hooks to make
   debugging of network interactions easy, and/or is designed in such a
   way that debugging protocol behaviors is easy.  Hand-waving this away
   is not something that operators like ...

I think you are including this by hand-waving!
What do you mean by "hooks"?
Surely any protocol can be debugged simply by sniffing it.
If you have specific hooks in mind you must set them out and not
leave the reader guessing.
But if you force the implementation I shall be peeved!
You might, for example, say that there should be alerts issued when
messages are received, or when there are state transitions in the
protocol state machine, but you must recall that:
- the messages can be seen by sniffing
- the state machine is not part of the protocol specification.
This latter point is key. The state machine explains how the
protocol works in order that an implementer can decide how to
react to a received event, but it does not require that the
machine is implemented.

===

4.1.  Interoperability

   Interoperability needs to be considered on the syntactic level and
   the semantic level.  While it can be irritating and time-consuming,
   application designers including operators who write their own scripts
   can make their processing conditional to accommodate differences

s/differences/syntactic differences/ ?

   Then, whether the counter is gathered via SNMP or a CLI
   command or a SYSLOG message, the counter will have similar meaning.

"similar"? I think that might not be good enough. Don't you need
"the same"?

   o  [RFC3805] - Printer MIB v2 (contains both an IM and a DM

Missing ")"

===

4.2.  Management Information

I would find it really helpful if you were able to recommend a
language in which to write the information model. Ideally, all
IETF IMs would be in the same language.

   It is important to be able to separately fetch configuration data,
   operational state data, and statistics from devices, and to be able
   to compare current state to initial state, and to compare data
   between devices.

You seem to have drifted into DMs here. You cannot fetch data from
an IM. It simply explains what data elements exist. In fact, an IM
states all of the data that exists and so there is no confusion.
Confusion can only arise as the DM is constructed and multiple IM
data elements are conflated into single DM objects.

===

4.3.  Fault Management

   The protocol designer should document the basic faults and health
   indicators that need to be instrumented for the new protocol, and the
   alarms and events that must be propagated to management applications
   or exposed through a data model.

Isn't "propagated to management applications" a subset of "exposed
through a data model"?

   The protocol designer should consider how faults information will be

s/faults/fault/

   Should the event reporting provide gyaranteed accurate delivery of
s/gyaranteed/guaranteed/

===

4.3.3.  Fault Isolation

   It might be useful to isolate faults, such as a system that emits
   malformed messages necessary to coordinate connections properly.
   Spanning tree comes to mind.  This might be able to be done by
   configuring next-hop devices to drop the faulty messages to prevent
   them from entering the rest of the network.

This is not what *I* mean by fault isolation!
Fault isolation is about working out where in the network the fault
is. For example, if end-to-end data delivery is failing (reported by
a notification), fault isolation will find the failed link or node
in the end-to-end path.

You seem to be describing the process of quarantining faulty
devices. This may also be valuable.

"Spanning tree comes to mind" is a rather strange sentence without
much context.

===

4.4.  Configuration Management

   A protocol desoigners should document what the basic configuration

s/desoigners/designers/

   "Requirements for Configuration Management of IP-based Networks"
   [RFC3139] discusses requirements for configuration management.
More precisely,
   "Requirements for Configuration Management of IP-based Networks"
   [RFC3139] discusses requirements for configuration management of
   IP-based networks.
Which makes me wonder about the purpose of the sentence :-)

   This document includes...
This document or that document?

   A number of efforts have existed in the IETF to develop policy-based
   management.  "Terminology for Policy-Based Management" [RFC3198] was
   written to standardize the terminology across these efforts.  Some of
   the efforts resulted in proposals that are NOT RECOMMENDED, so a
   protocol designer should check the current recommendation status
   before depending on a specific protocol suggestion.

You have used RFC2119 language, but in section 1.1 you said
   This document deliberately does not use the (capitalized) keywords
   described in RFC 2119 [RFC2119].
Furthermore, it is not clear to me whether this I-D is doing the
NOT RECOMMENDing or whether some other RFC has done it. If it is
this I-D, you will need to give reasons. If is some other RFC, you
will need to give references.

   It is highly desirable that text processing tools such as diff, and
   version management tools such as RCS or CVS or SVN, can be used to
   process configurations.  This approach simplifies comparing the
   current operational state to the initial configuration.  It is
   commonplace to compare configuration changes to e.g., last day, last
   week, last month, etc. -- having configuration in a text, and human-
   understandable format is very valuable for various reasons such as
   change control (or verification), configuration consistency checks,
   etc.

Yes, but what has this to do with the protocol design? It might be
related to device implementation. Or it might be relevant to
EMS/NMS implementation.

As you note in the subsequent paragraph, this may be impossible
even in some text-based DMs. It would certainly be impossible in
an SNMP MIB module.

   To simplify such configuration comparisons, devices should not
   arbitrarily reorder data such as access control lists.  If a protocol
   designer defines mechanisms for configuration, it would be desirable
   to standardize the order of elements for consistency of configuration
   and of reporting across vendors, and across releases from vendors.

This example seems a bit specific. And in any case, it is perfectly
possible to sort a list before comparing.

   There are two parts to this: 1.  An NMS system could optimize ACLs
   for performance reasons 2.  Unless the device/NMS systems has correct
   rules/a lot of experience, reordering ACLs can lead to a huge
   security issue.

Maybe what you are saying is:
   Implementations should not arbitrarily modify configuration data.
   In some cases (such as Access Control Lists) the order of data
   items is significant and comprises part of the configured data.

   Network wide configurations are ideally stored in central master
   databases

This looks like an unsubstantiated assertion. You might get away
with "Many operators prefer that..."

   It is important to distinguish between the distribution of
   configurations and the activation of a certain configuration.
   Devices should be able to hold multiple configurations.

This is really an implementation issue and nothing to do with the
protocol under design.

   Consensus of the 2002 IAB Workshop was that textual configuration
   files should be able to contain international characters.

Do you have a citation?

   These timers may need default values
   suggested in the protocol specification and do not need to be
   otherwise configurable.

I had formed the opinion that timers must be given default values
in IETF protocol specifications, but I can't lay my hands on a
reference.

===

4.5.  Accounting Management

   A protocol designer should consider whether it would be appropriate
   to collect usage information related to this protocol, and if so,
   what usage information would be appropriate to collect?

s/collect?/collect./

   "Introduction to Accounting Management" [RFC2975] discusses a number
   of factors relevant to monitoring usage of protocols for purposes of
   capacity and trend analysis, cost allocation, auditing, and billing.
   This document also discusses how Remote Authentication Dial In User
This or that?

   Service (RADIUS) [RFC2865], Terminal Access Controller Access Control
   System (TACACS) [RFC1492], SNMP, IPFIX, and Packet Sampling (PSAMP)
   Protocol [I-D.ietf-psamp-protocol] protocols can be used for these
   purposes.

Surely RFC 2975 does not mention PSAMP? Which makes me wonder what
you are trying to achieve with a list of example protocols. Are
they intended as recommendations (in which case say so and why),
or as a non-exclusive list (in which case a "such as" would be
useful). At the moment it reads like these are your favorites and
you would be upset if someone used a different protocol!

===

4.6.  Performance Management

   There are several parts to performance management to be considered:
   protocol monitoring, services monitoring, and device monitoring (the
   impact of the new protocol/service activation on the device).

s/services/service/

   Consider scalability, such as whether performance will be affected by
   the number of protocol connections.

Maybe say "protocol entities" rather than connections?


This section seems to flit between the performance of the protocol
under design and the network running the protocol under design, and
the performance of the management of the protocol. Each is
important, but it may be constructive to split them out into
separate sections. The latter will also include a consideration of
how the act of monitoring impacts on network performance.


   The Benchmarking Methodology WG (bmwg) has defined recommendations
Later you have BMWG.

===

4.7.  Security Management

   It must be possible to do consistency checks of access control lists
   across devices.  Protocol designers should consider information
   models to promote comparisons across devices and across vendors to
   permit checking the consistency of security configurations.

I think this "must" is massively excessive!
I would expect each router to have a different access list in the
control plane (a list of its neighbors). Such a mechanism would
actually open up a security vulnerability that has to be protected.
Perhaps you are talking about management plane access lists. This
takes us into the area of securing management activity - see my
comments on Section 7.

   RADIUS and DIAMETER can provide authentication and authorization.  A
   protocol designer should consider which attributes would be
   appropriate for their protocol.

This paragraph is apropos of what? What are you trying to tell me?
Should I run my management using Radius? Should I try to build
Diameter into my new protocol?

   Different protocols use different assumptions about message security
s/protocols/management protocols/

   and data access controls.  A protocol designer that recommends using
   different protocols should consider how security will be applied in a
s/protocols/management protocols/


Although you talk about (presumably management) access lists, I think
I would have expected a discussion of authority levels and policy
with regard to management tasks.

===

7.  Security Considerations

   This document is informational and provides guidelines for
   considering manageability and operations.  It introduces no new
   security concerns.

I think that, although you introduce no new security concerns
directly, you have some things to talk about.

Firstly, the provision of a management portal to a network device
provides a doorway through which an attack on the device may be
launched. Since you are recommending that the protocol under
development be manageable through the management protocol, you are
requiring that the protocol be vulnerable to a new source of
attacks. Therefore, you must talk about securing the protocol
against these attacks. This might be as simple as saying that only
management protocols with adequate (for some definition of
adequate) security apparatus be used.

I think it is also of note that a standard description of the
manageable knobs and whistles on a protocol makes it easier for an
attacker to understand what they may try to control and how to
tweak it.

On the other hand, it would be well worth pointing out that a
well-designed protocol is usually more stable and secure. And that
a protocol that can be managed and inspected offers the operator
a better chance of spotting and quarantining any attacks.
Conversely making a protocol inspectable, is a risk if the wrong
person inspects it!

If security events cause logs and or notifications/alerts, can
a concerted attack be mounted by causing an excess of these
events? In other words, to the security management mechanisms
constitute the security vulnerability?

Lastly, the management of security aspects is important, and you
can just reference back to section 4.7.

===

9.  Informative References

I (of course) would favor an informational reference to draft-ietf-pce-
manageability-requirements-06.txt

===

   [RFC2821]                       Klensin, J., "Simple Mail Transfer
                                   Protocol", RFC 2821, April 2001.

I think this has been obsoleted.

===

Appendix A.  Operations and Management Review Checklist

Maybe a forward reference to this appendix from sections 1 and 5?

===

A.3.  Documentation

   Does the proposed protocol have a significant operational impact on
   the Internet.  If it does, and the document under review targets
   standards track, is their enough proof of implementation and/or
   operational experience to grant Proposed Standard status?

Whoah!

You are going to use this I-D to try to block standards track I-Ds
from becoming RFCs? You are going to get into such a fight!!!

IMHO this sentence is a complete show-stopper in your I-D.

(Oh, and s/their/there/)

===

Appendix B.  Change Log

Suggest you flag this for removal upon publication as an RFC.

===

> Sent: Sunday, January 25, 2009 6:40 PM
> To: [e-mail]
> Subject: [OPSAWG] (final) WGLC on
> draft-ietf-opsawg-operations-and-management
>
> This is a (hopefully final) WGLC for
> draft-ietf-opsawg-operations-and-management
> there was not enough respose to the last one back in October
> please let the list know what you think of the draft