Re: [OPSAWG] (final) WGLC ondraft-ietf-opsawg-operations-and-management

["tom.petch" <[e-mail]>]

I was going to say +1 to Juergen's comments and then I was going to say +1 to
Mike's comments and then I was going to say +1 to Adrian's comments and while I
was pondering, ... I left it too late:-(

But I do echo much of what has been said.  I cannot see that this I-D lives up
to its billing.  It is a good idea, and it is good stuff (give or take some
infelicities of wording, most of which have been pointed out already) but I do
not see it as good stuff for a working group coming up with a new protocol, say
an MPLS-TP or sieve or sidr. And that is what I think is needed.  If something
is not designed to be managed ab initio, then it may be costly, impossible even,
to manage it in the wild.

Tom Petch


----- Original Message -----
From: "Adrian Farrel" <[e-mail]>
To: <[e-mail]>
Cc: "David B Harrington" <[e-mail]>; <[e-mail]>
Sent: Wednesday, February 04, 2009 8:54 PM
Subject: Re: [OPSAWG] (final) WGLC ondraft-ietf-opsawg-operations-and-management


> Hi,
>
> Dan asked me to have a look at this which I do primarily with a routing area
> view on the world.
>
> I'm sorry that I haven't been following this I-D very closely and apologise
> if you have already discussed and reached conclusion on some of the points I
> make.
>
> I am largely supportive of this work (see
> draft-ietf-pce-manageability-requirements-06.txt), but I do have some
> issues. I would be comforted to know that the I-D had had extensive review
> from within the Ops Area without relying just on my comments!
>
> I would also like to understand how the recommendations parts of the draft
> will be interpreted by the Ops ADs as they perform IESG review of I-Ds from
> other areas. Depending on the answer (which I would like to see encapsulated
> in the document!) I might have other harsher comments about the content.
>
> More details below.
>
> Cheers,
> Adrian
>
> ===
> Title is...
>  Guidelines for Considering Operations and Management of New Protocols
> ...but the Abstract immediately includes...
>    New protocols or protocol extensions
> ===
> Obviously needs the new IPR boilerplate.
> ===
> Abstract line 3
> s/protocol/protocols/
> ===
> I am somewhat worried by "recommend appropriate standard management
> protocols ... to address the relevant operations and management
> needs."
>
> I understand the theory behind this guidance in terms of making
> equipment that will interwork with management systems, but I
> question its practicality given the vast array of management
> protocols available. And actually, I am not sure that this has a
> direct impact on the success of or development of a protocol.
>
> The introduction says that this "is similar to a WG considering
> which security threats are relevant to their protocol, and then
> recommending appropriate standard security protocols to mitigate
> the relevant threats." I dispute this. The security considerations
> normally make modifications to the protocol under development to
> make it secure, or mandate security features in other protocols
> that the protocol under development uses. But where there is a
> need for an external protocol, this is not usually mandated. For
> example, where key exchange is needed the text will usually say
> something like "may use some form of secure key exchange protocol
> (such as the Internet Key Exchange protocol v2 (IKE) [RFC4306]."
> This is far looser than a recommendation of a specific protocol.
>
> I believe that in practice operators want to go on using the
> management protocols that they already have deployed. If, for
> example, I have deployed routers that I manage using SNMP, I will
> not want to deploy a new routing protocol on those routers if that
> protocol is strongly recommended to be managed using TL1.
>
> Since data modeling is usually tied closely to the protocol used
> for management, it follows from this that developing specific
> data formats (SNMP MIB modules, XML schema, etc.) might also not
> be so very practical. (Arguably we have found this out with MIB
> modules over the last 10 years, but still churn them out.)
>
> What *would* be beneficial, however, is an abstract data model
> for management of the protocol. There are many ways of
> representing the data (XML, BNF, ASN.1, graphical representation,
> etc.) and the Ops Area could usefully pick one and standardise
> it. From there we would have a solid data set that all
> implementations could make available and which separate documents
> could easily map to suitable formats for use by specific
> management protocols. Smart implementers would be able to support
> multiple management protocols with ease.
>
> You do come to this to some extent in section 4.1, and the text
> there seems to be at odds with what you have said in the
> Introduction.
>
> ===
> Section 1
>
> Can you expand "WG" on first usage?
>
> ===
> Section 1
>
>    We want to avoid seeming to impose a solution by putting in place a
>    strict terminology - for example implying that a formal data model,
>    or even using a management protocol is mandatory.
>
> <grin mode=wry>
>
> You want to avoid seeming to impose it?
> So, you want to impose it, but you don't want anyone to notice?
>
> </grin>
>
> ===
> Section 1
>
>    Making a Management Considerations section a mandatory publication
>    requirement for IETF documents is the responsibility of the IESG, or
>    specific area directors, or working groups, and this document avoids
>    recommending any mandatory publication requirements.
>
> I think you may need to go a step further to get this through the
> IESG. It would not be funny to see this paragraph as written, but
> to get a DISCUSS on an I-D from an Ops AD because the I-D did not
> contain a Management Considerations section.
>
> ===
> Introduction
>
>    We provide some objective criteria to promote interoperability
>
> <grin mode=laconic>
>
> There seems to be only one author. Are you royalty?
> Perhaps... "This document provides..."
>
> </grin>
>
> ===
> Section 1 final line
>
> s/??/?"/
>
> ===
>
> 2.1.  IETF Management Framework
>
>    For years the IETF has stressed the use of the IETF Standard
>    Management Framework and MIB modules [RFC2578] for managing new
>    protocols.  The IETF designed these to permit multiple protocols to
>    utilize the MIB data [RFC1052], but it became a common
>
> s/became/is/
>
> ===
>
> 3.1.  Operations Model
>
>    o  what are the typical user interfaces - Command line (CLI) or
>       graphical user interface (GUI)?
>
> I am not sure that this last item is particularly relevant for a
> large number of the protocols designed in the IETF. We might say
> that the question is valid for a protocol such as FTP, but is it
> relevant for OSPF? Yes, there is a configuration interface to a
> device that runs OSPF, but it is surely nothing to do with the
> protocol design how that interface works. Indeed, you might
> reduce the question to: are the configuration interfaces local or
> remote? But I think you will find that the answer to the question
> is "both" as in the subsequent paragraph.
>
> ===
>
> 3.1.  Operations Model
>
>    Auto-configuration and default parameters might be
>    possible for some new protocols.
>
> I would make a much bigger noise about default parameters. There is
> a key question about how the protocol can operate "out of the box".
> If implementers are free to select their own defaults, the protocol
> needs to operate well with any choice of values. If there are
> sensible defaults, these need to be stated.
>
> Indeed, section 3.2 seems to say this. Perhaps a forward reference?
>
> ===
>
> 3.1.  Operations Model
>
>    There may be a need to support a human interface, e.g., for
>    troubleshooting, and a programmatic interface, e.g., for automated
>    monitoring and root cause analysis.  It might be important that the
>    internal method routines used by the application programming
>    interfaces and the human interfaces should be the same to ensure that
>    data exchanged between these two interfaces is always consistent.
>    Mixing methods leads to inconsistency, so identifying consistent
>    methods of retrieving information is relevant.
>
> This is really sailing too close to the implementation. People
> must be free to construct sub-standard and unmarketable
> implementations. The IETF is not responsible for making all
> implementations successful. The job is to make it possible for
> all implementations to be successful given core competences in the
> developers.
>
> I think the furthest a protocol spec RFC could go would be to say
> what information must be available to an operator at a human
> interface for troubleshooting
>
> ===
>
> 3.2.  Installation and Initial Setup
>
> In view of the reasons you give why defaults might become out-
> dated, you might point out that sometimes it is not advisable to
> have defaults. In these cases explicit configuration is required.
>
> ===
>
> 3.2.  Installation and Initial Setup
>
> Is the text at the top of page 8 supposed to be a bullet list?
>
> ===
>
> 3.4.  Requirements on Other Protocols and Functional Components
>
>    These considerations should generally remain illustrative to avoid
>    creating restrictions or dependencies, or potentially impacting the
>    behavior of existing protocols, or restricting the extensibility of
>    other protocols, or assuming other protocols will not be extended in
>    certain ways.
>
> Well, this is quite interesting.
>
> Suppose you decide to utilise a particular feature of a transport
> protocol. You are creating a hard requirement that that feature is
> implemented in all implementations of the set {your new protocol,
> the transport protocol}. But more importantly, perhaps, you are
> requiring that new versions of the transport protocol do not
> deprecate the feature you require.
>
> In short, I don't think you should be too nervous about creating
> restrictions or dependencies. If they exist, they must be stated.
>
> ===
>
> 3.2.  Installation and Initial Setup
>
>    For example, the design of Resource ReSerVation Protocol (RSVP)
>
> I think this example is not well expressed.
>
> The problem in RSVP was that the IP packet was addressed to the
> ultimate RSVP-capable destination, but the RSVP-capable routers
> along the path needed to intercept and process the Path message.
> This they could only do by "deep packet inspection" or by the
> router alert option.
>
> I wonder whether this example adds significantly to the I-D.
>
> ===
>
> 4.  Management Considerations
>
>    NETCONF addresses this in a generic manner by
>
> Add citation.
>
> ===
>
> 4.  Management Considerations
>
>    However, debugging on-the-wire interactions is a protocol
>    issue: it is enormously helpful if a protocol has hooks to make
>    debugging of network interactions easy, and/or is designed in such a
>    way that debugging protocol behaviors is easy.  Hand-waving this away
>    is not something that operators like ...
>
> I think you are including this by hand-waving!
> What do you mean by "hooks"?
> Surely any protocol can be debugged simply by sniffing it.
> If you have specific hooks in mind you must set them out and not
> leave the reader guessing.
> But if you force the implementation I shall be peeved!
> You might, for example, say that there should be alerts issued when
> messages are received, or when there are state transitions in the
> protocol state machine, but you must recall that:
> - the messages can be seen by sniffing
> - the state machine is not part of the protocol specification.
> This latter point is key. The state machine explains how the
> protocol works in order that an implementer can decide how to
> react to a received event, but it does not require that the
> machine is implemented.
>
> ===
>
> 4.1.  Interoperability
>
>    Interoperability needs to be considered on the syntactic level and
>    the semantic level.  While it can be irritating and time-consuming,
>    application designers including operators who write their own scripts
>    can make their processing conditional to accommodate differences
>
> s/differences/syntactic differences/ ?
>
>    Then, whether the counter is gathered via SNMP or a CLI
>    command or a SYSLOG message, the counter will have similar meaning.
>
> "similar"? I think that might not be good enough. Don't you need
> "the same"?
>
>    o  [RFC3805] - Printer MIB v2 (contains both an IM and a DM
>
> Missing ")"
>
> ===
>
> 4.2.  Management Information
>
> I would find it really helpful if you were able to recommend a
> language in which to write the information model. Ideally, all
> IETF IMs would be in the same language.
>
>    It is important to be able to separately fetch configuration data,
>    operational state data, and statistics from devices, and to be able
>    to compare current state to initial state, and to compare data
>    between devices.
>
> You seem to have drifted into DMs here. You cannot fetch data from
> an IM. It simply explains what data elements exist. In fact, an IM
> states all of the data that exists and so there is no confusion.
> Confusion can only arise as the DM is constructed and multiple IM
> data elements are conflated into single DM objects.
>
> ===
>
> 4.3.  Fault Management
>
>    The protocol designer should document the basic faults and health
>    indicators that need to be instrumented for the new protocol, and the
>    alarms and events that must be propagated to management applications
>    or exposed through a data model.
>
> Isn't "propagated to management applications" a subset of "exposed
> through a data model"?
>
>    The protocol designer should consider how faults information will be
>
> s/faults/fault/
>
>    Should the event reporting provide gyaranteed accurate delivery of
> s/gyaranteed/guaranteed/
>
> ===
>
> 4.3.3.  Fault Isolation
>
>    It might be useful to isolate faults, such as a system that emits
>    malformed messages necessary to coordinate connections properly.
>    Spanning tree comes to mind.  This might be able to be done by
>    configuring next-hop devices to drop the faulty messages to prevent
>    them from entering the rest of the network.
>
> This is not what *I* mean by fault isolation!
> Fault isolation is about working out where in the network the fault
> is. For example, if end-to-end data delivery is failing (reported by
> a notification), fault isolation will find the failed link or node
> in the end-to-end path.
>
> You seem to be describing the process of quarantining faulty
> devices. This may also be valuable.
>
> "Spanning tree comes to mind" is a rather strange sentence without
> much context.
>
> ===
>
> 4.4.  Configuration Management
>
>    A protocol desoigners should document what the basic configuration
>
> s/desoigners/designers/
>
>    "Requirements for Configuration Management of IP-based Networks"
>    [RFC3139] discusses requirements for configuration management.
> More precisely,
>    "Requirements for Configuration Management of IP-based Networks"
>    [RFC3139] discusses requirements for configuration management of
>    IP-based networks.
> Which makes me wonder about the purpose of the sentence :-)
>
>    This document includes...
> This document or that document?
>
>    A number of efforts have existed in the IETF to develop policy-based
>    management.  "Terminology for Policy-Based Management" [RFC3198] was
>    written to standardize the terminology across these efforts.  Some of
>    the efforts resulted in proposals that are NOT RECOMMENDED, so a
>    protocol designer should check the current recommendation status
>    before depending on a specific protocol suggestion.
>
> You have used RFC2119 language, but in section 1.1 you said
>    This document deliberately does not use the (capitalized) keywords
>    described in RFC 2119 [RFC2119].
> Furthermore, it is not clear to me whether this I-D is doing the
> NOT RECOMMENDing or whether some other RFC has done it. If it is
> this I-D, you will need to give reasons. If is some other RFC, you
> will need to give references.
>
>    It is highly desirable that text processing tools such as diff, and
>    version management tools such as RCS or CVS or SVN, can be used to
>    process configurations.  This approach simplifies comparing the
>    current operational state to the initial configuration.  It is
>    commonplace to compare configuration changes to e.g., last day, last
>    week, last month, etc. -- having configuration in a text, and human-
>    understandable format is very valuable for various reasons such as
>    change control (or verification), configuration consistency checks,
>    etc.
>
> Yes, but what has this to do with the protocol design? It might be
> related to device implementation. Or it might be relevant to
> EMS/NMS implementation.
>
> As you note in the subsequent paragraph, this may be impossible
> even in some text-based DMs. It would certainly be impossible in
> an SNMP MIB module.
>
>    To simplify such configuration comparisons, devices should not
>    arbitrarily reorder data such as access control lists.  If a protocol
>    designer defines mechanisms for configuration, it would be desirable
>    to standardize the order of elements for consistency of configuration
>    and of reporting across vendors, and across releases from vendors.
>
> This example seems a bit specific. And in any case, it is perfectly
> possible to sort a list before comparing.
>
>    There are two parts to this: 1.  An NMS system could optimize ACLs
>    for performance reasons 2.  Unless the device/NMS systems has correct
>    rules/a lot of experience, reordering ACLs can lead to a huge
>    security issue.
>
> Maybe what you are saying is:
>    Implementations should not arbitrarily modify configuration data.
>    In some cases (such as Access Control Lists) the order of data
>    items is significant and comprises part of the configured data.
>
>    Network wide configurations are ideally stored in central master
>    databases
>
> This looks like an unsubstantiated assertion. You might get away
> with "Many operators prefer that..."
>
>    It is important to distinguish between the distribution of
>    configurations and the activation of a certain configuration.
>    Devices should be able to hold multiple configurations.
>
> This is really an implementation issue and nothing to do with the
> protocol under design.
>
>    Consensus of the 2002 IAB Workshop was that textual configuration
>    files should be able to contain international characters.
>
> Do you have a citation?
>
>    These timers may need default values
>    suggested in the protocol specification and do not need to be
>    otherwise configurable.
>
> I had formed the opinion that timers must be given default values
> in IETF protocol specifications, but I can't lay my hands on a
> reference.
>
> ===
>
> 4.5.  Accounting Management
>
>    A protocol designer should consider whether it would be appropriate
>    to collect usage information related to this protocol, and if so,
>    what usage information would be appropriate to collect?
>
> s/collect?/collect./
>
>    "Introduction to Accounting Management" [RFC2975] discusses a number
>    of factors relevant to monitoring usage of protocols for purposes of
>    capacity and trend analysis, cost allocation, auditing, and billing.
>    This document also discusses how Remote Authentication Dial In User
> This or that?
>
>    Service (RADIUS) [RFC2865], Terminal Access Controller Access Control
>    System (TACACS) [RFC1492], SNMP, IPFIX, and Packet Sampling (PSAMP)
>    Protocol [I-D.ietf-psamp-protocol] protocols can be used for these
>    purposes.
>
> Surely RFC 2975 does not mention PSAMP? Which makes me wonder what
> you are trying to achieve with a list of example protocols. Are
> they intended as recommendations (in which case say so and why),
> or as a non-exclusive list (in which case a "such as" would be
> useful). At the moment it reads like these are your favorites and
> you would be upset if someone used a different protocol!
>
> ===
>
> 4.6.  Performance Management
>
>    There are several parts to performance management to be considered:
>    protocol monitoring, services monitoring, and device monitoring (the
>    impact of the new protocol/service activation on the device).
>
> s/services/service/
>
>    Consider scalability, such as whether performance will be affected by
>    the number of protocol connections.
>
> Maybe say "protocol entities" rather than connections?
>
>
> This section seems to flit between the performance of the protocol
> under design and the network running the protocol under design, and
> the performance of the management of the protocol. Each is
> important, but it may be constructive to split them out into
> separate sections. The latter will also include a consideration of
> how the act of monitoring impacts on network performance.
>
>
>    The Benchmarking Methodology WG (bmwg) has defined recommendations
> Later you have BMWG.
>
> ===
>
> 4.7.  Security Management
>
>    It must be possible to do consistency checks of access control lists
>    across devices.  Protocol designers should consider information
>    models to promote comparisons across devices and across vendors to
>    permit checking the consistency of security configurations.
>
> I think this "must" is massively excessive!
> I would expect each router to have a different access list in the
> control plane (a list of its neighbors). Such a mechanism would
> actually open up a security vulnerability that has to be protected.
> Perhaps you are talking about management plane access lists. This
> takes us into the area of securing management activity - see my
> comments on Section 7.
>
>    RADIUS and DIAMETER can provide authentication and authorization.  A
>    protocol designer should consider which attributes would be
>    appropriate for their protocol.
>
> This paragraph is apropos of what? What are you trying to tell me?
> Should I run my management using Radius? Should I try to build
> Diameter into my new protocol?
>
>    Different protocols use different assumptions about message security
> s/protocols/management protocols/
>
>    and data access controls.  A protocol designer that recommends using
>    different protocols should consider how security will be applied in a
> s/protocols/management protocols/
>
>
> Although you talk about (presumably management) access lists, I think
> I would have expected a discussion of authority levels and policy
> with regard to management tasks.
>
> ===
>
> 7.  Security Considerations
>
>    This document is informational and provides guidelines for
>    considering manageability and operations.  It introduces no new
>    security concerns.
>
> I think that, although you introduce no new security concerns
> directly, you have some things to talk about.
>
> Firstly, the provision of a management portal to a network device
> provides a doorway through which an attack on the device may be
> launched. Since you are recommending that the protocol under
> development be manageable through the management protocol, you are
> requiring that the protocol be vulnerable to a new source of
> attacks. Therefore, you must talk about securing the protocol
> against these attacks. This might be as simple as saying that only
> management protocols with adequate (for some definition of
> adequate) security apparatus be used.
>
> I think it is also of note that a standard description of the
> manageable knobs and whistles on a protocol makes it easier for an
> attacker to understand what they may try to control and how to
> tweak it.
>
> On the other hand, it would be well worth pointing out that a
> well-designed protocol is usually more stable and secure. And that
> a protocol that can be managed and inspected offers the operator
> a better chance of spotting and quarantining any attacks.
> Conversely making a protocol inspectable, is a risk if the wrong
> person inspects it!
>
> If security events cause logs and or notifications/alerts, can
> a concerted attack be mounted by causing an excess of these
> events? In other words, to the security management mechanisms
> constitute the security vulnerability?
>
> Lastly, the management of security aspects is important, and you
> can just reference back to section 4.7.
>
> ===
>
> 9.  Informative References
>
> I (of course) would favor an informational reference to draft-ietf-pce-
> manageability-requirements-06.txt
>
> ===
>
>    [RFC2821]                       Klensin, J., "Simple Mail Transfer
>                                    Protocol", RFC 2821, April 2001.
>
> I think this has been obsoleted.
>
> ===
>
> Appendix A.  Operations and Management Review Checklist
>
> Maybe a forward reference to this appendix from sections 1 and 5?
>
> ===
>
> A.3.  Documentation
>
>    Does the proposed protocol have a significant operational impact on
>    the Internet.  If it does, and the document under review targets
>    standards track, is their enough proof of implementation and/or
>    operational experience to grant Proposed Standard status?
>
> Whoah!
>
> You are going to use this I-D to try to block standards track I-Ds
> from becoming RFCs? You are going to get into such a fight!!!
>
> IMHO this sentence is a complete show-stopper in your I-D.
>
> (Oh, and s/their/there/)
>
> ===
>
> Appendix B.  Change Log
>
> Suggest you flag this for removal upon publication as an RFC.
>
> ===
>
> > Sent: Sunday, January 25, 2009 6:40 PM
> > To: [e-mail]
> > Subject: [OPSAWG] (final) WGLC on
> > draft-ietf-opsawg-operations-and-management
> >
> > This is a (hopefully final) WGLC for
> > draft-ietf-opsawg-operations-and-management
> > there was not enough respose to the last one back in October
> > please let the list know what you think of the draft
>
> _______________________________________________
> OPSAWG mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/opsawg