Re: [OPSAWG] (final) WGLC ondraft-ietf-opsawg-operations-and-management

["David Harrington" <[e-mail]>]

Wow! Adrian wants me to do some work!
Thanks for the thorough review.

dbh 

> -----Original Message-----
> From: [e-mail] 
> [mailto:[e-mail]] On Behalf Of Adrian Farrel
> Sent: Wednesday, February 04, 2009 2:55 PM
> To: [e-mail]
> Cc: David B Harrington; [e-mail]
> Subject: Re: [OPSAWG] (final) WGLC 
> ondraft-ietf-opsawg-operations-and-management
> 
> Hi,
> 
> Dan asked me to have a look at this which I do primarily with 
> a routing area 
> view on the world.
> 
> I'm sorry that I haven't been following this I-D very closely 
> and apologise 
> if you have already discussed and reached conclusion on some 
> of the points I 
> make.
> 
> I am largely supportive of this work (see 
> draft-ietf-pce-manageability-requirements-06.txt), but I do have
some 
> issues. I would be comforted to know that the I-D had had 
> extensive review 
> from within the Ops Area without relying just on my comments!
> 
> I would also like to understand how the recommendations parts 
> of the draft 
> will be interpreted by the Ops ADs as they perform IESG 
> review of I-Ds from 
> other areas. Depending on the answer (which I would like to 
> see encapsulated 
> in the document!) I might have other harsher comments about 
> the content.
> 
> More details below.
> 
> Cheers,
> Adrian
> 
> ===
> Title is...
>  Guidelines for Considering Operations and Management of New
Protocols
> ...but the Abstract immediately includes...
>    New protocols or protocol extensions
> ===
> Obviously needs the new IPR boilerplate.
> ===
> Abstract line 3
> s/protocol/protocols/
> ===
> I am somewhat worried by "recommend appropriate standard management
> protocols ... to address the relevant operations and management
> needs."
> 
> I understand the theory behind this guidance in terms of making
> equipment that will interwork with management systems, but I
> question its practicality given the vast array of management
> protocols available. And actually, I am not sure that this has a
> direct impact on the success of or development of a protocol.
> 
> The introduction says that this "is similar to a WG considering
> which security threats are relevant to their protocol, and then
> recommending appropriate standard security protocols to mitigate
> the relevant threats." I dispute this. The security considerations
> normally make modifications to the protocol under development to
> make it secure, or mandate security features in other protocols
> that the protocol under development uses. But where there is a
> need for an external protocol, this is not usually mandated. For
> example, where key exchange is needed the text will usually say
> something like "may use some form of secure key exchange protocol
> (such as the Internet Key Exchange protocol v2 (IKE) [RFC4306]."
> This is far looser than a recommendation of a specific protocol.
> 
> I believe that in practice operators want to go on using the
> management protocols that they already have deployed. If, for
> example, I have deployed routers that I manage using SNMP, I will
> not want to deploy a new routing protocol on those routers if that
> protocol is strongly recommended to be managed using TL1.
> 
> Since data modeling is usually tied closely to the protocol used
> for management, it follows from this that developing specific
> data formats (SNMP MIB modules, XML schema, etc.) might also not
> be so very practical. (Arguably we have found this out with MIB
> modules over the last 10 years, but still churn them out.)
> 
> What *would* be beneficial, however, is an abstract data model
> for management of the protocol. There are many ways of
> representing the data (XML, BNF, ASN.1, graphical representation,
> etc.) and the Ops Area could usefully pick one and standardise
> it. From there we would have a solid data set that all
> implementations could make available and which separate documents
> could easily map to suitable formats for use by specific
> management protocols. Smart implementers would be able to support
> multiple management protocols with ease.
> 
> You do come to this to some extent in section 4.1, and the text
> there seems to be at odds with what you have said in the
> Introduction.
> 
> ===
> Section 1
> 
> Can you expand "WG" on first usage?
> 
> ===
> Section 1
> 
>    We want to avoid seeming to impose a solution by putting in place
a
>    strict terminology - for example implying that a formal data
model,
>    or even using a management protocol is mandatory.
> 
> <grin mode=wry>
> 
> You want to avoid seeming to impose it?
> So, you want to impose it, but you don't want anyone to notice?
> 
> </grin>
> 
> ===
> Section 1
> 
>    Making a Management Considerations section a mandatory
publication
>    requirement for IETF documents is the responsibility of 
> the IESG, or
>    specific area directors, or working groups, and this 
> document avoids
>    recommending any mandatory publication requirements.
> 
> I think you may need to go a step further to get this through the
> IESG. It would not be funny to see this paragraph as written, but
> to get a DISCUSS on an I-D from an Ops AD because the I-D did not
> contain a Management Considerations section.
> 
> ===
> Introduction
> 
>    We provide some objective criteria to promote interoperability
> 
> <grin mode=laconic>
> 
> There seems to be only one author. Are you royalty?
> Perhaps... "This document provides..."
> 
> </grin>
> 
> ===
> Section 1 final line
> 
> s/??/?"/
> 
> ===
> 
> 2.1.  IETF Management Framework
> 
>    For years the IETF has stressed the use of the IETF Standard
>    Management Framework and MIB modules [RFC2578] for managing new
>    protocols.  The IETF designed these to permit multiple protocols
to
>    utilize the MIB data [RFC1052], but it became a common
> 
> s/became/is/
> 
> ===
> 
> 3.1.  Operations Model
> 
>    o  what are the typical user interfaces - Command line (CLI) or
>       graphical user interface (GUI)?
> 
> I am not sure that this last item is particularly relevant for a
> large number of the protocols designed in the IETF. We might say
> that the question is valid for a protocol such as FTP, but is it
> relevant for OSPF? Yes, there is a configuration interface to a
> device that runs OSPF, but it is surely nothing to do with the
> protocol design how that interface works. Indeed, you might
> reduce the question to: are the configuration interfaces local or
> remote? But I think you will find that the answer to the question
> is "both" as in the subsequent paragraph.
> 
> ===
> 
> 3.1.  Operations Model
> 
>    Auto-configuration and default parameters might be
>    possible for some new protocols.
> 
> I would make a much bigger noise about default parameters. There is
> a key question about how the protocol can operate "out of the box".
> If implementers are free to select their own defaults, the protocol
> needs to operate well with any choice of values. If there are
> sensible defaults, these need to be stated.
> 
> Indeed, section 3.2 seems to say this. Perhaps a forward reference?
> 
> ===
> 
> 3.1.  Operations Model
> 
>    There may be a need to support a human interface, e.g., for
>    troubleshooting, and a programmatic interface, e.g., for
automated
>    monitoring and root cause analysis.  It might be important that
the
>    internal method routines used by the application programming
>    interfaces and the human interfaces should be the same to 
> ensure that
>    data exchanged between these two interfaces is always consistent.
>    Mixing methods leads to inconsistency, so identifying consistent
>    methods of retrieving information is relevant.
> 
> This is really sailing too close to the implementation. People
> must be free to construct sub-standard and unmarketable
> implementations. The IETF is not responsible for making all
> implementations successful. The job is to make it possible for
> all implementations to be successful given core competences in the
> developers.
> 
> I think the furthest a protocol spec RFC could go would be to say
> what information must be available to an operator at a human
> interface for troubleshooting
> 
> ===
> 
> 3.2.  Installation and Initial Setup
> 
> In view of the reasons you give why defaults might become out-
> dated, you might point out that sometimes it is not advisable to
> have defaults. In these cases explicit configuration is required.
> 
> ===
> 
> 3.2.  Installation and Initial Setup
> 
> Is the text at the top of page 8 supposed to be a bullet list?
> 
> ===
> 
> 3.4.  Requirements on Other Protocols and Functional Components
> 
>    These considerations should generally remain illustrative to
avoid
>    creating restrictions or dependencies, or potentially impacting
the
>    behavior of existing protocols, or restricting the extensibility
of
>    other protocols, or assuming other protocols will not be 
> extended in
>    certain ways.
> 
> Well, this is quite interesting.
> 
> Suppose you decide to utilise a particular feature of a transport
> protocol. You are creating a hard requirement that that feature is
> implemented in all implementations of the set {your new protocol,
> the transport protocol}. But more importantly, perhaps, you are
> requiring that new versions of the transport protocol do not
> deprecate the feature you require.
> 
> In short, I don't think you should be too nervous about creating
> restrictions or dependencies. If they exist, they must be stated.
> 
> ===
> 
> 3.2.  Installation and Initial Setup
> 
>    For example, the design of Resource ReSerVation Protocol (RSVP)
> 
> I think this example is not well expressed.
> 
> The problem in RSVP was that the IP packet was addressed to the
> ultimate RSVP-capable destination, but the RSVP-capable routers
> along the path needed to intercept and process the Path message.
> This they could only do by "deep packet inspection" or by the
> router alert option.
> 
> I wonder whether this example adds significantly to the I-D.
> 
> ===
> 
> 4.  Management Considerations
> 
>    NETCONF addresses this in a generic manner by
> 
> Add citation.
> 
> ===
> 
> 4.  Management Considerations
> 
>    However, debugging on-the-wire interactions is a protocol
>    issue: it is enormously helpful if a protocol has hooks to make
>    debugging of network interactions easy, and/or is designed 
> in such a
>    way that debugging protocol behaviors is easy.  
> Hand-waving this away
>    is not something that operators like ...
> 
> I think you are including this by hand-waving!
> What do you mean by "hooks"?
> Surely any protocol can be debugged simply by sniffing it.
> If you have specific hooks in mind you must set them out and not
> leave the reader guessing.
> But if you force the implementation I shall be peeved!
> You might, for example, say that there should be alerts issued when
> messages are received, or when there are state transitions in the
> protocol state machine, but you must recall that:
> - the messages can be seen by sniffing
> - the state machine is not part of the protocol specification.
> This latter point is key. The state machine explains how the
> protocol works in order that an implementer can decide how to
> react to a received event, but it does not require that the
> machine is implemented.
> 
> ===
> 
> 4.1.  Interoperability
> 
>    Interoperability needs to be considered on the syntactic level
and
>    the semantic level.  While it can be irritating and
time-consuming,
>    application designers including operators who write their 
> own scripts
>    can make their processing conditional to accommodate differences
> 
> s/differences/syntactic differences/ ?
> 
>    Then, whether the counter is gathered via SNMP or a CLI
>    command or a SYSLOG message, the counter will have similar
meaning.
> 
> "similar"? I think that might not be good enough. Don't you need
> "the same"?
> 
>    o  [RFC3805] - Printer MIB v2 (contains both an IM and a DM
> 
> Missing ")"
> 
> ===
> 
> 4.2.  Management Information
> 
> I would find it really helpful if you were able to recommend a
> language in which to write the information model. Ideally, all
> IETF IMs would be in the same language.
> 
>    It is important to be able to separately fetch configuration
data,
>    operational state data, and statistics from devices, and to be
able
>    to compare current state to initial state, and to compare data
>    between devices.
> 
> You seem to have drifted into DMs here. You cannot fetch data from
> an IM. It simply explains what data elements exist. In fact, an IM
> states all of the data that exists and so there is no confusion.
> Confusion can only arise as the DM is constructed and multiple IM
> data elements are conflated into single DM objects.
> 
> ===
> 
> 4.3.  Fault Management
> 
>    The protocol designer should document the basic faults and health
>    indicators that need to be instrumented for the new 
> protocol, and the
>    alarms and events that must be propagated to management 
> applications
>    or exposed through a data model.
> 
> Isn't "propagated to management applications" a subset of "exposed
> through a data model"?
> 
>    The protocol designer should consider how faults 
> information will be
> 
> s/faults/fault/
> 
>    Should the event reporting provide gyaranteed accurate delivery
of
> s/gyaranteed/guaranteed/
> 
> ===
> 
> 4.3.3.  Fault Isolation
> 
>    It might be useful to isolate faults, such as a system that emits
>    malformed messages necessary to coordinate connections properly.
>    Spanning tree comes to mind.  This might be able to be done by
>    configuring next-hop devices to drop the faulty messages to
prevent
>    them from entering the rest of the network.
> 
> This is not what *I* mean by fault isolation!
> Fault isolation is about working out where in the network the fault
> is. For example, if end-to-end data delivery is failing (reported by
> a notification), fault isolation will find the failed link or node
> in the end-to-end path.
> 
> You seem to be describing the process of quarantining faulty
> devices. This may also be valuable.
> 
> "Spanning tree comes to mind" is a rather strange sentence without
> much context.
> 
> ===
> 
> 4.4.  Configuration Management
> 
>    A protocol desoigners should document what the basic
configuration
> 
> s/desoigners/designers/
> 
>    "Requirements for Configuration Management of IP-based Networks"
>    [RFC3139] discusses requirements for configuration management.
> More precisely,
>    "Requirements for Configuration Management of IP-based Networks"
>    [RFC3139] discusses requirements for configuration management of
>    IP-based networks.
> Which makes me wonder about the purpose of the sentence :-)
> 
>    This document includes...
> This document or that document?
> 
>    A number of efforts have existed in the IETF to develop 
> policy-based
>    management.  "Terminology for Policy-Based Management" 
> [RFC3198] was
>    written to standardize the terminology across these 
> efforts.  Some of
>    the efforts resulted in proposals that are NOT RECOMMENDED, so a
>    protocol designer should check the current recommendation status
>    before depending on a specific protocol suggestion.
> 
> You have used RFC2119 language, but in section 1.1 you said
>    This document deliberately does not use the (capitalized)
keywords
>    described in RFC 2119 [RFC2119].
> Furthermore, it is not clear to me whether this I-D is doing the
> NOT RECOMMENDing or whether some other RFC has done it. If it is
> this I-D, you will need to give reasons. If is some other RFC, you
> will need to give references.
> 
>    It is highly desirable that text processing tools such as diff,
and
>    version management tools such as RCS or CVS or SVN, can be used
to
>    process configurations.  This approach simplifies comparing the
>    current operational state to the initial configuration.  It is
>    commonplace to compare configuration changes to e.g., last 
> day, last
>    week, last month, etc. -- having configuration in a text, 
> and human-
>    understandable format is very valuable for various reasons such
as
>    change control (or verification), configuration consistency
checks,
>    etc.
> 
> Yes, but what has this to do with the protocol design? It might be
> related to device implementation. Or it might be relevant to
> EMS/NMS implementation.
> 
> As you note in the subsequent paragraph, this may be impossible
> even in some text-based DMs. It would certainly be impossible in
> an SNMP MIB module.
> 
>    To simplify such configuration comparisons, devices should not
>    arbitrarily reorder data such as access control lists.  If 
> a protocol
>    designer defines mechanisms for configuration, it would be 
> desirable
>    to standardize the order of elements for consistency of 
> configuration
>    and of reporting across vendors, and across releases from
vendors.
> 
> This example seems a bit specific. And in any case, it is perfectly
> possible to sort a list before comparing.
> 
>    There are two parts to this: 1.  An NMS system could optimize
ACLs
>    for performance reasons 2.  Unless the device/NMS systems 
> has correct
>    rules/a lot of experience, reordering ACLs can lead to a huge
>    security issue.
> 
> Maybe what you are saying is:
>    Implementations should not arbitrarily modify configuration data.
>    In some cases (such as Access Control Lists) the order of data
>    items is significant and comprises part of the configured data.
> 
>    Network wide configurations are ideally stored in central master
>    databases
> 
> This looks like an unsubstantiated assertion. You might get away
> with "Many operators prefer that..."
> 
>    It is important to distinguish between the distribution of
>    configurations and the activation of a certain configuration.
>    Devices should be able to hold multiple configurations.
> 
> This is really an implementation issue and nothing to do with the
> protocol under design.
> 
>    Consensus of the 2002 IAB Workshop was that textual configuration
>    files should be able to contain international characters.
> 
> Do you have a citation?
> 
>    These timers may need default values
>    suggested in the protocol specification and do not need to be
>    otherwise configurable.
> 
> I had formed the opinion that timers must be given default values
> in IETF protocol specifications, but I can't lay my hands on a
> reference.
> 
> ===
> 
> 4.5.  Accounting Management
> 
>    A protocol designer should consider whether it would be
appropriate
>    to collect usage information related to this protocol, and if so,
>    what usage information would be appropriate to collect?
> 
> s/collect?/collect./
> 
>    "Introduction to Accounting Management" [RFC2975] 
> discusses a number
>    of factors relevant to monitoring usage of protocols for 
> purposes of
>    capacity and trend analysis, cost allocation, auditing, 
> and billing.
>    This document also discusses how Remote Authentication Dial In
User
> This or that?
> 
>    Service (RADIUS) [RFC2865], Terminal Access Controller 
> Access Control
>    System (TACACS) [RFC1492], SNMP, IPFIX, and Packet Sampling
(PSAMP)
>    Protocol [I-D.ietf-psamp-protocol] protocols can be used for
these
>    purposes.
> 
> Surely RFC 2975 does not mention PSAMP? Which makes me wonder what
> you are trying to achieve with a list of example protocols. Are
> they intended as recommendations (in which case say so and why),
> or as a non-exclusive list (in which case a "such as" would be
> useful). At the moment it reads like these are your favorites and
> you would be upset if someone used a different protocol!
> 
> ===
> 
> 4.6.  Performance Management
> 
>    There are several parts to performance management to be
considered:
>    protocol monitoring, services monitoring, and device 
> monitoring (the
>    impact of the new protocol/service activation on the device).
> 
> s/services/service/
> 
>    Consider scalability, such as whether performance will be 
> affected by
>    the number of protocol connections.
> 
> Maybe say "protocol entities" rather than connections?
> 
> 
> This section seems to flit between the performance of the protocol
> under design and the network running the protocol under design, and
> the performance of the management of the protocol. Each is
> important, but it may be constructive to split them out into
> separate sections. The latter will also include a consideration of
> how the act of monitoring impacts on network performance.
> 
> 
>    The Benchmarking Methodology WG (bmwg) has defined
recommendations
> Later you have BMWG.
> 
> ===
> 
> 4.7.  Security Management
> 
>    It must be possible to do consistency checks of access 
> control lists
>    across devices.  Protocol designers should consider information
>    models to promote comparisons across devices and across vendors
to
>    permit checking the consistency of security configurations.
> 
> I think this "must" is massively excessive!
> I would expect each router to have a different access list in the
> control plane (a list of its neighbors). Such a mechanism would
> actually open up a security vulnerability that has to be protected.
> Perhaps you are talking about management plane access lists. This
> takes us into the area of securing management activity - see my
> comments on Section 7.
> 
>    RADIUS and DIAMETER can provide authentication and 
> authorization.  A
>    protocol designer should consider which attributes would be
>    appropriate for their protocol.
> 
> This paragraph is apropos of what? What are you trying to tell me?
> Should I run my management using Radius? Should I try to build
> Diameter into my new protocol?
> 
>    Different protocols use different assumptions about 
> message security
> s/protocols/management protocols/
> 
>    and data access controls.  A protocol designer that 
> recommends using
>    different protocols should consider how security will be 
> applied in a
> s/protocols/management protocols/
> 
> 
> Although you talk about (presumably management) access lists, I
think
> I would have expected a discussion of authority levels and policy
> with regard to management tasks.
> 
> ===
> 
> 7.  Security Considerations
> 
>    This document is informational and provides guidelines for
>    considering manageability and operations.  It introduces no new
>    security concerns.
> 
> I think that, although you introduce no new security concerns
> directly, you have some things to talk about.
> 
> Firstly, the provision of a management portal to a network device
> provides a doorway through which an attack on the device may be
> launched. Since you are recommending that the protocol under
> development be manageable through the management protocol, you are
> requiring that the protocol be vulnerable to a new source of
> attacks. Therefore, you must talk about securing the protocol
> against these attacks. This might be as simple as saying that only
> management protocols with adequate (for some definition of
> adequate) security apparatus be used.
> 
> I think it is also of note that a standard description of the
> manageable knobs and whistles on a protocol makes it easier for an
> attacker to understand what they may try to control and how to
> tweak it.
> 
> On the other hand, it would be well worth pointing out that a
> well-designed protocol is usually more stable and secure. And that
> a protocol that can be managed and inspected offers the operator
> a better chance of spotting and quarantining any attacks.
> Conversely making a protocol inspectable, is a risk if the wrong
> person inspects it!
> 
> If security events cause logs and or notifications/alerts, can
> a concerted attack be mounted by causing an excess of these
> events? In other words, to the security management mechanisms
> constitute the security vulnerability?
> 
> Lastly, the management of security aspects is important, and you
> can just reference back to section 4.7.
> 
> ===
> 
> 9.  Informative References
> 
> I (of course) would favor an informational reference to 
> draft-ietf-pce-
> manageability-requirements-06.txt
> 
> ===
> 
>    [RFC2821]                       Klensin, J., "Simple Mail
Transfer
>                                    Protocol", RFC 2821, April 2001.
> 
> I think this has been obsoleted.
> 
> ===
> 
> Appendix A.  Operations and Management Review Checklist
> 
> Maybe a forward reference to this appendix from sections 1 and 5?
> 
> ===
> 
> A.3.  Documentation
> 
>    Does the proposed protocol have a significant operational impact
on
>    the Internet.  If it does, and the document under review targets
>    standards track, is their enough proof of implementation and/or
>    operational experience to grant Proposed Standard status?
> 
> Whoah!
> 
> You are going to use this I-D to try to block standards track I-Ds
> from becoming RFCs? You are going to get into such a fight!!!
> 
> IMHO this sentence is a complete show-stopper in your I-D.
> 
> (Oh, and s/their/there/)
> 
> ===
> 
> Appendix B.  Change Log
> 
> Suggest you flag this for removal upon publication as an RFC.
> 
> ===
> 
> > Sent: Sunday, January 25, 2009 6:40 PM
> > To: [e-mail]
> > Subject: [OPSAWG] (final) WGLC on
> > draft-ietf-opsawg-operations-and-management
> >
> > This is a (hopefully final) WGLC for
> > draft-ietf-opsawg-operations-and-management
> > there was not enough respose to the last one back in October
> > please let the list know what you think of the draft
> 
> _______________________________________________
> OPSAWG mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/opsawg
>