Re: [OPSAWG] (final) WGLCondraft-ietf-opsawg-operations-and-management

["David Harrington" <[e-mail]>]

Besides the comments form Juergen, Mike, and Adrian, what needs to be
done to make this live up to its billing? Do you have explicit
recommendations of text?

dbh 

> -----Original Message-----
> From: [e-mail] 
> [mailto:[e-mail]] On Behalf Of tom.petch
> Sent: Tuesday, February 10, 2009 12:05 PM
> To: Adrian Farrel; [e-mail]
> Cc: David B Harrington; [e-mail]
> Subject: Re: [OPSAWG] (final) 
> WGLCondraft-ietf-opsawg-operations-and-management
> 
> I was going to say +1 to Juergen's comments and then I was 
> going to say +1 to
> Mike's comments and then I was going to say +1 to Adrian's 
> comments and while I
> was pondering, ... I left it too late:-(
> 
> But I do echo much of what has been said.  I cannot see that 
> this I-D lives up
> to its billing.  It is a good idea, and it is good stuff 
> (give or take some
> infelicities of wording, most of which have been pointed out 
> already) but I do
> not see it as good stuff for a working group coming up with a 
> new protocol, say
> an MPLS-TP or sieve or sidr. And that is what I think is 
> needed.  If something
> is not designed to be managed ab initio, then it may be 
> costly, impossible even,
> to manage it in the wild.
> 
> Tom Petch
> 
> 
> ----- Original Message -----
> From: "Adrian Farrel" <[e-mail]>
> To: <[e-mail]>
> Cc: "David B Harrington" <[e-mail]>; 
> <[e-mail]>
> Sent: Wednesday, February 04, 2009 8:54 PM
> Subject: Re: [OPSAWG] (final) WGLC 
> ondraft-ietf-opsawg-operations-and-management
> 
> 
> > Hi,
> >
> > Dan asked me to have a look at this which I do primarily 
> with a routing area
> > view on the world.
> >
> > I'm sorry that I haven't been following this I-D very 
> closely and apologise
> > if you have already discussed and reached conclusion on 
> some of the points I
> > make.
> >
> > I am largely supportive of this work (see
> > draft-ietf-pce-manageability-requirements-06.txt), but I do 
> have some
> > issues. I would be comforted to know that the I-D had had 
> extensive review
> > from within the Ops Area without relying just on my comments!
> >
> > I would also like to understand how the recommendations 
> parts of the draft
> > will be interpreted by the Ops ADs as they perform IESG 
> review of I-Ds from
> > other areas. Depending on the answer (which I would like to 
> see encapsulated
> > in the document!) I might have other harsher comments about 
> the content.
> >
> > More details below.
> >
> > Cheers,
> > Adrian
> >
> > ===
> > Title is...
> >  Guidelines for Considering Operations and Management of 
> New Protocols
> > ...but the Abstract immediately includes...
> >    New protocols or protocol extensions
> > ===
> > Obviously needs the new IPR boilerplate.
> > ===
> > Abstract line 3
> > s/protocol/protocols/
> > ===
> > I am somewhat worried by "recommend appropriate standard
management
> > protocols ... to address the relevant operations and management
> > needs."
> >
> > I understand the theory behind this guidance in terms of making
> > equipment that will interwork with management systems, but I
> > question its practicality given the vast array of management
> > protocols available. And actually, I am not sure that this has a
> > direct impact on the success of or development of a protocol.
> >
> > The introduction says that this "is similar to a WG considering
> > which security threats are relevant to their protocol, and then
> > recommending appropriate standard security protocols to mitigate
> > the relevant threats." I dispute this. The security considerations
> > normally make modifications to the protocol under development to
> > make it secure, or mandate security features in other protocols
> > that the protocol under development uses. But where there is a
> > need for an external protocol, this is not usually mandated. For
> > example, where key exchange is needed the text will usually say
> > something like "may use some form of secure key exchange protocol
> > (such as the Internet Key Exchange protocol v2 (IKE) [RFC4306]."
> > This is far looser than a recommendation of a specific protocol.
> >
> > I believe that in practice operators want to go on using the
> > management protocols that they already have deployed. If, for
> > example, I have deployed routers that I manage using SNMP, I will
> > not want to deploy a new routing protocol on those routers if that
> > protocol is strongly recommended to be managed using TL1.
> >
> > Since data modeling is usually tied closely to the protocol used
> > for management, it follows from this that developing specific
> > data formats (SNMP MIB modules, XML schema, etc.) might also not
> > be so very practical. (Arguably we have found this out with MIB
> > modules over the last 10 years, but still churn them out.)
> >
> > What *would* be beneficial, however, is an abstract data model
> > for management of the protocol. There are many ways of
> > representing the data (XML, BNF, ASN.1, graphical representation,
> > etc.) and the Ops Area could usefully pick one and standardise
> > it. From there we would have a solid data set that all
> > implementations could make available and which separate documents
> > could easily map to suitable formats for use by specific
> > management protocols. Smart implementers would be able to support
> > multiple management protocols with ease.
> >
> > You do come to this to some extent in section 4.1, and the text
> > there seems to be at odds with what you have said in the
> > Introduction.
> >
> > ===
> > Section 1
> >
> > Can you expand "WG" on first usage?
> >
> > ===
> > Section 1
> >
> >    We want to avoid seeming to impose a solution by putting 
> in place a
> >    strict terminology - for example implying that a formal 
> data model,
> >    or even using a management protocol is mandatory.
> >
> > <grin mode=wry>
> >
> > You want to avoid seeming to impose it?
> > So, you want to impose it, but you don't want anyone to notice?
> >
> > </grin>
> >
> > ===
> > Section 1
> >
> >    Making a Management Considerations section a mandatory 
> publication
> >    requirement for IETF documents is the responsibility of 
> the IESG, or
> >    specific area directors, or working groups, and this 
> document avoids
> >    recommending any mandatory publication requirements.
> >
> > I think you may need to go a step further to get this through the
> > IESG. It would not be funny to see this paragraph as written, but
> > to get a DISCUSS on an I-D from an Ops AD because the I-D did not
> > contain a Management Considerations section.
> >
> > ===
> > Introduction
> >
> >    We provide some objective criteria to promote interoperability
> >
> > <grin mode=laconic>
> >
> > There seems to be only one author. Are you royalty?
> > Perhaps... "This document provides..."
> >
> > </grin>
> >
> > ===
> > Section 1 final line
> >
> > s/??/?"/
> >
> > ===
> >
> > 2.1.  IETF Management Framework
> >
> >    For years the IETF has stressed the use of the IETF Standard
> >    Management Framework and MIB modules [RFC2578] for managing new
> >    protocols.  The IETF designed these to permit multiple 
> protocols to
> >    utilize the MIB data [RFC1052], but it became a common
> >
> > s/became/is/
> >
> > ===
> >
> > 3.1.  Operations Model
> >
> >    o  what are the typical user interfaces - Command line (CLI) or
> >       graphical user interface (GUI)?
> >
> > I am not sure that this last item is particularly relevant for a
> > large number of the protocols designed in the IETF. We might say
> > that the question is valid for a protocol such as FTP, but is it
> > relevant for OSPF? Yes, there is a configuration interface to a
> > device that runs OSPF, but it is surely nothing to do with the
> > protocol design how that interface works. Indeed, you might
> > reduce the question to: are the configuration interfaces local or
> > remote? But I think you will find that the answer to the question
> > is "both" as in the subsequent paragraph.
> >
> > ===
> >
> > 3.1.  Operations Model
> >
> >    Auto-configuration and default parameters might be
> >    possible for some new protocols.
> >
> > I would make a much bigger noise about default parameters. There
is
> > a key question about how the protocol can operate "out of the
box".
> > If implementers are free to select their own defaults, the
protocol
> > needs to operate well with any choice of values. If there are
> > sensible defaults, these need to be stated.
> >
> > Indeed, section 3.2 seems to say this. Perhaps a forward
reference?
> >
> > ===
> >
> > 3.1.  Operations Model
> >
> >    There may be a need to support a human interface, e.g., for
> >    troubleshooting, and a programmatic interface, e.g., for 
> automated
> >    monitoring and root cause analysis.  It might be 
> important that the
> >    internal method routines used by the application programming
> >    interfaces and the human interfaces should be the same 
> to ensure that
> >    data exchanged between these two interfaces is always
consistent.
> >    Mixing methods leads to inconsistency, so identifying
consistent
> >    methods of retrieving information is relevant.
> >
> > This is really sailing too close to the implementation. People
> > must be free to construct sub-standard and unmarketable
> > implementations. The IETF is not responsible for making all
> > implementations successful. The job is to make it possible for
> > all implementations to be successful given core competences in the
> > developers.
> >
> > I think the furthest a protocol spec RFC could go would be to say
> > what information must be available to an operator at a human
> > interface for troubleshooting
> >
> > ===
> >
> > 3.2.  Installation and Initial Setup
> >
> > In view of the reasons you give why defaults might become out-
> > dated, you might point out that sometimes it is not advisable to
> > have defaults. In these cases explicit configuration is required.
> >
> > ===
> >
> > 3.2.  Installation and Initial Setup
> >
> > Is the text at the top of page 8 supposed to be a bullet list?
> >
> > ===
> >
> > 3.4.  Requirements on Other Protocols and Functional Components
> >
> >    These considerations should generally remain 
> illustrative to avoid
> >    creating restrictions or dependencies, or potentially 
> impacting the
> >    behavior of existing protocols, or restricting the 
> extensibility of
> >    other protocols, or assuming other protocols will not be 
> extended in
> >    certain ways.
> >
> > Well, this is quite interesting.
> >
> > Suppose you decide to utilise a particular feature of a transport
> > protocol. You are creating a hard requirement that that feature is
> > implemented in all implementations of the set {your new protocol,
> > the transport protocol}. But more importantly, perhaps, you are
> > requiring that new versions of the transport protocol do not
> > deprecate the feature you require.
> >
> > In short, I don't think you should be too nervous about creating
> > restrictions or dependencies. If they exist, they must be stated.
> >
> > ===
> >
> > 3.2.  Installation and Initial Setup
> >
> >    For example, the design of Resource ReSerVation Protocol (RSVP)
> >
> > I think this example is not well expressed.
> >
> > The problem in RSVP was that the IP packet was addressed to the
> > ultimate RSVP-capable destination, but the RSVP-capable routers
> > along the path needed to intercept and process the Path message.
> > This they could only do by "deep packet inspection" or by the
> > router alert option.
> >
> > I wonder whether this example adds significantly to the I-D.
> >
> > ===
> >
> > 4.  Management Considerations
> >
> >    NETCONF addresses this in a generic manner by
> >
> > Add citation.
> >
> > ===
> >
> > 4.  Management Considerations
> >
> >    However, debugging on-the-wire interactions is a protocol
> >    issue: it is enormously helpful if a protocol has hooks to make
> >    debugging of network interactions easy, and/or is 
> designed in such a
> >    way that debugging protocol behaviors is easy.  
> Hand-waving this away
> >    is not something that operators like ...
> >
> > I think you are including this by hand-waving!
> > What do you mean by "hooks"?
> > Surely any protocol can be debugged simply by sniffing it.
> > If you have specific hooks in mind you must set them out and not
> > leave the reader guessing.
> > But if you force the implementation I shall be peeved!
> > You might, for example, say that there should be alerts issued
when
> > messages are received, or when there are state transitions in the
> > protocol state machine, but you must recall that:
> > - the messages can be seen by sniffing
> > - the state machine is not part of the protocol specification.
> > This latter point is key. The state machine explains how the
> > protocol works in order that an implementer can decide how to
> > react to a received event, but it does not require that the
> > machine is implemented.
> >
> > ===
> >
> > 4.1.  Interoperability
> >
> >    Interoperability needs to be considered on the syntactic 
> level and
> >    the semantic level.  While it can be irritating and 
> time-consuming,
> >    application designers including operators who write 
> their own scripts
> >    can make their processing conditional to accommodate
differences
> >
> > s/differences/syntactic differences/ ?
> >
> >    Then, whether the counter is gathered via SNMP or a CLI
> >    command or a SYSLOG message, the counter will have 
> similar meaning.
> >
> > "similar"? I think that might not be good enough. Don't you need
> > "the same"?
> >
> >    o  [RFC3805] - Printer MIB v2 (contains both an IM and a DM
> >
> > Missing ")"
> >
> > ===
> >
> > 4.2.  Management Information
> >
> > I would find it really helpful if you were able to recommend a
> > language in which to write the information model. Ideally, all
> > IETF IMs would be in the same language.
> >
> >    It is important to be able to separately fetch 
> configuration data,
> >    operational state data, and statistics from devices, and 
> to be able
> >    to compare current state to initial state, and to compare data
> >    between devices.
> >
> > You seem to have drifted into DMs here. You cannot fetch data from
> > an IM. It simply explains what data elements exist. In fact, an IM
> > states all of the data that exists and so there is no confusion.
> > Confusion can only arise as the DM is constructed and multiple IM
> > data elements are conflated into single DM objects.
> >
> > ===
> >
> > 4.3.  Fault Management
> >
> >    The protocol designer should document the basic faults and
health
> >    indicators that need to be instrumented for the new 
> protocol, and the
> >    alarms and events that must be propagated to management 
> applications
> >    or exposed through a data model.
> >
> > Isn't "propagated to management applications" a subset of "exposed
> > through a data model"?
> >
> >    The protocol designer should consider how faults 
> information will be
> >
> > s/faults/fault/
> >
> >    Should the event reporting provide gyaranteed accurate 
> delivery of
> > s/gyaranteed/guaranteed/
> >
> > ===
> >
> > 4.3.3.  Fault Isolation
> >
> >    It might be useful to isolate faults, such as a system that
emits
> >    malformed messages necessary to coordinate connections
properly.
> >    Spanning tree comes to mind.  This might be able to be done by
> >    configuring next-hop devices to drop the faulty messages 
> to prevent
> >    them from entering the rest of the network.
> >
> > This is not what *I* mean by fault isolation!
> > Fault isolation is about working out where in the network the
fault
> > is. For example, if end-to-end data delivery is failing (reported
by
> > a notification), fault isolation will find the failed link or node
> > in the end-to-end path.
> >
> > You seem to be describing the process of quarantining faulty
> > devices. This may also be valuable.
> >
> > "Spanning tree comes to mind" is a rather strange sentence without
> > much context.
> >
> > ===
> >
> > 4.4.  Configuration Management
> >
> >    A protocol desoigners should document what the basic 
> configuration
> >
> > s/desoigners/designers/
> >
> >    "Requirements for Configuration Management of IP-based
Networks"
> >    [RFC3139] discusses requirements for configuration management.
> > More precisely,
> >    "Requirements for Configuration Management of IP-based
Networks"
> >    [RFC3139] discusses requirements for configuration management
of
> >    IP-based networks.
> > Which makes me wonder about the purpose of the sentence :-)
> >
> >    This document includes...
> > This document or that document?
> >
> >    A number of efforts have existed in the IETF to develop 
> policy-based
> >    management.  "Terminology for Policy-Based Management" 
> [RFC3198] was
> >    written to standardize the terminology across these 
> efforts.  Some of
> >    the efforts resulted in proposals that are NOT RECOMMENDED, so
a
> >    protocol designer should check the current recommendation
status
> >    before depending on a specific protocol suggestion.
> >
> > You have used RFC2119 language, but in section 1.1 you said
> >    This document deliberately does not use the 
> (capitalized) keywords
> >    described in RFC 2119 [RFC2119].
> > Furthermore, it is not clear to me whether this I-D is doing the
> > NOT RECOMMENDing or whether some other RFC has done it. If it is
> > this I-D, you will need to give reasons. If is some other RFC, you
> > will need to give references.
> >
> >    It is highly desirable that text processing tools such 
> as diff, and
> >    version management tools such as RCS or CVS or SVN, can 
> be used to
> >    process configurations.  This approach simplifies comparing the
> >    current operational state to the initial configuration.  It is
> >    commonplace to compare configuration changes to e.g., 
> last day, last
> >    week, last month, etc. -- having configuration in a 
> text, and human-
> >    understandable format is very valuable for various 
> reasons such as
> >    change control (or verification), configuration 
> consistency checks,
> >    etc.
> >
> > Yes, but what has this to do with the protocol design? It might be
> > related to device implementation. Or it might be relevant to
> > EMS/NMS implementation.
> >
> > As you note in the subsequent paragraph, this may be impossible
> > even in some text-based DMs. It would certainly be impossible in
> > an SNMP MIB module.
> >
> >    To simplify such configuration comparisons, devices should not
> >    arbitrarily reorder data such as access control lists.  
> If a protocol
> >    designer defines mechanisms for configuration, it would 
> be desirable
> >    to standardize the order of elements for consistency of 
> configuration
> >    and of reporting across vendors, and across releases 
> from vendors.
> >
> > This example seems a bit specific. And in any case, it is
perfectly
> > possible to sort a list before comparing.
> >
> >    There are two parts to this: 1.  An NMS system could 
> optimize ACLs
> >    for performance reasons 2.  Unless the device/NMS 
> systems has correct
> >    rules/a lot of experience, reordering ACLs can lead to a huge
> >    security issue.
> >
> > Maybe what you are saying is:
> >    Implementations should not arbitrarily modify configuration
data.
> >    In some cases (such as Access Control Lists) the order of data
> >    items is significant and comprises part of the configured data.
> >
> >    Network wide configurations are ideally stored in central
master
> >    databases
> >
> > This looks like an unsubstantiated assertion. You might get away
> > with "Many operators prefer that..."
> >
> >    It is important to distinguish between the distribution of
> >    configurations and the activation of a certain configuration.
> >    Devices should be able to hold multiple configurations.
> >
> > This is really an implementation issue and nothing to do with the
> > protocol under design.
> >
> >    Consensus of the 2002 IAB Workshop was that textual
configuration
> >    files should be able to contain international characters.
> >
> > Do you have a citation?
> >
> >    These timers may need default values
> >    suggested in the protocol specification and do not need to be
> >    otherwise configurable.
> >
> > I had formed the opinion that timers must be given default values
> > in IETF protocol specifications, but I can't lay my hands on a
> > reference.
> >
> > ===
> >
> > 4.5.  Accounting Management
> >
> >    A protocol designer should consider whether it would be 
> appropriate
> >    to collect usage information related to this protocol, and if
so,
> >    what usage information would be appropriate to collect?
> >
> > s/collect?/collect./
> >
> >    "Introduction to Accounting Management" [RFC2975] 
> discusses a number
> >    of factors relevant to monitoring usage of protocols for 
> purposes of
> >    capacity and trend analysis, cost allocation, auditing, 
> and billing.
> >    This document also discusses how Remote Authentication 
> Dial In User
> > This or that?
> >
> >    Service (RADIUS) [RFC2865], Terminal Access Controller 
> Access Control
> >    System (TACACS) [RFC1492], SNMP, IPFIX, and Packet 
> Sampling (PSAMP)
> >    Protocol [I-D.ietf-psamp-protocol] protocols can be used 
> for these
> >    purposes.
> >
> > Surely RFC 2975 does not mention PSAMP? Which makes me wonder what
> > you are trying to achieve with a list of example protocols. Are
> > they intended as recommendations (in which case say so and why),
> > or as a non-exclusive list (in which case a "such as" would be
> > useful). At the moment it reads like these are your favorites and
> > you would be upset if someone used a different protocol!
> >
> > ===
> >
> > 4.6.  Performance Management
> >
> >    There are several parts to performance management to be 
> considered:
> >    protocol monitoring, services monitoring, and device 
> monitoring (the
> >    impact of the new protocol/service activation on the device).
> >
> > s/services/service/
> >
> >    Consider scalability, such as whether performance will 
> be affected by
> >    the number of protocol connections.
> >
> > Maybe say "protocol entities" rather than connections?
> >
> >
> > This section seems to flit between the performance of the protocol
> > under design and the network running the protocol under design,
and
> > the performance of the management of the protocol. Each is
> > important, but it may be constructive to split them out into
> > separate sections. The latter will also include a consideration of
> > how the act of monitoring impacts on network performance.
> >
> >
> >    The Benchmarking Methodology WG (bmwg) has defined 
> recommendations
> > Later you have BMWG.
> >
> > ===
> >
> > 4.7.  Security Management
> >
> >    It must be possible to do consistency checks of access 
> control lists
> >    across devices.  Protocol designers should consider information
> >    models to promote comparisons across devices and across 
> vendors to
> >    permit checking the consistency of security configurations.
> >
> > I think this "must" is massively excessive!
> > I would expect each router to have a different access list in the
> > control plane (a list of its neighbors). Such a mechanism would
> > actually open up a security vulnerability that has to be
protected.
> > Perhaps you are talking about management plane access lists. This
> > takes us into the area of securing management activity - see my
> > comments on Section 7.
> >
> >    RADIUS and DIAMETER can provide authentication and 
> authorization.  A
> >    protocol designer should consider which attributes would be
> >    appropriate for their protocol.
> >
> > This paragraph is apropos of what? What are you trying to tell me?
> > Should I run my management using Radius? Should I try to build
> > Diameter into my new protocol?
> >
> >    Different protocols use different assumptions about 
> message security
> > s/protocols/management protocols/
> >
> >    and data access controls.  A protocol designer that 
> recommends using
> >    different protocols should consider how security will be 
> applied in a
> > s/protocols/management protocols/
> >
> >
> > Although you talk about (presumably management) access 
> lists, I think
> > I would have expected a discussion of authority levels and policy
> > with regard to management tasks.
> >
> > ===
> >
> > 7.  Security Considerations
> >
> >    This document is informational and provides guidelines for
> >    considering manageability and operations.  It introduces no new
> >    security concerns.
> >
> > I think that, although you introduce no new security concerns
> > directly, you have some things to talk about.
> >
> > Firstly, the provision of a management portal to a network device
> > provides a doorway through which an attack on the device may be
> > launched. Since you are recommending that the protocol under
> > development be manageable through the management protocol, you are
> > requiring that the protocol be vulnerable to a new source of
> > attacks. Therefore, you must talk about securing the protocol
> > against these attacks. This might be as simple as saying that only
> > management protocols with adequate (for some definition of
> > adequate) security apparatus be used.
> >
> > I think it is also of note that a standard description of the
> > manageable knobs and whistles on a protocol makes it easier for an
> > attacker to understand what they may try to control and how to
> > tweak it.
> >
> > On the other hand, it would be well worth pointing out that a
> > well-designed protocol is usually more stable and secure. And that
> > a protocol that can be managed and inspected offers the operator
> > a better chance of spotting and quarantining any attacks.
> > Conversely making a protocol inspectable, is a risk if the wrong
> > person inspects it!
> >
> > If security events cause logs and or notifications/alerts, can
> > a concerted attack be mounted by causing an excess of these
> > events? In other words, to the security management mechanisms
> > constitute the security vulnerability?
> >
> > Lastly, the management of security aspects is important, and you
> > can just reference back to section 4.7.
> >
> > ===
> >
> > 9.  Informative References
> >
> > I (of course) would favor an informational reference to 
> draft-ietf-pce-
> > manageability-requirements-06.txt
> >
> > ===
> >
> >    [RFC2821]                       Klensin, J., "Simple 
> Mail Transfer
> >                                    Protocol", RFC 2821, April
2001.
> >
> > I think this has been obsoleted.
> >
> > ===
> >
> > Appendix A.  Operations and Management Review Checklist
> >
> > Maybe a forward reference to this appendix from sections 1 and 5?
> >
> > ===
> >
> > A.3.  Documentation
> >
> >    Does the proposed protocol have a significant 
> operational impact on
> >    the Internet.  If it does, and the document under review
targets
> >    standards track, is their enough proof of implementation and/or
> >    operational experience to grant Proposed Standard status?
> >
> > Whoah!
> >
> > You are going to use this I-D to try to block standards track I-Ds
> > from becoming RFCs? You are going to get into such a fight!!!
> >
> > IMHO this sentence is a complete show-stopper in your I-D.
> >
> > (Oh, and s/their/there/)
> >
> > ===
> >
> > Appendix B.  Change Log
> >
> > Suggest you flag this for removal upon publication as an RFC.
> >
> > ===
> >
> > > Sent: Sunday, January 25, 2009 6:40 PM
> > > To: [e-mail]
> > > Subject: [OPSAWG] (final) WGLC on
> > > draft-ietf-opsawg-operations-and-management
> > >
> > > This is a (hopefully final) WGLC for
> > > draft-ietf-opsawg-operations-and-management
> > > there was not enough respose to the last one back in October
> > > please let the list know what you think of the draft
> >
> > _______________________________________________
> > OPSAWG mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/opsawg
> 
> _______________________________________________
> OPSAWG mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/opsawg
>