Re: [sacm] Proposed use cases to move forward

[Stephen Hanna <[e-mail]>]

Adam Montville wrote:
> For the sake of playing devil's advocate, if we have deliverables and
> milestones (implying order), what warrants "to much" work?

There's no hard-and-fast rule for this but I was just on
a boring concall so I ran some numbers on the existing
IETF SEC area Working Groups.

* The number of WG documents varies from 0 to 11 with 3/4 of the
  WGs having between 2 & 6 WG documents.

* The number of authors per WG document (not double-counting authors
  who are on more than one doc in a single WG) varies from 1.25 to
  3.5 with 2/3 of the WGs having between 1.4 and 2.3 authors per spec.

* The number of emails per month varies between about 8 and about 150.
  Removing two WGs with lots of legacy RFCs and therefore an extra
  amount of email traffic (PKIX and IPSECME), the number of emails
  per month per WG document varies between 6 and 20 with the
  average around 12.

Looking at SACM, 9 WG documents would be way more than average
for the SEC area. Our email list traffic is about 60 emails per
month. That would indicate we can support about 5 WG documents.
I think that's consistent with the number of active participants
and likely document authors that I have seen on the list so far.
I see about 15 active participants on the list in the last few
months. We could gain or lose some as time goes on but that's
what I see. If half of those become document editors, we could
easily support 4-5 WG documents.

Maybe we should figure out how to stage our work into two stages.
That way, we could have 4-5 documents in a first stage and 4-5
documents in a second stage. Many WGs do that. We can list all
the documents in our charter and have milestones that stage them.

Now that I reread your email, I think that's exactly what you
were suggesting. So maybe we're agreeing! But I still think
that remediation shouldn't be in the first two stages. It's
the least understood aspect of this work.

Thanks,

Steve

> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of
> Adam Montville
> Sent: Tuesday, August 21, 2012 10:58 AM
> To: Stephen Hanna
> Cc: [e-mail]; [e-mail]
> Subject: Re: [sacm] Proposed use cases to move forward
>
>
>
> On Aug 21, 2012, at 7:55 AM, "Stephen Hanna" <[e-mail]>
> wrote:
>
> > In the current charter, I see 3 areas of focus, 2 Informational
> > documents, and at least 9 Standards Track documents. That's more
> > than one Working Group should be taking on at one time. We need
> > to focus on narrowing our scope not expanding it.
>
>
> For the sake of playing devil's advocate, if we have deliverables and
> milestones (implying order), what warrants "to much" work?
>
>
>
>
> >
> > Remediation is a great topic but we should not work on it in
> > this (proposed) working group at this time. Instead, we should
> > include hooks for sending remediation instructions (like the
> > NEA Working Group did in PA-TNC) and encourage others to create
> > and experiment with various remediation systems. When they have
> > found something that works, we can bring it into SACM or just
> > publish it straight as an RFC.
> >
> > At least, that's my view.
> >
> > Thanks,
> >
> > Steve
> >
> >> -----Original Message-----
> >> From: [e-mail] [mailto:[e-mail]] On Behalf
> Of
> >> Adam Montville
> >> Sent: Tuesday, August 21, 2012 10:11 AM
> >> To: [e-mail]; [e-mail]
> >> Subject: Re: [sacm] Proposed use cases to move forward
> >>
> >> On 8/20/12 12:59 PM, "[e-mail]"
> >> <[e-mail]> wrote:
> >>> One question that I did have is the draft charter talks about
> >>> remediation.  What are other's thoughts on including that as a
> >>> 'potential' for the work to be done under SACM?
> >>
> >>
> >>
> >> I would like to see it included.  We may need to rethink some of the
> >> work
> >> that has already been done in this area.  Maybe we start by
> providing
> >> some
> >> way to standardize the minimum set of information that should be
> >> represented in manual remediation instructions before we move on to
> >> automated remediation - essentially, walk before we run.
> >>
> >>
> >>
> >>>
> >>> ---------------------------------
> >>> Security Automation Continuous Monitoring (SACM)
> >>>
> >>>
> >>> Proposed Working Group Charter
> >>>
> >>>
> >>> Chairs:
> >>> TBD
> >>> TBD
> >>>
> >>>
> >>> Security Area Directors:
> >>>    Stephen Farrell <[e-mail]>
> >>>    Sean Turner <[e-mail]>
> >>>
> >>>
> >>> Security Area Advisor:
> >>>    Sean Turner <[e-mail]>
> >>>
> >>>
> >>> Mailing Lists:
> >>>    General Discussion: [e-mail]
> >>>    To Subscribe:
> http://www.ietf.org/mailman/listinfo/sacm
> >>>    Archive:                http://www.ietf.org/mail-
> archive/web/sacm
> >>>
> >>>
> >>> Description of Working Group
> >>>
> >>>
> >>> Securing information and the systems that store, process, and
> transmit
> >>> that information has become a challenging task for organizations of
> >> all
> >>> sizes, and we find that security practitioners spend most of their
> >> time
> >>> on manual processes relegating them to
> >>> ineffectiveness. Security automation is the key to escaping this
> rut.
> >>> This working group will develop security automation standards in
> >> support
> >>> of information security processes and practices where practical.
> These
> >>> standards will support security practitioners
> >>> to be better utilized within their organizations by allowing them
> to
> >>> meet the more advanced needs of the security community (e.g.
> >> information
> >>> sharing, continuous monitoring, remediation and response, result
> >>> aggregation and analysis). The initial focus of this
> >>> work is to address enterprise and SOHO use cases. The working group
> >> will
> >>> achieve this by consuming and continuing (with cooperation) the
> >> security
> >>> automation work already performed by various organizations around
> the
> >>> world.
> >>>
> >>>
> >>> The initial work has been fruitful, and the data formats previously
> >>> published are ready for expansion on the international stage. Of
> >>> particular interest to this working group are the security
> automation
> >>> specifications supporting asset, change, configuration,
> >>> and vulnerability management. Of additional interest to this
> working
> >>> group are the emerging security automation interfaces and data
> formats
> >>> relating to event management and continuous monitoring.
> >>>
> >>>
> >>> By undertaking this work, we recognize that there are multiple
> >> categories
> >>> of problems in the security automation domain: defining expressions
> >> for
> >>> particular domain concepts (i.e. data formats), establishing a
> >>> standards-based foundation supporting the curation
> >>> and exchange of security automation content collections in content
> >>> repositories and enabling interoperability through the development
> and
> >>> use of interfaces and communications protocols. Content based on
> rich
> >>> data standards and protocols will provide the authoritative
> >>> instructions needed by data-driven tools to enable the automated
> >>> collection and exchange of configuration and vulnerability data
> >>> pertaining to enterprise assets. Information produced by these
> tools
> >> will
> >>> provide accurate and timely situational awareness in
> >>> support of organizational decision making.
> >>>
> >>>
> >>> This working group will provide solutions to these categories of
> >> problems
> >>> and the main areas of focus for this working group are described as
> >>> follows:
> >>>
> >>>
> >>> 1. Define, either by normative reference, adoption, or creation, a
> set
> >> of
> >>> standards that can be used for the purpose of assessing,
> aggregating
> >> and
> >>> comparing device states against expected values, and reporting on
> >> those
> >>> results in a predefined or ad hoc
> >>> manner.
> >>>
> >>>
> >>>
> >>> 2. Define, either by normative reference, adoption, or creation, a
> set
> >> of
> >>> standards that can be used to continuously monitor and report on
> the
> >>> state of systems, composed of many different types of devices and
> >>> networks, operated by varying personnel, to
> >>> ensure security process effectiveness in a pre-defined or ad-hoc
> >> manner.
> >>>
> >>>
> >>> 3. Create relationships between existing operations management
> >> standards
> >>> to enable a comprehensive view of security automation, leveraging
> >>> existing work and implementations.
> >>
> >>
> >>
> >> At some point I believe we had a 4 and 5 here as well.  I can't
> >> remember,
> >> were those recommended to be removed?
> >>
> >> Otherwise, I agree with the three "main" areas of focus.
> >>
> >>
> >>
> >>>
> >>>
> >>> This working group will produce the following:
> >>>
> >>>
> >>> * An Informational document providing an overview of security
> >> automation
> >>> and continuous monitoring to include a reference model
> >>
> >>
> >> Is this a reference to the use case document or something else (I.e.
> >> the
> >> informational document I've talked about in the past)?
> >>
> >>
> >>
> >>> * A Standards Track document specifying benchmark configuration
> >>> representation (XCCDF)
> >>> * An Informational document stating guidelines / requirements for
> >>> specifying checking languages
> >>> * Standards Track documents specifying device state checking
> languages
> >>> (OVAL, ECL, ACEML, Š)
> >>> * A Standards Track document specifying an interrogative checking
> >>> language (OCIL)
> >>> * A Standards Track document specifying platform naming, matching
> and
> >>> applicability (CPE)
> >>> * Standards Track documents specifying asset identification and
> >> reporting
> >>> information (Asset ID, ARF)
> >>> * Standards Track documents specifying interfaces and communication
> >>> protocols used for security automation and continuous monitoring
> >>> * A Standards Track document describing the messages and network
> >>> protocols for distributing Security Automation Content  (content
> >>> repository)
> >>
> >>
> >> I would remove the word "network" from "network protocols."
> >>
> >>
> >>> * Standards Track document describing integrating security
> automation
> >> and
> >>> Network Endpoint Assessment capabilities (if-m for SCAP?)
> >>> * A Standards Track document describing protocols and data formats
> for
> >>> securely sharing dynamic network state information among security
> >> systems
> >>> (if-map)
> >>
> >>
> >>
> >> I was working on some thought exercises yesterday for the Use Case
> >> document, and I think we might have some gaps in existing
> capabilities
> >> that we should address (either through normative reference or by
> >> creating
> >> something new):
> >>
> >> 1. Do we need a way to identify, express, and perhaps score patches
> in
> >> a
> >> way that differs from vulnerabilities?  This may be something akin
> to
> >> CPE
> >> - perhaps an extension to it.
> >>
> >> 2. Should we consider breaking apart the constituents of a technical
> >> control checking system in a way that allows us to talk about
> security
> >> and
> >> non-security related system objects?  There are almost certainly
> ties
> >> to
> >> other areas of IETF and perhaps other SDOs here, but I can use OVAL
> as
> >> a
> >> frame of reference.  OVAL talks about definitions, tests, objects,
> >> states,
> >> and values (ultimately assigned to states).  Is there a need to talk
> >> about
> >> objects, and their applicable range of states, outside the context
> of
> >> what
> >> we would generically call a test?
> >>
> >> 3. Do we need a way to talk about assets in a non-identifying way?
> I'm
> >> thinking things like criticality, classification (I.e. level of
> >> secrecy),
> >> and other properties that may not necessarily be used for the
> purposes
> >> of
> >> identification?
> >>
> >> 4. Do we need to consider documents dealing with targeting issues?
> >>
> >> These four points essentially speak to functional components that
> may
> >> be
> >> lacking in the existing set of specifications/formats.  It is my
> >> expectation that the Use Case document will identify and highlight
> such
> >> gaps, where we know we need certain functional capabilities
> (security
> >> configuration management including configuration assessment and
> >> remediation, asset characterization/management, and vulnerability
> >> assessment, perhaps among others).
> >>
> >>
> >>
> >>
> >>>
> >>>
> >>> Goals and Milestones
> >>>
> >>>
> >>> TBD
> >>> --------------------------------------------
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> Thanks.
> >>>
> >>>
> >>> Kent Landfield
> >>>
> >>> McAfee | An Intel Company
> >>> Direct: +1.972.963.7096
> >>> Mobile: [phone]
> >>> Web: www.mcafee.com <http://www.mcafee.com/>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> From: Adam Montville <[e-mail]>
> >>> Date: Friday, August 17, 2012 12:33 PM
> >>> To: David Waltermire <[e-mail]>, Stephen Hanna
> >>> <[e-mail]>, Luis Nunez <[e-mail]>,
> >>> Omar Santos <[e-mail]>
> >>> Cc: "[e-mail]" <[e-mail]>
> >>> Subject: Re: [sacm] Proposed use cases to move forward
> >>>
> >>>
> >>>
> >>>> I like taking the scope down as well - and I think we all believe
> >> we've
> >>>> agreed to UC1 and UC3.  I don't have an issue having the other use
> >> cases
> >>>> (2, 4, and 5) described in the use case document, but not fleshed
> out
> >> in
> >>>> the functional capabilities, components, and data/protocol
> sections.
> >> The
> >>>> charter can be derived from the use cases we take on, and a
> separate
> >>>> informational document can help place those use cases in the right
> >>>> context
> >>>> with respect to the "grand scheme of things."
> >>>>
> >>>>
> >>>> No matter how we slice it, right now we need to focus on the use
> >> cases
> >>>> and
> >>>> charter, and to focus on the use cases we would ideally need to
> work
> >> on
> >>>> setting the context.  I've tried to do that (to a limited extent)
> >> with
> >>>> the
> >>>> informational document I sent to the list.
> >>>>
> >>>>
> >>>> Additionally, I think these efforts need to be done in parallel
> >>>> irrespective of any idealistic dependencies.  For example, we
> should
> >> be
> >>>> able to work on a charter, a use case doc, and an informational
> doc
> >> all
> >>>> at
> >>>> the same time, but join the threads for a final alignment pass
> before
> >>>> submitting them.
> >>>>
> >>>>
> >>>> Adam
> >>>>
> >>>>
> >>>> On 8/17/12 9:30 AM, "Waltermire, David A."
> >> <[e-mail]>
> >>>> wrote:
> >>>>
> >>>>
> >>>>> Steve,
> >>>>>
> >>>>>
> >>>>> These are all valid concerns that I completely agree with.  What
> I
> >> am
> >>>>> struggling with is insuring that the work we are embarking on
> >> remains
> >>>>> relevant in the broader context.  My fear is that if we narrow
> our
> >>>>> thinking too much, we may inadvertently make decisions that drift
> >> from
> >>>>> addressing the broader set of use cases.  I like the idea of
> working
> >> on
> >>>>> an individual draft to address the larger context.  What I am not
> >> sure
> >>>>> about is how introduce a feedback loop that would result in
> >> minimizing
> >>>>> this kind of risk.  I guess this type of issue is something that
> we
> >> will
> >>>>> need to collectively monitor and address as needed.
> >>>>>
> >>>>>
> >>>>> Sincerely,
> >>>>> Dave
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: Stephen Hanna [mailto:[e-mail]]
> >>>>> Sent: Wednesday, August 15, 2012 3:55 PM
> >>>>> To: Waltermire, David A.; Adam Montville; Luis Nunez; Omar Santos
> >>>>> Cc: [e-mail]
> >>>>> Subject: RE: [sacm] Proposed use cases to move forward
> >>>>>
> >>>>>
> >>>>> I love broad scope. Don't get me wrong! But I'm concerned about
> >> having a
> >>>>> use cases document and an architecture document for this working
> >> group
> >>>>> that goes way beyond the charter and initial scope for the group.
> >>>>>
> >>>>>
> >>>>> I'm concerned that this will lead to lots of discussions on the
> sacm
> >>>>> list
> >>>>> and lots of effort being spent on topics that are out of scope,
> >>>>> diverting
> >>>>> us from the tasks at hand and slowing our progress.
> >>>>>
> >>>>>
> >>>>> I'm concerned that the working group chairs won't be able to cut
> off
> >>>>> discussion of topics by saying they're out of scope for the
> working
> >>>>> group.
> >>>>>
> >>>>>
> >>>>> I'm concerned that we'll get sidetracked or even derailed by
> >>>>> controversies that aren't relevant to the scope of the group.
> >>>>>
> >>>>>
> >>>>> I'm concerned that by including all of security automation within
> >> our
> >>>>> use
> >>>>> cases and architecture, we may actually prevent the formation of
> >> other
> >>>>> working groups that could work on those other use cases.
> >>>>>
> >>>>>
> >>>>> We have plenty of work to keep us busy for years with UC1 and
> UC3.
> >>>>> There's agreement that the technology needed is mature enough for
> >> IETF
> >>>>> standardization. I suggest that we scope this working group to
> >> address
> >>>>> only those use cases and that our use cases and architecture be
> >> limited
> >>>>> to them.
> >>>>>
> >>>>>
> >>>>> I'm sorely tempted to create an ambitious architecture for
> security
> >>>>> automation that encompasses all of the use cases that we have
> >> described
> >>>>> so far and maybe more. I understand that people have a hard time
> >>>>> understanding how the NEA and MILE and SACM standards fit
> together
> >> and I
> >>>>> would love to address that in an IETF RFC. But I have seen
> several
> >> grand
> >>>>> architecture efforts die or come to nothing in IETF. IETF is
> filled
> >> with
> >>>>> smart engineers with clever ideas. We're great at solving
> problems
> >> but
> >>>>> if
> >>>>> we can't agree on the problem to solve or if we choose the wrong
> >>>>> problem,
> >>>>> we can easily spin an intricate and pointless web.
> >>>>>
> >>>>>
> >>>>> I think we'll do better if we scope our effort properly.
> >>>>>
> >>>>>
> >>>>> Maybe a few people can create an individual submission describing
> a
> >>>>> grand
> >>>>> architecture and roadmap for security automation and ask people
> in
> >> SACM
> >>>>> to review and provide feedback on that document. That would be a
> >> good
> >>>>> way
> >>>>> to get a grand architecture while still ensuring that the SACM
> >> effort
> >>>>> keeps its focus. In IETF as in so many things, you must keep your
> >> focus
> >>>>> or you'll never succeed.
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Steve
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: [e-mail] [mailto:[e-mail]] On
> >> Behalf
> >>>>>> Of Waltermire, David A.
> >>>>>> Sent: Wednesday, August 15, 2012 1:13 PM
> >>>>>> To: Stephen Hanna; Adam Montville; Luis Nunez; Omar Santos
> >>>>>> Cc: [e-mail]
> >>>>>> Subject: Re: [sacm] Proposed use cases to move forward
> >>>>>> +1 on scoping the charter to UC1 and UC3.
> >>>>>> I think we are better served with a use cases document, and
> >> eventually
> >>>>>> an architecture documemnt, that is broader scoped.  Both of
> these
> >>>>>> documents will help inform the work we will do under the charter
> as
> >> it
> >>>>>> relates to the larger context.
> >>>>>> To this end I would suggest we focus on expanding the use case
> >>>>>> document primarily in the areas of UC1 and UC3 for now.  We can
> >> expand
> >>>>>> the other use cases later.
> >>>>>> Sincerely,
> >>>>>> Dave
> >>>>>>> -----Original Message-----
> >>>>>>> From: [e-mail] [mailto:[e-mail]] On
> >> Behalf
> >>>>>> Of
> >>>>>>> Stephen Hanna
> >>>>>>> Sent: Wednesday, August 15, 2012 11:24 AM
> >>>>>>> To: Adam Montville; Luis Nunez; Omar Santos
> >>>>>>> Cc: [e-mail]
> >>>>>>> Subject: Re: [sacm] Proposed use cases to move forward
> >>>>>>>
> >>>>>>> I agree. Let's work on UC1 and UC3. The other use cases are
> >> valuable
> >>>>>>> but just doing UC1 and UC3 is plenty of work for this group for
> >> the
> >>>>>>> next year or two (maybe five!).
> >>>>>>>
> >>>>>>> I saw several emails in favor of this a few weeks ago. I
> thought
> >>>>>>> that it was settled. I'd like to see a revised charter and use
> >> case
> >>>>>>> document, scoped down to focus on just UC1 and UC3.
> >>>>>>>
> >>>>>>> What do others think? Do we have rough consensus on this?
> >>>>>>> If so, let's get moving.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> Steve
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Adam Montville [mailto:[e-mail]]
> >>>>>>>> Sent: Wednesday, August 15, 2012 10:30 AM
> >>>>>>>> To: Adam Montville; Stephen Hanna; Luis Nunez; Omar Santos
> >>>>>>>> Cc: [e-mail]
> >>>>>>>> Subject: Re: [sacm] Proposed use cases to move forward
> >>>>>>>>
> >>>>>>>> On 8/6/12 7:27 AM, "Adam Montville" <[e-mail]>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> UC1 and UC3 both require assessment of endpoint state. If not
> >> for
> >>>>>>> the
> >>>>>>>> NEA
> >>>>>>>>> ties in UC1, it seems a subset of UC3.  So, we should be able
> >> to
> >>>>>>>>> start with the main concern of UC3: Security Configuration
> >>>>>>> Management.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I haven't seen much activity on this thread (there was another
> >>>>>>> thread,
> >>>>>>>> "Using the Frame of Reference," discussing some approaches we
> >> can
> >>>>>> use
> >>>>>>>> to keep us focused and meaningful).
> >>>>>>>>
> >>>>>>>> Are there any objections to tackling UC3 followed by UC1?
> Does
> >>>>>>> anyone
> >>>>>>>> disagree with my assertion that UC3 is a subset of UC1 and
> that
> >> a
> >>>>>>>> reasonable starting point is Security Configuration
> Management?
> >>>>>>>>
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> sacm mailing list
> >>>>>>> [e-mail]
> >>>>>>> https://www.ietf.org/mailman/listinfo/sacm
> >>>>>> _______________________________________________
> >>>>>> sacm mailing list
> >>>>>> [e-mail]
> >>>>>> https://www.ietf.org/mailman/listinfo/sacm
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> sacm mailing list
> >>>> [e-mail]
> >>>> https://www.ietf.org/mailman/listinfo/sacm
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> sacm mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/sacm
> > _______________________________________________
> > sacm mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/sacm
> >
>
>
> _______________________________________________
> sacm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/sacm