[Cacao] Charter Feedback

[Roman Danyliw <[e-mail]>]

Hello!

I've reviewed the latest (01/18/2019) charter language per https://mailarchive.ietf.org/arch/msg/cacao/FErdmnk2MhOqG7kkDk2oBIthbqQ.  As is, I have a few concerns.
 
In broad strokes: 
** The notional output of the WG isn't clear
** There are no references to related work in the IETF (e.g., MILE, I2NSF) or in the community

See inline comments of the charter text for additional comments.

> # Problem
> Threat Actors and Intrusion Sets are advancing at an increasing rate
> relative to an organization's ability to defend against and respond to cyber
> attacks. 

Editorial: "advancing" how?  I'm stumbling with this opening sentence.

> In addition, it is common that defenders need to manually identify
> and process prevention, mitigation, and remediation steps in order to
> protect their systems, networks, data, and users. 

I agree with this text in that manual identification of what to do in response to an attack is prevalent.  However, I push back on the "need" to manually process prevention, mitigation and remediation steps -- there is a lot of options for IT automation/SecOps (ansible, puppet, etc.) once the action is encoded.

> There is also currently no
> standard means to easily and dynamically share proposed prevention,
> mitigation, or remediation steps and the operational experience gained
> from these attacks or their associated successful responses among a trusted
> set of organizations.

Concur.  It's the lack of a standardized approach to share these COAs across organizations and interoperability across tech stacks which strikes me as a/the key issue.

> >
> > Due to the increasing sophistication and amplitude of cyber attacks the
> need for a secure collaborative set of systems providing coordinated
> detection and response across hosts, networks, and security infrastructure
> has raised significantly. This solution is necessary to effectively respond to
> threats in machine relevant time. While some attacks may be well known to
> certain security experts and researchers they are often not documented in a
> way that would enable automated prevention, mitigation, or remediation.
> >
> > Key to this coordinated cyber attack response is a coordinated threat
> response including a standard information model; a set of functional
> capabilities, associated interfaces, and protocols. These key requirements
> would be defined in each of the system components across host; network
> and security infrastructure to ensure that each system can work together in a
> coordinated manner.

For me, these two paragraphs didn't add anything describing the problem.  They read more as a solution to the first paragraph which seems more appropriate in later parts of the charter text.

> > # Working Group
> > To enable efficient collaboration and facilitate the rapid sharing of
> preventative, mitigative, and remediative actions this working group will
> focus on defining the set of technologies (protocols, interfaces, functional
> capabilities, and information model) required to detect, prevent, mitigate,
> and remediate threats. 

IMO, the statement "[T]his working group will focus on defining the set of technologies to detect, prevent, mitigate and remediate threat" seems too broad.  This WG seems to be proposing a particular solution, automated COA exchange.  Furthermore, I see no deliverables related to threat detection.

Editorial: The sentence states "prevent, mitigate and remediate" twice.  

Editorial: I recommend the first sentence of this section to be a declarative statement of what this WG would do.  The simpler language used in the slide #2 of the 10/24/2018 Intro Webex, https://mailarchive.ietf.org/arch/msg/cacao/-cgdaJG6uM71IeNwp3oxl6rRQiE, is to the point.

> This solution will also define the machine-readable
> actions to enable an action-oriented defensive system. This effort will focus
> on providing the functionality requirements for each system that would
> participate in a coordinated threat response; the interfaces they should
> support including the transport mechanism used and finally the information
> model across those systems to enable the coordinated actions in a
> structured secure manner.

This is the third time in the text where the list of functional requirements, interfaces, and info model is enumerated as a need.  The first time was in the Problem section, the second was parenthetically in the previous paragraph, and it will again be discussed in the Goals and Deliverables.  I don't think this paragraph is needed.  

> > Each collaborative course of action will consist of a sequence of cyber
> defense actions that can be executed by the various systems that those
> actions target. Further, these COAs can be coordinated and deployed across
> heterogeneous cyber security systems such that both the actions requested
> and the resultant outcomes may be monitored and verified. These actions
> will be referenceable in a connected data structure that provides support for
> connected data object and efficient operational use of those data objects
> such as Threat Actors, Campaigns, Intrusion Sets, Malware, Attack Patterns,
> and other adversarial techniques, tactics, and procedures (TTPs).

What are "data objects"?  I recommend merely describing the class of data that needs to be exchanged in the COA; and with whom and avoid further specificity that would require additional definitions. 

Why are "Threat Actors, Campaigns ..." capitalized like proper nouns in this context (I appreciate that these are STIX objects).

> > Where possible the working group will leverage existing efforts that *may*
> define the atomic actions to be included in a process or sequence. 

I recommend you specify these related efforts.

>  The
> working group will not consider how shared actions are used/enforced,
> except where a response is expected for such a received shared action by a
> receiving party. It will also focus on the requirements for the correct
> construction and correct distribution of the structured actions and their
> corresponding interfaces and protocols.

Is "requirements for the correct construction and ..." the same thing as specifying a MTI protocol or data model?

> > # Goals
> > This working group has the following major goals:
> > 	* Document the use cases and requirements
> > 	* Identify and document the system functions and roles that must
> exist with associated protocols for a coordinated threat response system to
> operate effectively

I initially read this as an informational architecture draft per the deliverable list.  However, the language "must exist" suggests something stronger.  Is this the architecture draft?

Are the "associated protocols" different than the ones that would be specified by this WG?

What is the dependency between the "roles"/"system functions" documentation and the data model and protocol work?

> > 	* Identify and document the configuration for a series of protocols
> that can be used to distribute courses of action in both direct delivery and
> publish-subscribe methods

"Identify and document" reads to me like a new protocol won't be specified.  However, the deliverable list states that there will be a "CACAO Distribution and Response Application Layer Protocol".  Could you please clarity.  If you intend to make a standard track document with a protocol, I recommend you use the verb "specify"/"standardize"

> > 	* Identify and document the mechanism(s) required to monitor,
> report and alert on effective distribution of CACAO actions and the potential
> threat response to those actions

To which deliverable does this map?  What are "mechanisms"?

> > 	* Create an information and data model that can capture and
> enable collaborative courses of action (sometimes called playbooks) that
> will be used in the coordinated threat response systems

This text references information and data models, the deliverables only mention a data model.  Is the information model going to be specified in the JSON draft?

> > 	* Define and create a series of tests and documents to assist with
> interoperability of the various systems involved in the coordinated threat
> response system.

This bullet doesn't appear to have a corresponding item in the deliverable section.  Furthermore, per the language "series of tests and documents" how will the "tests" manifest if not in a document?  If it's not in a draft, it is not a WG work item.

> > # Deliverables
> > The working group plans to create informational and standards track
> documents, some of which may be published through the IETF RFC stream.
> > 	* CACAO Use Cases and Requirements
> > 	* CACAO Functional Architecture: Roles and Interfaces
> > 	* CACAO Interface Specification
> > 	* CACAO JSON Data Model

draft-jordan-cacao-introduction made reference to a CBOR data model.  Is that still in scope?

> > 	* CACAO Distribution and Response Application Layer Protocol
> >

IMO, it would be clearer to collapse the current list of goals with the deliverables.  

Roman