Re: [scap_interest] Gaps in Risk Management

"Lloyd, Mike" <> Wed, 15 February 2012 15:07 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 0F73121F86B6 for <>; Wed, 15 Feb 2012 07:07:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.523
X-Spam-Status: No, score=-2.523 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, HTML_MESSAGE=0.001]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 82cvvAX4eEjY for <>; Wed, 15 Feb 2012 07:07:14 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id A1D6E21F866C for <>; Wed, 15 Feb 2012 07:07:14 -0800 (PST)
Received: from localhost (localhost []) by (Postfix) with ESMTP id 47A2D9EED95; Wed, 15 Feb 2012 07:07:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id NbPqFSW-Ionp; Wed, 15 Feb 2012 07:06:56 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id D30BF9EEE0D; Wed, 15 Feb 2012 07:06:56 -0800 (PST)
From: "Lloyd, Mike" <>
To: "'Luis Nunez'" <>, <>
References: <> <> <> <> <>
In-Reply-To: <>
Date: Wed, 15 Feb 2012 07:06:56 -0800 (PST)
Message-ID: <>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_17CC_01CCEBB0.6595A860"
X-Mailer: Microsoft Outlook 14.0
X-Mailer: Zimbra 7.1.3_GA_3374 (ZimbraConnectorForOutlook/
Content-Language: en-us
Subject: Re: [scap_interest] Gaps in Risk Management
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion List for IETFers interested in the Security Content Automation Protocol \(SCAP\)." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 15 Feb 2012 15:07:20 -0000

(Hopefully this is not a duplicate for most folks - I wrote a note below
while not yet a list member, but have since joined & cancelled my earlier
attempt to send.  I would be most interested in SCAP and metrics related
discussion with anyone attending either BSides or RSA in San Francisco




From: Lloyd, Mike [] 
Sent: Tuesday, February 14, 2012 3:36 PM
To: 'Luis Nunez'; ''
Cc: ''com'; ''
Subject: RE: [scap_interest] Gaps in Risk Management


Absolutely - thanks for adding me to the thread, Luis.  As Jerome noted,
I'm speaking at BSides - an event I've found can be quite worthwhile.
I'll also be speaking on the show floor at RSA (at the RedSeal Networks
booth), but the show floor talk will necessarily be much less detailed.
I'd be happy to discuss metrics with members of this group at either of
these venues, or online.


Some recent comments I made on metrics showed up here:


For folks thinking about SCAP, I didn't highlight SCAP in that interview,
but I do see SCAP as a fundamental building block of the metrics I
recommend.  Specifically, correlation of CVE instances and correlation
with the CVSS framework are fundamental.  However, I find I have to build
a few layers on top of that - at first into the "environmental" component,
but eventually beyond that, because I need to generate not just metrics on
vulnerabilities, but also on hosts, and even for an entire network.


Anyone who has a BrightTalk login, or is willing to register for one, can
also review a recent webinar we recorded - a co-presentation by Mike
Rothman of Securosis (who covers general themes and difficulties) and
myself (covering our actual measurement approach, in a short segment at
the end):





From: Luis Nunez [] 
Sent: Tuesday, February 14, 2012 2:59 PM
Cc:;; Mike Lloyd
Subject: Re: [scap_interest] Gaps in Risk Management



great idea and thanks for mentioning this talk. It would be an interesting
discussion to have with Dr. Mike on the topic.  Lets hook up.


Since you mentioned Geer.  Here is an excellent talk Dan recent gave.




On Feb 14, 2012, at 5:47 PM, <> wrote:


Hi Jerome,


I think it is a really good idea to start from use cases and what we want
to get out of this work to drive the standards efforts.  The work being
done here should result in the ability to report on meaningful and
actionable metrics.  Brainstorming like this at the RSA Conference and on
list would be great.  Activities like this may be tied to the frameworks
as well - how do we get good risk metrics against the frameworks in which
we want to measure our security programs? 


Luis brought up CSA's control matrix, maybe we want to think about what
was done for 800-53 and how that could apply out to other frameworks like
ISO27002 +ISO27017 (Virtualization/cloud), HIPAA HITECH, etc.  performing
checks, assessing risk, and producing metrics are the drivers for these
underlying activities.


Dan Greer has some good presentations online related to metrics and is
running a series in IEE Security and Privacy on the topic as well.  I do
need to admit that I have to catch up on my reading, so I can't tell you
too much about that effort J


Any other thoughts on these topics?





[] On Behalf Of Jerome Athias
Sent: Tuesday, February 14, 2012 5:34 PM
Subject: Re: [scap_interest] Gaps in Risk Management


Hi list,

could it be possible to have a brainstorming at the RSA Conference?

or / because i identifed the following talk (at BSides San Francisco, //
free event):

Metrics That Don't Suck: A New Way To Measure Security Effectiveness ~ Dr.
Mike Lloyd

How does your organization measure and report its security posture and
performance?  Do you have spreadsheets that show how many vulnerabilities
you found last month, or how many viruses your AV system stopped? Those
numbers might pacify your management, but any security pro can tell you
that they are no way to benchmark the real work you do - or how much
danger your enterprise might be in.

Maybe the problem is that we're all trying to use the data we already have
- host metrics, network metrics, applications data - instead of building
the data we actually need.  We need metrics that show the current range of
threats, and the enterprise's exposure. We need data that shows whether
our security tools and programs are actually working or not. We need
methods for demonstrating that our security teams are performing well -
not only this month, but over a period of time.

In this thought-provoking presentation, we'll describe methods for
building an enterprise security metrics program that's completely
different from the current, sucky model of counting vulnerabilities or
numbers of patches applied. We'll outline methods for monitoring the
threat landscape, and your organization's exposure. We'll offer some best
practices for measuring the effectiveness of current security tools and
systems. Best of all, we'll outline a way to build a maturity model for
security, so that you can show your security team's performance on a
month-to-month basis, and demonstrate its continuing improvement over

Want to stop reporting a bunch of crap and start building a real set of
data that accurately measures your organization's risk and its
effectiveness in controlling it?  Want to learn how to integrate security
data across hosts, networks, and applications?  Want your performance -
and your company's security posture - to be monitored using metrics that
don't suck?  Here's a chance to look at the picture from a whole new

Speakers-and-Topics-Lineup.html )

I see these events as a good place to exchange our ideas.

Just my 2 cents


scap_interest mailing list