Good news: incidents are down. Better news: you know the countermeasures you’ve put in place over the past six months have been responsible for much—if not all—of that downward trend. If you could only get that budget increase approved, you know you could accomplish even more. But here’s the bad news: no one except you cares—unless of course you can show them the proof.
And although it may sometimes seem like the C-suite makes decisions based on hunches, the truth is there’s no room for ambiguity in the board room. It doesn’t matter if you know you’re right… You need the numbers to back you up.
Enter metrics.
Metrics take advantage of data to produce usable, objective information and insights that guide decisions and provide you more clarity into the problems you’re trying to solve. Understand risks and losses, discern trends, manage performance—you can do it all with metrics.
And if you can do it right, you’ll also be able to use metrics to report clearly and accurately to executive management, and present your findings in a way that demonstrates the value of your security team and delivers persuasive, justifiable rationale for decision-making.
But here’s the million-dollar question… What are the metrics that matter?
This definition from the ASIS Foundation puts it plainly:
“Security metrics support the value proposition of an organization’s security operation. Without compelling metrics, security professionals and their budgets continue largely on the intuition of company leadership. With metrics, the security function grounds itself on measurable results that correlate with investment, and the security professional can speak to leadership in a familiar business language.”
ASIS goes on to explain, however, that although security metrics are vital, the industry has settled on few tested metrics, and little guidance exists on using metrics effectively.
In other words, no one can agree on which ones are important—and they’re not always sure how to use them.
In the age of “indicator overload,” it’s easy to set up dashboard after dashboard that is supposed to tell you… something… but ends up only confusing you.
Perhaps it’s best to start at the most basic— defining what “metrics” actually means.
In Security Metrics Management: How to Manage the Costs of an Assets Protection Program, Koranic and Halibozek (2005) define a metric as “a standard of measurement using quantitative, statistical, and/or mathematical analyses.”
In their taxonomy, a security metric is, “the application of quantitative, statistical, and/or mathematical analyses to measuring security functional costs, benefits, successes, failures, trends and workload—in other words, tracking the status of each security function in those terms.”
On the other hand, the National Institute of Standards and Technology (2008) states that “while a case can be made for using different terms for more detailed and aggregated items, such as ‘metrics’ and ‘measures,’ [this report] standardizes on ‘measures’ to mean the results of data collection, analysis, and reporting.” The same source refers to the process of data collection, analysis and reporting as “measurement.”
But whatever you decide to call metrics, the fact remains that you’re likely not using them.
One source suggests that only about a third of CSOs collect and analyze metrics (Kohl, 2009). Specifically, in a survey by the Security Executive Council (SEC), only 31 percent of survey respondents “gather security program data in order to create statistical reports to present to senior management.”
According to Peter Ohlhausen of Ohlhausen Research, that is a mistake. In a recent Resolver webinar, Peter states: “The use of metrics leads to decisions that are scientifically and logically more valid than intuition—and they tend to be more persuasive to others, especially senior management. In a very simple bottom line sense, metrics are tools for making good decisions—and for convincing others that those decisions are good.”
Thinking about useful metrics often requires thinking “outside of the box.”
You might imagine that the best metrics are the ones that track the issues you’re interested in. For example:
But unorthodox measures could be just as helpful—or more so—when trying to demonstrate your value to senior management.
For example, one organization examined in the report that using only data collected when employees badge in and badge out helps its senior management better allocate office space. Another organization tracks cease-and-desist letters and their effect on hundreds of websites a year that violate compliance and infringe on its intellectual property.
A full discussion of 16 peer-reviewed metrics—including expert commentary on their effectiveness— is available in Persuading Senior Management with Effective, Evaluated Security Metrics, by Peter Ohlhausen et. al. Reading the report and its examination of security metrics could go a long way toward improving your program.
In the same webinar mentioned above, Ohlhausen recommends five tips for presenting metrics to senior management.
You may have a vast collection of metrics that are useful to you internally, but if they’re not going to be of interest to senior management, it goes without saying that you’ll be wasting everyone’s time in the boardroom.
The best metrics, then, are the ones that your C-suite cares about.
This may seem like a tall order, but fortunately the Ohlhausen report also offers a tool to gauge the robustness of your metrics using a wide variety of factors, including:
All of these factors are explained further in the report.
And in case “measuring measurements” leaves you feeling like you’ve fallen down the rabbit hole, remember this, in the words of Ohlhausen himself: “Your metrics will be more convincing— and will in fact be better tools for your decision making” if they meet scientific rigor and can be weighed against standards.
To keep senior management interested, your metrics should be telling of the unfolding of events over time.
Don’t overdo it however—remember that you’ll typically have only five to ten minutes to present your metrics.
Data gets old—fast. That means it can be beneficial to get in front of senior management regularly if you can. Annually is not enough; strive for monthly, or at the very least quarterly.
You understand by now that you need to separate what the C-suite needs from the in-depth analytics your security leadership relies on.
And you’ve learned how to think about your incident data in a whole new way—and how to produce meaningful metrics that remove assumption and instinct from the conversation. But getting started can be difficult.
You’ll most likely encounter one of two barriers:
The first potential hurdle—that management has not made a specific request for metrics— shouldn’t matter. Senior management may not know that metrics are the best way to obtain good results, but that doesn’t negate your responsibility to manage risk and to inform management on your status. Take metrics to them—don’t wait to be asked.
The second challenge—the cost of measuring—is also less of a problem than you might first imagine.
Dedicated security software, for example, can make the process efficient and enable you to gather metrics without any additional staff. With the right software, data can be collected and input by numerous staff members, leaving you time to conduct the necessary analysis—and, coincidentally, also making that analysis much easier.
Once the hurdles are out of the way, it’s a relatively simple process: identify your objectives, determine who you need to report to, hone in on what they want, and find a way to give it to them.
Do so while also demonstrating your contribution to enterprise risk management or the company’s overall strategy and objectives— or, ideally, both—and you’ll be ahead of the game. Transforming mountains of data into metrics (and insights) that both benefit your organization and consistently prove your value.
