During many of my customer meetings, I often hear security leaders ask the question: “What technology could I remove to free up budget to enable the implementation of FireEye?”
My natural response is to inquire how and when they assess the real value, not the ROI, they get from their existing solutions. Whilst every security solution provides an “ROI” – often a metric based around industry data on how many security “events” they return – this assessment should not focus on noisy “ROI,” but which solution gives your company the most valuable information. Considering the nature and pace of change when it comes to malware and advanced attacks, this is something to validate regularly and involves looking at more factors than a generic ROI tool can factor in.
In a small survey of about 30 European CxOs we ran in December 2013, I asked the question of how they validated the value of security controls, and, surprisingly, at least 36 percent still didn’t conduct any annual assessment.
Having spent quite a bit of time looking at analyst models and what exists publically today, the fact that some still don’t conduct these assessments emphasizes that there still isn’t a well-defined model to correlate business value against investment for security solutions.
Take, for example, the outsourced model where Key Performance Indicators (KPIs) are typically established. Too often I hear anecdotal examples where KPIs were based on incidents found; this simply encourages dialing-up the technologies being monitored so that every incident – malicious or not – is tracked and reported, drowning out the ability to identify real threats.
For example, companies investing in big data solutions that gather and equate the millions to billions of events delivered each week to value. However, because they are too resource-constrained to convert these into actionable data, the true value is not extracted and, because doing so in a resource-constrained environment takes so long, the return here would seem extremely poor to a sheer numbers-based evaluation.
However, if the value of said product is measured by the actions taken to mitigate a major security event, that extra time spent executing on a few major items rather than not executing on a large amount of items becomes invaluable to the business. As such, we must blend together the quantitative metrics such as the costs of a solution (capex & opex), incident levels and overlay those values with qualitative insight. These evaluation criterion would look something like the below:
- Noise to incident ratio – What is an acceptable incident to noise (i.e., false positives or irrelevant alerts) ratio?
- Volume versus impact - We can alert and respond to a million incidents but it’s the one outage of a critical system or breach of business IP or customer records will have a far more significant impact on the business.
- How actionable is the solution – Critical to an alert is timeliness, how long does it take to identify an incident and what level of human skills are required to interpret the results. Spotting a breach is hard, doing the forensics is harder, and understanding the motives of the attacker is harder still.
- Business outcome – Did a technology mitigate, reduce or simply delay the business impact. Did it tick a compliance control to avoid penalty fees or did it protect IP & customer data?
In the coming months we are going to delve into the economics of security in greater detail. Whilst there are many tools out there discussing security process and ROI tools showing generic return, we all have limited budget and resources. With the scale and scope of security tools continues to grow we must innovate our thinking, in how we each quantify the value of our investments.
No comments:
Post a Comment