This post is also available in: 日本語 (Japanese)
The following post on SOC metrics is adapted from the book, “Elements of Security Operations,” a guide to building and optimizing effective and scalable security operations. Download a free copy today.
Some metrics that security operations centers (SOCs) widely use to evaluate their performance have the potential to drive poor behavior.
One example is mean time to resolution (MTTR). This is a fine metric when used in a network operations center (where uptime is key) but it can be detrimental when used in a SOC. Holding security analysts accountable for MTTR incentivizes them to rush to close incidents rather than rewarding full investigations that feed learning back into the controls to prevent future attacks. Similarly, ranking performance by number of incidents handled may lead to analysts “cherry picking” incidents that they know are fast to resolve. This will not produce better outcomes or reduced risk for the business.
Another poor metric is counting the number of firewall rules deployed. 10,000 firewall rules can be in place, but if the first bypasses the rest (e.g., any-any), then they are useless. This is similar to measuring the number of data feeds into a security information and event management platform (SIEM). If there are 15 data feeds into a SIEM but only one use case, then the data feeds aren’t being utilized and are a potentially expensive waste.
When determining good metrics for your business, always keep in mind the mission of the SOC and the value it provides to the business. The business wants confidence that the SOC can prevent attacks and that if/when a breach does occur, then the team is able to handle it quickly, limit the impact and learn from it. Good metrics should provide insight into whether the business should have confidence or not. There are two types of confidence to focus on: configuration confidence and operational confidence.
Configuration confidence is knowing that your technology is properly configured to prevent an attack, that you can automatically remediate it and/or that the proper intelligence can be gathered for analysis by a human. Example questions to answer are:
Operational confidence is knowing that the right people and processes are in place to handle a breach if/when it occurs. Example questions to answer are:
Metrics should be used to improve protections and provide confidence to the business that the security operations organization is executing on its mission – which requires measuring quality, not just volume. Each metric has specific and limited value; no one metric tells the whole story, but together, they can help drive continued improvement and confidence that the business is properly set up to prevent and contain a breach. To learn more best practices for building effective security operations, download a free copy of our book, “Elements of Security Operations.”
Watch for future posts in Kerry Matre’s series on “Elements of Security Operations.” Next up: "3 SecOps Strategies to Enable Your Smart People to Focus on Smart Things."
By submitting this form, you agree to our Terms of Use and acknowledge our Privacy Statement. Please look for a confirmation email from us. If you don't receive it in the next 10 minutes, please check your spam folder.