Objective measurement is important for monitoring security performance, especially since the modern threat landscape is constantly evolving. According to the SANS Institute, leveraging a comprehensive security metrics program enables organizations to achieve several goals, including improved decision-making, enhanced visibility, the ability to evaluate an internal security program against industry benchmarks, and demonstrating the value of the security department to key stakeholders and throughout the organization.
The first step to implementing a comprehensive security metrics program is identifying what metrics to measure and then, where to obtain the raw data. To help you determine what metrics to measure, we reached out to a panel of security pros and analytics experts and asked them to answer this question:
"What are the most important cybersecurity metrics used in organizations today?"
Meet Our Panel of Security and Analytics Professionals:
Find out what security metrics your organization should be monitoring by reading our experts' responses below.
Ian McClarty
Ian McClarty is the President of PhoenixNAP Global IT Services.
"Cybersecurity metrics vary depending on business needs and the risk aversion level of the organization..."
An example of this would be: Company A, which has a small Internet-facing presence but produces valuable goods or information, may be interested in tracking all attacks on their Internet-facing services, while Company B has an extremely large Internet-facing presence which is perpetually under attack. Company B may take the stance of focusing their tracking on internal systems and networks to better understand potential issues inside the perimeter defenses.
Defining “good” metrics can be difficult. Those developing metrics should take in business considerations, such as what information needs to be protected and why. Metrics collected and reported on should follow something similar to the “SMART” structure:
- Specific - Targeted to the area being measured, not a byproduct or result.
- Measurable - Data collected is accurate and complete
- Actionable – Easy to understand the data and to take action
- Relevant – Measure what’s important with the data
- Timely – The data is available when you need it
When defining metrics, organizations must attempt to avoid some of the most common mistakes. Lack of management’s commitment to make changes based on metrics, measuring too much, too soon, too little, or too late, measuring the wrong things, imprecise metric definitions, using data to evaluate individual or personnel performance, using metrics to motivate rather than to understand, collecting data that isn’t used, lack of communication and training, and misinterpreting data are all pitfalls many organizations face.
The most simplistic approach would be to look at the overall Information Security program maturity from a top-down view. Identifying the main categories that the organization needs to measure is the first step. Sub-metrics then contribute to the overall scores of the main categories. This will give one of the most holistic views of the organization's global metrics.
Organizations may want to look at how they measure some of the following areas (in no particular order; these are only a small portion of what can be measured):
- Vulnerability Data
- Internal vs. external vulnerabilities
- Vulnerabilities by criticality/severity/priority ratings
- Vulnerability aging
- Security Policy/Compliance Adherence
- “Exception” tracking and documentation
- Configuration compliance tracking
- Firewall/switch audit data
- Patching levels
- Regulatory control compliance
- Training and Awareness
- Track employee completion statistics for any required training
- Information Security training
- Monitoring and Response
- Tool performance and availability metrics
- Amount of data being collected
- # of events/alerts/etc. being collected
- # of resulting “incidents” and data to incident conversions
- Mean time-to-detect (MTTD)
- Mean time-to-respond/remediate (MTTR)
- Virus/Malware Metrics
- Project Completion Metrics for New Tools or Services
- Personnel/Resource/Hiring metrics
Of course, all of the items mentioned above would need to be driven and accepted by the organization. If the items above don’t meet the SMART criteria, they should be tabled until the company can accurately and meaningfully measure the component.
Guy Dulberger
Guy Dulberger is an Information Security executive with over 15 years of battle scars to prove it. He works for Ritchie Bros as Director of Security. He is passionate about security, investigation, and justice in general.
"I think an important aspect of what makes a great security report KPI or metric is..."
Understanding your business and where your greatest organizational risk lies. Spend the time understanding your risk before composing a list of your information security objectives. You can then report based on those objectives. It's also important to note that your executive management or board of directors will lose their attention if you keep your KPIs or metrics too long. Important ones in general are:
- Number of incidents that were reported in the period especially if they lead to non-conformities
- Malware
- Outages as a result of attacks (ex: DDoS, ransomware, disgruntled employee)
- Lost or stolen corporate devices
- 3rd-party related incidents
- Number of staff taking security awareness and average scores. Try to break by department or verticals to isolate a problematic group and cater your security awareness material to their needs
- The effectiveness of the training program. Is there a reduction of incidents as a result of the training?)
- Vulnerability scan report based on criticality – number of Critical and High as well as exploitable vulnerabilities found on the network through regular scanning. This can also be a list of the current states of critical and security patches deployed
- 3rd-party vendor risk assessment
- List of top critical vendors and their risk ratings based on:
- Volume of information
- Type of information processed (ex: PII, PHI, PCI etc)
- Size of the commitment
- Criticality of the service (how important are they for your organization)
- Ease of replacement (if they were to suffer a prolonged outage or dissolve)
- Brand reputation such as through social sentiments
- Threat intelligence (monitoring on brand, domain, and individuals)
Bryce Austin
Bryce Austin, CISM is the CEO for TCE Strategy and a leading voice on emerging technology and cybersecurity issues. With over 10 years of experience as a Chief Information Officer and Chief Information Security Officer, Bryce actively advises the boards of companies in industries as diverse as financial services, retail, healthcare, technology, and manufacturing industries. He was the CIO and CISO of Wells Fargo Business Payroll Services from 2004 to 2012 and a Senior Group Manager at Target Corporation during the 2013/2014 PCI data breach. Bryce holds a CISM certification and is known as a thought leader, cybersecurity expert, and internationally-recognized professional speaker. He started his technology career on a Commodore PET computer and a cassette tape drive.
"The most important metrics used in cybersecurity today are..."
- Number of unidentified devices on the internal network. Internet of Things devices and employee-owned devices are making their way into internal networks with increasing regularity. These devices represent a serious risk, in that they are unlikely to have strong antivirus systems, up-to-date patches, and good network segmentation. In order to understand this threat, the devices have to be identified.
- Number of cybersecurity incidents reported by end users to the cybersecurity team. All companies have cybersecurity issues. Good companies will welcome the reporting of those issues, and higher numbers are better in this category.
- Cybersecurity awareness training results. Cybersecurity training is pivotal for all employees, regardless of their position in the organization. Often senior executives are overlooked in this training, which is the worst possible choice of people to exclude, as many new spear phishing attempts are made on senior executives.
- Mean time between security patch release and implementation. Patches must go in quickly to be effective, and must be verified to have been installed across all computers. Exactly the way that Equifax didn't.
- Number of known vulnerabilities on externally facing systems. External scans can reveal these.
- Number of known vulnerabilities on internal systems. These are often ignored by organizations and they shouldn't be.
- Results of SIEM (Security Information Event Monitoring) results. A good SIEM system will act as a dashboard of activity on a network, and is important to collect trending information from it.
Mihai Corbuleac
Mihai Corbuleac is a Senior IT Consultant at Bigstep Inc., an IT company providing full-stack big data ecosystem running in a high-performance bare metal cloud.
"Measuring ‘security’ is quite tough, but there are several security metrics that matter..."
A comprehensive security audit should include relevant security metrics, such as data breach response metrics – time, plan effectiveness, number of security incidents based on specific severity levels, and types of incidents, such as malware infection, unauthorized access, destructive attacks, persistent threats, etc. You should also measure how many devices have relevant scanning tools deployed. In case your system actually gets infected, try to accurately determine how exposed your business is, and you should have your own metrics for that. If your multi-layer security has been breached, measuring how far they got will help you prevent future attacks. You would be surprised how sensitive data is handled sometimes; this is why business security experts constantly check for unapproved storage of sensitive information. Another important aspect is to always compare with your main competitors the percentage of your budget actually spent on securing your business (make sure that’s more than 5%).
Rick Deacon
Rick Deacon is CEO & Founder of Apozy, a YCombinator backed cybersecurity company whose aim is to stop phishing and malware in the browser while creating a real-time browser forensics and incident response platform. Rick became a security professional early on, hacking into Fortune 500 companies and securing their networks.
"Cybersecurity metrics are important. They're so important that companies are providing big data, AI, machine learning, and managed services just to help saturate you with data..."
Security professionals need to focus on the basics and keep track of the critical hit list of metrics if they want to make use of them. Here's the critical list every company needs to follow, from startup to multinational conglomerate.
- Device Inventory and Health: It's 10PM - do you know where your devices are? Or how many there are? Or what they're running? You'd probably be surprised. Keeping an up-to-date device inventory is critical to understanding security posture. By tracking where machines are and what they're running, you have a basis for any other metric. A new metric to this category is browser data - what browser, what versions, and what extensions they're running. This can be accomplished via fingerprinting or services.
- Attack & Threat Frequency : How many attacks do you see every day? How many phishing emails come in? This information helps security pros properly adjust decisions and focus on what matters most. Is a new firewall necessary if all you're seeing is social engineering? By tracking attacks on the network, on the machine, application layer, and in the browser, you can understand what needs to be fixed first.
- Incidents & Responses: How many attacks succeeded? How quickly were they handled? Did they spread? Where did they originate? These types of questions can be answered by tracking the number of incidents your SOC and IR teams encounter. Often times security incidents require significant effort just to recover from them, so understanding numbers falls to the wayside. The browser is becoming the new OS and incidents within it need to be tracked very closely alongside the existing attack surface.
- Patching & Patch Latency - How many machines did we patch and how quickly? This is one of the most basic questions with the most significant impact. Without these metrics, your company becomes the next Equifax or victim of WannaCry. Tracking time to patch and the number of updated machines should be near the top of your list of important metrics.
Donny C. Shimamoto
Donny Shimamoto was the first Certified Information Technology Professional (CITP) in the State of Hawaii, and is one of only four in the state. The CITP credential is a specialty designation of the American Institute of Certified Public Accountants (AICPA) that identifies Certified Public Accountants (CPAs) with the unique ability to bridge between business and technology; meeting the strict requirements for a CPA license as well as additional training and experience in: technology strategic planning, IT architecture, business process enablement, system development and acquisition, IT audit and control, and IT governance.
"Organizations need to track a mix of..."
- Direct technical security metrics that help prevent an incident (e.g. critical patches not installed, machines with out-of-date antivirus solutions, intrusion attempts, etc.);
- Indirect technical security metrics that help with recovery from an incident (e.g. successful backups, resolution of backup issues); and
- Non-technical security metrics that also help prevent an incident (e.g. end user training completion, end user response to phishing attack testing, number of exceptions for compliance with policy and procedure by IT, etc.)
Just as a layered defense is essential to technical security, a multi-faceted metrics approach is essential to measuring the effectiveness of an information security program.
Kean Graham
Kean Graham is the CEO of MonetizeMore, a leading ad tech firm that is a Google Certified Partner.
"The top 2 security metrics to track every day are..."
- Non-human traffic (NHT): Many websites and business can be compromised by bot traffic otherwise known as NHT. It’s important to detect NHT on a daily basis so you are aware of any spikes to check if anything was compromised. You can take further steps and block NHT with certain softwares.
- Average Time to Upgrade: A lot of technology gets compromised when they have not upgraded to the latest version. Therefore, companies should measure the average time it takes between the time an update is available and until it is actually upgraded. For example, Wordpress sites should minimize this metric to as little average time as possible to keep their site as secure as possible.
Pieter VanIperen
Pieter VanIperen is a Founding Member of Code Defenders, a collective that protects the long tail of the internet, an Adjunct Professor of Code Security at NYU, a Certified Penetration Testing Engineer (Ethical Hacker) and a Certified Secure Web Application Engineer. He is a veteran programmer and security expert. He is currently a resident software architect and secure coding expert for a major online discount brokerage. He has also consulted for multiple financial, insurance and law enforcement institutions. He has worked in over 20 programming languages and is the author of the HAZL programming language. He has also served as the CTO of several digital companies and has advised multiple startups.
"Normally, I would talk about attack surface, points of vulnerability, and high risk results from static analysis and pen tests..."
But given current events like KRACK and Equifax, I think the most important metric should be days to patch. That number should be shrinking across all layers of the tech stack until it is in the single digits. For a long time now, most major attacks against corporations have exploited known and published vulnerabilities, not zero days and most frequently within a window of time after a patch has been released. Most security patches, if you are creating securely coded products, should not have an effect on functionality. Further, many times, patching can be fully automated. So in a sense, this shouldn't even be a metric, but unfortunately it is in most organizations, and the numbers are usually not in the single digits.
Gregory Morawietz
Gregory is an IT Security Specialist with over twenty years of network and security experience. He has worked with hundreds of firms on improving IT environments, consulting and integrating technology for the enterprise network. He is the VP of Operations at Single Point of Contact.
"There are several important security metrics..."
Login attempts against servers and other infrastructure items should be monitored as well as the number of stolen laptops or physical security violations. Pay attention to the number of people who have taken security training and the number of servers with security vulnerabilities. There are metrics that cover your network environment, server environment, and user environment. Each of these areas have critical metrics that should be monitored and observed. If you have security infrastructure tools, there are a host of metrics they provide, but you have to own that technology. How often outsiders attempt to gain access to your network is the main thing you need to keep in mind and monitor. All the different methods that are used to accomplish this should be discovered and watched.
Zack Stark
Zack Stark is the owner of Light Speed IT, a locally owned private IT firm located in Fort Worth, Texas. They offer technology solutions for growth-oriented small and mid-sized businesses. They also work alongside your current IT department to monitor the health of your network and relay critical errors proactively to your team to ensure maximum uptime.
"As you know, cybersecurity is becoming an increasingly important consideration for businesses and a hot topic in the media; especially since the recent Equifax data breach as well as Sonic’s customer credit card breach..."
There are three metrics that I would consider most important. Number one on the list would be response time for incidents. In other words, how quickly can an organization identify and remediate any potential vulnerabilities or security threats? This is something that can be internally tracked or tracked with the use of software.
Secondly, high risk items requiring remediation since the last audit or assessment is another significant metric. Once potential vulnerabilities are identified, is the organization or IT provider acting on these recommendations, and what does that implementation time look like?
Lastly, patching frequency is huge. Oftentimes, major data breaches that occur could have been prevented had a simple security patch been applied. Measuring patch frequency can help identify the possibility of vulnerabilities ahead of time. If patching frequency is not an option, implement other security measures such as firewall-based filtering.
Muhammad Habibullah
Muhammad Habibullah from Lahore, Pakistan, went by the cyber name, "Dr.Virus." He has been working with computers since 1988, and his passion has been Computer Security and Hacking since 1999. He also worked at RahmatLaw, Pakistan’s Law Library Portal, as a Product Manager, as it had been his father’s company. He has a video sharing blog on the topic of Hacking and Security, IAmHabib.net.
"In today’s world, online security has become a hot topic..."
Cyber-attacks have been growing lately, and online theft, identity theft, and other cyber-attacks can be catastrophic. The damage they cause can have long lasting effects.
Following are the most important security metrics used in organizations today:
- Return on Security Investment - Compare how much you spent vs. how much you got from the program. Use a line chart with an area map showing the difference between risk before and after.
- Relative Size of Security Team - This metric means how large your security team is relevant to what they are securing. Take the number of security team members, and divide by total number of persons in the security department.
- Percentage of Assets Under Security Management - This metric focuses on what percentage of applications are under security management. You can’t secure what you don’t know. Compare the list of total applications to the list of applications protected in your risk management program.
- Patch Latency - This metric tells you how far your organization is behind on updates for applications used. Use a patch management system to get the information. To display it, a bell curve could be nice, showing how many are completely updated and how many are years behind.
- Password Strength - This metric is very important, particularly if your organization still has a lot of weak passwords. It’s critical to see and to fix it.
Sebastian Igreti
TechMD is a top-tier IT services firm in Orange County, CA that helps organizations develop their IT strategy, leverage the cloud the right way, and manage their IT infrastructure securely. As President of TechMD for the last 15 years, Sebastian has had the privilege of working with hundreds of organizations to learn what technologies work best to move businesses forward. His passion is to share what he has learned with others, so he can play a small role in their success.
"Security metrics should be based on business objectives..."
As a result, some organizations require more compliance, while others can get by with less. Generally speaking, the following security metrics apply to most organizations:
- Patch policy compliance
- Mean time to patch
- Risk assessment scoring
- Mean time to remediation of identified risk
- User security awareness training engagement
- Virus infection activity (real time notification)
- Disaster recovery test results
- Number of security policies & standards that have been fully implemented and adopted
- Network probing attempts
Samuel Bethea
Samuel Bethea is President of The Rosewood Group and a small business security expert. Sam has been a serial entrepreneur and enjoys helping people achieve success in their business ventures. Sam has an Executive MBA and a Bachelor's degree in Electrical Engineering with over 30 years of corporate leadership experience.
"When it comes to security measures in a small business, the metrics generally fall into two categories..."
The metrics are associated with internal protection and external credibility.
Metrics for internal protection safeguard your employee data, your financial data, and key business data. Clearly understanding how your information is managed is key.
Cybersecurity ensures that all sources of digital penetration from an external source have been mitigated in your system. In a small business, it might be wise to partner with cloud resources that have cybersecurity built in to their security systems.
It is just as important to protect your customer data. With the increase of fraud and cyber attacks, encryption and security badges serve very important purposes.
Incorporating these two features into your overall system of security will
provide assurances to your customers that you have taken precautions to
protect their information and will convey a sense of digital sophistication
and understanding of the threats that loom on the internet today.
According to Security Week
- Number of vulnerabilities
- Number of incidents
- Average time a vulnerability remains unpatched
These metrics provide a sense of the threat and help to define the level of effort needed to protect your assets against a cyber attack. More advanced monitoring would evaluate those threats and quantify the cost of an attack, proving good insights into the cost of a breach in your security system and making it easier to justify the right-sized system to protect your business and your customers from an increasing threat.
Lenny Liebmann
Lenny Liebmann is a Partner at Morgan Armstrong. Lenny wrote his first line of code in 1973 and first used a mobile device to connect to the cloud in 1975.
"One critical security metric is..."
The percentage of phishing emails opened by your end-users. You can only capture this metric by sending your people your own fake phishing emails in a way that allows you to track open and click rates – and to pinpoint the worst offenders. This metric is critical. You can lock down as much as your environment as you want – but if social engineering techniques allow bad actors to do all the things phishing can allow them to do, you're super-vulnerable. Phishing scams around C-level executive credentials are especially problematic.
Rhand Leal
Rhand Leal is one of the lead Advisera ISO experts and Information Security Analysts in charge of ISO 27001 compliance and other security standards. He has 10 years of experience in information security, and for the last 6 years he has continuously maintained а certified Information Security Management System based on ISO 27001.
"The most important security metrics for organizations today fall into three main categories..."
- Metrics related to demonstrating compliance with industry regulations and other legal requirements an organization must comply with;
- metrics that demonstrate the value of the security investment, either in people, processes, or technologies; and
- metrics related to the coverage and performance of implemented security controls, used to demonstrate the capability and maturity or security.
Swapnil Deshmukh
Swapnil Deshmukh is a Sr. Director at Visa. He leads a team responsible for attesting security for emerging technologies. He is coauthored the Hacking Exposed ebook series and is a member of OWASP. In his prior work he has helped Fortune 500 companies build secure operations center.
"From the board’s vantage point, the most important security metric is..."
The overall maturity index of their current security portfolio. There are many industry standards such as BISMM or OpenSAMM that calibrate an organization's maturity model and ranks them based on how their peers are performing. Along with that, control effectiveness is another important piece of the puzzle. Tools such as Critical Security Controls (CSC) and independent assessments performed by Gartner and Forrester provide a detailed view into control effectiveness. Last but not the least are the metrics provided by internal and external auditing bodies on security engagement models such as architecture review, code reviews, and pentesting.
Laith Pahlawan
Laith Pahlawan, B.Sc. Dip. MCSE is CEO/Partner of Orange Crew in Anaheim CA. He has over 30 years of experience in IT Services. Orange Crew, founded in 2002, secures over 50 businesses in California.
”Small business owners often don't think that they are a target or will ever be hit..."
They should closely monitor three areas:
- Backup. Backup is still king, after all these years, know your RPO (Recovery Point Objective) and your RTO (Recovery Time Objective). Set alerts in case the backup fails, and you will sleep a whole lot better at night.
- Patch your Operating System. Being able to produce patching reports and knowing how long it’s taken to patch a bug is critical to protecting users from many Ransomware and cyber-attacks.
- Antivirus, especially Next-Gen Antivirus. Next Generation AV doesn’t try to catch up to the latest virus definition, instead it learns your PC's behavior and prevents foreign code from running.
Carlos Rodrigo P. Montagner
Rodrigo Montagner is an Italian and Brazilian IT Executive with 20 years of experience managing IT and Cybersecurity in multiple businesses.
"From my personal experience, the most important metrics we've been using internationally are..."
The metrics directly connected to a log of events such as attack attempts, threats, and penetration attempts. I have used, in different and multinational businesses, the following meta data and gear to support me:
- Internet Syslog Events - Measured on a regular app or device plugged into the Firewall or the network protection system (IPS or NPS). The best possible monitoring routine is to check daily and weekly for log reports, to activate alarms in case any unexpected activity is reported, and to establish a good Wi-Fi and Firewall system that allows you to control Wi-Fi connected equipment.
- Proper Setup of Tools such as Wi-Fi Hardware and Firmware, and the Proper Configuration of Edge and Core Switches along with the IPS or APS/ASA Equipment. The Edge and Core setup enhance protection, and the combination of good Wi-Fi hardware, the Firewall, and the IPS system allows you to collect security metrics on a weekly basis so that you can trace penetration attempts, content metrics, and several other basic metrics that can show us the health of our cyber protection environment.
Jeff Carpenter
Jeff Carpenter is Director of Identity and Access Management solutions at Crossmatch, a worldwide leader in authentication and biometric solutions. He is both a CISSP and CCSP.
"The most important security metric being used today involves user identities..."
Password-related compromises are endemic and are the leading cause of breaches worldwide. Responsible organizations are using their authentication platforms to proactively measure and report on what applications users are accessing and what they used to authenticate (static password, token, smartcard, push OTP, biometric, other).
What percentage of your users, and this includes the traditional and non-traditional employees such as partners, suppliers, customers - are going beyond static passwords? It's a measure of strength of authentication and everyone from IT admins on up to the board level are asking this question now.
