Vulnerability Management Metrics

Everyone is always talking about metrics. Just like the 1970’s TV show that quipped, “Marcia, Marcia, Marcia,” analysts today are stressing Metrics, Metrics, Metrics. But what security metrics are important and how can we identify the best metrics from the not so useful metrics? When a CISO meets with other C-level executives and board members, the CISO must have a clear expectation of the useful metrics for that organization. Additionally the CISO must be able to communicate the metrics in a story form or narrative, helping the executives to fully understand how the metrics impact their business and overall risk to the organization.

Core vulnerability metrics

Each organization has different security metrics that are important to their operations and business. However, there are several vulnerability metrics that are common across industries and should be considered when discussing security issues with your C-level executives.

• Time to Detect - This metric is the delta from when a vulnerability is created until the time the vulnerability is detected. For example, when a user installs a vulnerable application, until the time the vulnerable application is discovered.
• Time to Containment or Mitigation - This metric indicates how quickly attacks are contained or how long mitigation of a vulnerability takes.
• Baseline Metrics - Establishing a baseline of normal behavior is important before developing a full metrics program. This metric provides a clear understanding of what normal is, and how to understand deviations from normal.
• Patch Management Efficiency - This metric contributes to the understanding of patch cycles and remediation efforts. This metric is different from the Time to Containment or Mitigation because these efforts are based on regular vendor patch release schedules, such as Microsoft’s Patch Tuesday.
• System Hardening Metric - This metric provides insight into the proper configuration of operating systems, applications, and network infrastructure devices.

How SecurityCenter can help

Tenable SecurityCenter® provides continuous network monitoring to identify vulnerabilities, reduce risk, and ensure compliance. When using active and agent based scanning, SecurityCenter can analyze each system and discover the operating system, installed applications, miscellaneous configuration settings, and the vulnerabilities associated with each risk factor. SecurityCenter also provides an easy to understand and maintain method of communicating the metrics associated with these identified risks and vulnerabilities. Assurance Report Cards, dashboards, and reports all help to communicate the metrics to different levels of information consumers. SecurityCenter can create simple to understand charts and graphs for executive level consumption, and at the same time provide the detail needed for the IT operations team.

When preparing presentations for executive boards, CISOs need a wide range of data sources. SecurityCenter has over 350 dashboards and 400 reports to assist with the collection of metrics and to answer a CEO’s question, “How secure are we?” For this blog post we have selected 5 SecurityCenter dashboards that address these metrics. These vulnerability metrics are often identified by security experts as foundational to a solid security program. (For more information about these metrics, see Using Security Metrics to Drive Action: Security Metrics That Tell a Story to the Board, an insightful ebook published by Mighty Guides, Inc. with Tenable.)

Time to Detect

Time to Detect is an interesting and vital metric to answer the question, “Are we secure?” This metric can lead to a false sense of security if the detection time is not highly scrutinized and verified using several methods. For example, an application can have a zero day vulnerability - a vulnerability that has not been identified by the vendor - for several years in some cases. Does not knowing about the vulnerability make your company more secure? The answer is no; not knowing about a vulnerability will not make you more secure. You cannot mitigate a risk unless you are aware of it, but you can harden systems, networks, and firewalls to lessen the likelihood of compromise. In the Executive Vulnerability Metrics dashboard, there are 4 matrices that correlate vulnerabilities from the date the vulnerability is published to the time the vulnerability is identified by SecurityCenter - the time a patch is available, and the time the risk is mitigated. These matrices provide you with numbers that executives need to understand and properly assess risk to the organization.

Time to Containment or Mitigation

Having the ability to report on how efficient your mitigation program is can be critical to communicating the effectiveness of a risk mitigation strategy. SecurityCenter contains a database solely for this purpose. During the scanning process, SecurityCenter compares the scan results of a host to data stored in the cumulative database. Any vulnerability that is not present in the latest scan is considered mitigated. During this correlation process, there are two dates that are recorded:

• Days to Mitigate = Tracks the number of days since a vulnerability was moved to the mitigated database
• Days Since Mitigation = Tracks the number of days since a vulnerability was mitigated

Using these two filters, you can easily gain an understanding of how your patch management and risk mitigation programs are functioning. Using the Days to Mitigate, you can determine the Patch Rate within your organization, which speaks directly to the Time to Containment. The Historic Patch Mitigation Status dashboard presents counts of mitigation and containment metrics. The data in this dashboard helps answer the question “How long was this vulnerability a known threat?” If your patch cycle is every 30 days, then your patch rate should always be less than 30 days. Use these metrics to show how efficient the patch program is working, or to show a need for more staff or additional resources.

Baseline Metrics

Another important question you may get asked is, “What is Normal?” When analyzing network traffic patterns, disk usage, or database storage, the base metrics can be very straightforward to calculate. However, when calculating the the base metric for Vulnerability Analysis, the baseline is much more suggestive and determines how much risk you would like to accept.

Some organizations set the policy for the baseline as not having any vulnerabilities over a certain CVSS score, while others use the difference between credentialed and non-credentialed scans. Whichever metric your organization chooses, make sure to have the metric clearly defined and achievable. Regardless of the metric, the Vulnerabilities by Plugin Family dashboard is a good place to start your analysis. This dashboard provides a breakdown per plugin family of all the vulnerabilities discovered. Each of the components provide a breakdown using Mitigated, Unmitigated, CVSS Score, Exploitable, and Patch Availability. Use this information to present a current risk assessment based on different categories and operating systems.

Patch Management Efficiency

Another common metric is the Patch Management Efficiency metric. By tracking Patch Management Efficiency, you can justify manpower, show mitigation actions, and track many common business metrics. Companies like Microsoft that report vulnerabilities on a regularly scheduled basis provide CISOs with a set of changes for planning purposes. However, other companies that release patches at random intervals make the reporting of metrics more difficult. Both methods have positive and negative effects, but you must be able to report the metric accurately and convey that the host risk is reduced. The Mitigated Patch Rates dashboard provides a trend analysis of vulnerabilities discovered over 3 months, and a breakdown of vulnerabilities based on operating systems and CVSS score.

System Hardening Metric

System hardening is a challenging effort to say the least. When hardened configurations are applied, certain applications will not work correctly; and yet the threat of not hardening creates high risk from zero day vulnerabilities. Thankfully, there are organizations such as the Center for Internet Security (CIS), National Institute of Standards and Technology (NIST), and others to assist with hardening standards. SecurityCenter has the ability to perform configuration audits on several systems and map the configuration settings to match standards. The Compliance Summary dashboard provides a simple view of how an organization is meeting the support standards.

Wrapping Up

Reporting metrics that are understandable to the business is a difficult and time-consuming task. SecurityCenter is a great place to start when communicating risk to executives. Using these 5 dashboards along with other more business-specific dashboards, your organization can be more secure and your executives will be better informed.