Security Metrics - How Often Should We Scan?

I get this question from Nessus users and Tenable customers very often. They want to know if they are scanning too often, not often enough and they also want to know what other organizations are doing as well. In this blog entry, we will discuss the many different reasons why people perform scans and what factors can contribute to their scanning schedule.

Situational Awareness

When vulnerability data is used for pure "situational awareness", it needs to be as up to date as possible. Many organizations have an incident response, security operations or other type of group that monitors security, but has no direct operational roll over the IT or mission critical servers on the network. These organizations need up to date vulnerability information to do their job.

The vulnerability data needs to be very timely and relevant such that it can be readily available to help correlate IDS events, to be fed to a SIM or to be at the fingertips of an incident response team. It also needs to be able to provide executive management visibility into the top security issues facing them.

In order to accomplish this in the most effective manner, vulnerability data needs to be as real time as possible. Daily scans and passive monitoring can ensure that all data is within 24 hours. If a network is so large that it takes a few days or even a week to complete a scan, the data is less useful, but better than no data at all.

Mitigating Risk

Other types of security organizations I've dealt with have a more direct relationship with operational IT. Based on the results of ongoing vulnerability audits, the security organization is in a position to weight in on change control such as modified firewall rules, deployed patches and tuning application and system configurations in response to security threats.

Many of the banks and government agencies which employ this strategy do not give direct control of the network over to the security monitoring group. I've blogged before about the dangers of narrowly placing your network administration in the hands of your vulnerability scanner output. Instead, these organizations take input from the security group to recommend courses of action that take the business's interest at heart first. In a crisis, such as a virus outbreak, active insider or DOS attack, the security group can be placed in a position of dictating tactical changes to the network, but typically, this group helps shape generic policy.

The timeliness and accuracy of the vulnerability data used by the group requires many of the same qualities as a group performing pure situational awareness with one exception -- there is a lower tolerance for false positives and negatives. In a pure situational awareness scenario, the more data for an analysts the better. However, when making operational recommendations to IT, I've seen organizations that use credentialed scanning as well as passive network monitoring become very successful at what they do. When compared to pure active scanning, credentialed scanning and passive monitoring offer many alternatives in accuracy, speed and detection of client-side vulnerabilities. Tenable refers to this combination as Blended Security Assessments.

Compliance Monitoring

Several of our customers and consultants that use Nessus employ it as a tool to test other business processes that are in charge of patch or configuration management.

For example, a bank may have their own software and patch distribution system which even has a self-audit function built it. It might be the most reliable system ever constructed for a network. But an auditor will want some sort of second independent method to verify this system and they can use a vulnerability scanner like Nessus to confirm that this tool is working as needed.

When it comes to compliance monitoring, I've seen a lot of organizations confuse the capabilities of their tools with the requirements of the compliance regulation. As an example, I've blogged in the past about how an organization can proceed to audit systems for missing patches older than 30 days. It is very trendy to focus on zero day exploits and the latest vulnerabilities from Microsoft Tuesday, but for an organization worried about systems management, finding any older vulnerabilities could indicate a critical lapse in an IT policy or process.

When you want to audit a business process, the driving factor in how often you should scan is how often the business wants to audit this process. It may have many other business processes to mitigate the risk of an un-patched system, but if the apparatus to test another process is a vulnerability scanner like Nessus, this requirement should be driven by a business process and not a need to scan for the most recent vulnerabilities.

Deterrent to Change

Another reason to perform scanning is not only to look for change, but to use this audit as a reason for others to not make unauthorized changes.

With active scanning, this form of deterrent works best if there is a published scheduled well-known scanning policy which is also complimented with occasional unannounced scans. The higher the frequency your organization can perform scans will also allow you to find and deter changes more effectively. I used to be a penetration tester and system's auditor and there were always vast differences in the type of data found between the announced and un-announced scans.

With passive scanning, the deterrent is continuous if the evidence of detected deviations is shared with the offenders and IT. Since there is no impact or any evidence of the passive monitoring, an organization might not know they are being audited unless there is some sort of feedback.

Conclusions and  Additional Information

There are likely many other reasons to perform scans more or less often, however, for enterprise networks, most of the experience I've had with our customers falls into one of these four categories.

There will be several more blog entries on security metrics in the next few days and weeks. If this topic is of interest to the reader, the following previous blog entries listed below are very relevant. Also, I highly recommend the http://www.securitymetrics.org web site.

• Knowing When to Patch
• Testing the Effectiveness of Your Patch Management System
• Exceeding NIST and CIS Benchmarks
• Finding Vulnerabilities Older than 30 Days
• Tenable Wednesday
• Order from Chaos on Large Enterprise Networks