EPSS Shows Strong Performance in Predicting Exploits, Says Study from Cyentia and FIRST

Tenable sponsored research from Cyentia and FIRST, which finds that while vulnerability exploitation is highly variable, EPSS is getting stronger in its ability to predict exploitation.

The number of CVEs published annually keeps growing, which means it is increasingly crucial to predict which ones require the attention of vulnerability management teams. New research from Cyentia Institute and the Forum of Incident Response and Security Teams (FIRST) finds the Exploit Prediction Scoring System (EPSS) scoring system is a useful input to help teams make more informed decisions about vulnerability prioritization.

The extensive research set out to explore the timing, prevalence, and volume of exploitation activity, and also collect and analyze feedback on the performance of EPSS. The result of this effort is the inaugural report, A Visual Exploration of Exploitation in the Wild. The report provides data points and analysis that will benefit the large and growing community of enterprise users and security products that leverage EPSS. In this two-part blog series, we’ll explore some of the key findings and insights from the research. Part one will answer the following questions:

• What proportion of vulnerabilities have been exploited?
• What’s the typical pattern of exploitation activity?
• How widespread is exploitation among organizations?
• How does EPSS perform in its ability to predict exploitations? 

What proportion of vulnerabilities have been exploited?

Believe it or not, we’re nearing a quarter million published CVEs. And this number has grown at a rate of 16% over the last seven years. No one has the time or resources to address all of these vulnerabilities, which makes identifying and prioritizing the ones that matter most so important. One critical step in prioritization efforts is to track and predict how many vulnerabilities are exploited. 

In Figure 1, you can see the cumulation of 13,807 CVEs with exploitation activity over time on the left plot, which tells you that the number of known-exploited vulnerabilities is steadily approaching 15,000. On the right plot, you can see the count as a percentage of published CVEs over time, which tells you that about 6% of all published CVEs have been exploited, and that rate is holding steady. 

Figure 1: Vulnerabilities with Exploitation Activity 

What’s the typical pattern of exploitation activity?

Now, let’s take a look at the typical pattern of exploitation activity…turns out, there isn’t one! 

Figure 2 shows the exploitation activity of five different CVEs over 2023. Each of these has a unique exploitation activity:

• The top CVE experienced exploitation that was short-lived and very sparse
• The second CVE experienced fairly regular weekday activity 
• The third CVE experienced daily to weekly exploit attempts, with a spike in mid-December 
• The fourth CVE showed sustained daily exploitation that was particularly high in Q1-Q2 
• And the final CVE experienced an extremely high, consistent rate of exploitation activity 

So, what does this all mean? Well, exploitation comes at different levels of intensity and duration. It would be wise not to treat "exploited" as a binary variable but instead dig deeper into additional variables like intensity and duration for prioritization efforts.

Figure 2: Disparity in Observed Exploitation Activity 

How widespread is exploitation among organizations?

Speaking of not treating "exploited" as a binary variable, let’s look at the prevalence of exploitation observed across a large population of over 100,000 organizations distributed around the world. A surprising observation is that not many organizations see exploit attempts targeting a particular vulnerability. Exploits hitting more than 1 in 10 organizations are rare (it’s less than 5%!). When vulnerabilities are reported as exploited in the wild, they are generally thought of as exploited everywhere. However, this is not the case, and indicates that we should not treat all exploitation reports equally.

Figure 3: The Prevalence of Exploitation Activity 

How does EPSS perform in its ability to predict exploitations?

According to FIRST, EPSS is a “data-driven effort for estimating the likelihood (probability) that a software vulnerability will be exploited in the wild.” EPSS takes a daily estimate over the next 30 days of all known CVEs and provides a probability score ranging from 0 to 1 (or 0 to 100%), indicating the likelihood of exploitation. 

As noted in Figure 4, each version of EPSS has shown stronger performance in its ability to predict exploitation. Three metrics measure performance: 

1. Coverage: Measures the completeness of prioritizing the exploitation activity (% of all known exploited vulnerabilities that were correctly prioritized) 
2. Efficiency: Measures the accuracy of prioritizations (% of vulnerabilities prioritized for remediation that were actually exploited) 
3. Effort: Measures the overall workload created by the prioritization strategy (% of prioritized vulnerabilities out of all vulnerabilities) 

Based on Figure 4, you can see that remediating vulnerabilities with an EPSS score of 0.6+ achieves coverage of ~60% with 80% efficiency, whereas remediating vulnerabilities with an EPSS score of 0.1+ changes to 80% coverage and 50% efficiency. Each organization will vary in its risk tolerance, which impacts prioritization strategies. Understanding coverage, efficiency, and effort metrics can help organizations make more informed decisions on the specific strategies they use for their vulnerability management programs. 

Figure 4: The Performance of the Exploit Prediction Scoring System (EPSS)

This research is really cool. Now what?

For more insights, download the full report. Leverage EPSS Support in Nessus 10.8.0 with a free trial or purchase a license. Stay tuned for part two of this blog series!