Passive SPAM Traffic Analysis

This blog entry concerns passive network monitoring with both the Passive Vulnerability Scanner (PVS), as well as the Log Correlation Engine. Tenable's research group has recently introduced PVS rules to identify mail clients and end nodes which are likely sending SPAM.

This blog entry will discuss this technique as well how the Security Center and Log Correlation Engine can be used to analyze traffic from hosts that are sending SPAM. None of the examples shown below provide a "smoking gun" of a detected SPAM server on purpose. The focus of this blog entry is to show how different types of analysis techniques can be used to understand network activity.

SPAM Analysis with PVS

PVS plugins #4122 through #4127, as well as plugins #4184 and #4185, detect potential email clients and end nodes that are engaged in SPAM activity. For example, the PVS will log a system as a potential SPAM source if an email is sent and a "551. Relaying Denied" message is returned. Other techniques are used as well. Below is an image of a detected SPAM server report:

These alerts can be sent in realtime via SYSLOG or consumed by the Security Center to see which of your internal organizations may be sending SPAM messages.

It's important to note that normal email activity can result in behavior that might be flagged as a potential SPAM source. For example, we've all sent email to an address which is incorrect, but we are not a SPAM server. However when SPAM emails are sent, very often they will attempt to email accounts that are no longer active. The idea with these PVS rules is to help classify which systems might be SPAM servers in order to more easily analyze them as we will do in the following sections.

Creating an Asset List of all SPAM systems

With the PVS rules detecting potential SPAM servers, we can use the Security Center to create a dynamic asset list of just these hosts. This will create a unique list of just the IP addresses that the PVS thinks might be a SPAM server.

To do this with the Security Center, go to the "Assets" button, and then choose "Dynamic Assets" and then "Add new". Choose a name of "SPAM Server" and create an empty list. From the list of existing rules, then select the rule for "SPAM Server" by clicking on the 'R' icon. From there. click on the large 'A' and then add a plugin ID match for 4122 which is the first PVS SPAM rule we will match on. Click on the 'A' and repeat for 4123 through 4127 and then also add in 4184 and 4185. When finished, you should see a rule which looks similar to this:

Once this rule is set, the Security Center will automatically process this and create an updated asset list. If one is not created, you can force this to occur by clicking on the "Update Asset List" button. When finished, clicking on the "Asset List Control" button can give you a list of all IP addresses that are suspected to be SPAM sources. As new data from the PVS arrives at the Security Center, it will also dynamically update this asset list.

Analyzing Traffic From SPAM Sources

Now that this asset list is created, it can be used to analyze data we've received from the network. Customers who have deployed the Log Correlation Engine can use the "SPAM Server" asset list to effectively find SPAM sources on their local network. This can be used to isolate port 25 network traffic, connections with known "blacklist" sources and focus on IDS events directed against these systems. The idea is to use the list of known SPAM servers as a potential filter on the immense amount of data typically collected by the Log Correlation Engine.

Example 1 - Connecting to a Blacklist

We've blogged before about how any type of event or network connection can be considered by the Log Correlation Engine for interaction with a published "black list" of IP addresses. If we have all of the network events for a large organization with suspected SPAM servers in it, a useful filter would be to see which blacklist events may have occurred to or from our list of SPAM servers.

Below are two examples from a test site which uses network IDSes, and the Tenable Network Monitor along with the Security Center, Nessus, PVS and Log Correlation Engine.

Blacklist Events	All Events

In the example on the left, a list of all matching "blacklist" event types has been produced for the "SPAM Server" asset list. It is interesting to note that there have been correlations with not only the SANS ISC list of scanning hosts, but also the Bleeding Threats list of Botnet Sources as well as the Spamhaus list of known spammers.

I would not consider the SANS correlations that significant, as this report came from a small university environment which is often scanned by hosts tracked by the SANS ISC. However, having an asset list of known potential SPAM servers connect to a list of botnets or SPAM sources is very interesting.

Both detection techniques (the PVS analysis to find potential SPAM servers as well as the blacklist correlation) are independent of each other. Knowing that some of these hosts may be under control of a botnet and being put to use for sending SPAM can help plan out a course of action and response. In this example, it is also interesting to note that two different forms of network IDS monitoring were in place and neither detected any odd infection, compromise or SPAM activity. 

The second example on the right was a list of all events recorded by the Log Correlation Engine for the "SPAM Server" asset list over the past 24 hours at the same location. All network sessions, blacklist events and PVS events are listed. There were even a few Never Before Seen and Crowd Surge events.

Example 2 - Activity Analysis

Considering the last five days of pure "network" activity for the "SPAM Server" asset list, the Security Center can produce activity graphs and port usage statistics as shown below:

Port Summary	Network Activity

In the port summary, as expected, the most common port used was 25. What is interesting is that the second most used port is port 80. It is very likely that some of these systems are desktops or servers which have been compromised and are also being used for regular web browsing.

In the second image, for the same five days of activity, all network activity on any port is graphed out. The three initial peaks correspond with the previous Wednesday, Thursday and Friday and are followed by two smaller peaks for Saturday and Sunday.

Being able to perform this sort of analysis can help identify trends and "top talkers". Keep in mind that with a dynamic asset list, this analysis is always looking at the activity for the most recent lists of IP addresses of interest reported by the PVS.

From a SPAM detection point of view, it would have been interesting to see if there was a spike of port 25 traffic or one or more hosts in particular that sent a disproportionate amount of spam. At this demonstration site, such traffic was not observed though.

Example 3 - Connection Activity

With the "SPAM Server" list in place and the PVS and LCE continuing to monitor the network, it would be interesting to analyze the connections to and from this suspected list of SPAM servers.  There are several types of connection tracking available to us:

• The PVS logs any internal IP address that has completed one or more TCP connections.
• The PVS logs any "browsed" UDP or TCP ports.
• The LCE can graph out the start time, stop time and total bandwidth of any TCP connection.

For the site in this demonstration, doing a summary on PVS plugin #00002 (Client Side Port Usage) shows us the following graph:

This is interesting as ports such as 3724 (used by World of Warcraft) , 5000 (well known for many Trojans), 6001 (X Windows) and 7000 (also well known for many Trojans) are more closely associated with client side applications or compromised nodes.

Knowing that these systems are email servers or systems that are sending email, it is also interesting to consider which ports are open on the "SPAM Server" list of IPs and accepting external connections from the Internet. Such as query produces the following graph:

It is understandable that many of these systems receive email on port 25 and also be a web server. The  other "high" ports are most likely web management interfaces. Using the 3D Tool, we can analyze which ports are open on which IP addresses with the "SPAM Server" asset list. Doing a query for the "SPAM Server" asset list as well as PVS rule #14 (Accepts External Connections) renders a port to IP address display such as this:

The bottom column (left to right) is ports and the axis moving away from us graphs matching IP addresses. The vertical line of columns is on port 25. This tells us that each matching IP address had accepted a port 25 connection from the outside of the network at least once. The other columns show the distribution of open ports across the other IP address. The address in the first row has accepted connections on port 22, 25 and several other high ports.

Performing this analysis in 3D with a few dozen IP addresses may seem like overkill, but the same techniques can be applied when analyzing 100s or even 1000s of nodes.

Any of these ports could be legitimate traffic, or it could also be a command and control channel for a botnet.  To dig into this further, we could analyze these browsed ports and open ports with traffic collected by from the Log Correlation Engine. We could also look at the clients and banners reported by the PVS on these ports in question. And lastly, we could try to do some active scanning on these hosts and see if that turns up any particular vulnerabilties or other open ports.   

For More Information

If this blog entry has interested you, Tenable has previously blogged about how basic passive network monitoring can be used to identify SPAM servers.