Event Analysis Training - Advanced Blacklist Analysis

DNS blacklists are publicly accessible lists of IP addresses that have been identified as associated with undesirable Internet behavior, mostly involving the sending of spam e-mails.

Using lists of IP addresses on a blacklist can be an effective way to look for potential security issues in your logs. We've previously blogged about using blacklists with the Log Correlation Engine. In this post, we will examine more of the basic assumptions and principals for analyzing traffic associated with blacklisted IPs.

A Quick Blacklist Review

There are many different free and commercially available lists of "blacklisted" IP addresses. These lists attempt to identify IP addresses that have a reputation for scanning, sending spam email, hosting malware and many other types of undesirable behavior.

The lists differ in how often they are updated,  how many IP addresses are published, the source of the IP addresses and the nature of the activity from the IP. For example, Arbor's Atlas list comes directly from their deployed sensors whereas the SANS Internet Storm Center derives its lists from voluntary user submissions.

Here Today -- Blacklisted Tomorrow

Using blacklists to filter your logs for evidence of connectivity with potentially hostile servers is an effective way to identify potential security issues on your network. However, if a system interacts with an IP address on a blacklist, don't forget to analyze what the IP did before it was blacklisted.

Consider this screen shot from the Log Correlation Engine deployed at a large university:

This is a 5-day graph of any log matching an IP address that the security administrator identified  after seeing this (sanitized) alert:

Outbound_Blacklist_Communication src – X.X.32.99 dst – X.X.182.175 , following is known about 217.175.182.175 , message - ISC-BlockList , dns_lookup – xxxx.xxxxl.fr

To determine the activities associated with this IP address for all systems (not just the “blacklist” events), the security administrator  summarized all normalized event types for the past 5 days. If you view the graph from left to right you can see that there was a large spike in firewall and connection events. This particular network has Stonegate firewalls and logs not only firewall “deny” events, but also permitted connections. There is also a variety of statistical and never-before-seen events from this IP address.

It is interesting to note that about 3 days in the past (around the middle of the graph of events) is when our first normalized blacklist event occurs. Sometime between 5 days ago and 3 days ago, the IP in question was involved in activities that caused it to be placed on a blacklist. Only after this occurred did we get blacklist correlation events -  prior to that the connections were logged and correlated purely on their own merit.

The point of this type of analysis is to always consider what the blacklisted IP addresses may have been doing prior to being placed on a blacklist.

Analyzing Blacklist Connection Directionality

The most basic form of blacklist correlation is to get a list of IP addresses and then use them as a filter to see if any logs match with the addresses on the list.

If you are analyzing logs from public facing servers and devices, the blacklist correlation will likely tell you that you are being scanned by "known" IP addresses that scan the Internet. Although interesting, this type of activity is not alarming and should be expected.

What is more interesting is to consider the direction of the traffic  in the logs that include blacklisted connections. It may be expected that we would be scanned from a blacklisted IP, but what we would be really concerned about is if one of our servers connected to the blacklisted IP.

Why might one of our systems connect to a known blacklisted IP address? Possibly, it has been compromised and is connecting back for a payload or is part of a command and control botnet.

On the other hand, "connections" can mean many things and some care should be made when looking at these types of correlations. Consider some of these scenarios where the "outbound" connection is a legitimate connection, but that would generate a false alarm:

• The attacking/scanning blacklisted IP address is actually a compromised DNS server. When this IP makes connections to your mail, web and other types of services that perform DNS lookups, they may in fact send DNS queries back to the blacklisted IP address because it is the DNS server. This might look like the host has been attacked and then compromised.
• If the blacklisted IP address has been placed on a list because it is a communication channel for a botnet, other legitimate users of this system will look like they are communicating with a hostile IP address. This often happens when a public IP address of an IRC system is blacklisted. When an innocent IRC user connects to this address, a correlation occurs, although there has been no compromise.
• In some SIMs I've encountered, the normalization of NIDS events and even system logs such as login failures swap the source and destination IP address. This can cause a correlation engine to conclude that an outbound connection has occurred. For example, consider a Snort rule that logged a VNC login failure event. This event has a source IP of the target host and a destination IP of the attacker's system. If a correlation engine were not smart enough to consider the direction of network traffic of this event, it could incorrectly conclude that a system had  made a new connection when in fact it had not.

Considering the network traffic direction of connections which terminate or originate with a blacklisted IP address is exactly what the Log Correlation Engine blacklist.tasl script performs. It considers any network connection event which was logged by a system, application, firewall, netflow or other type of network monitor and then alerts if the IP addresses involved correspond to a blacklisted IP address. Other correlation scripts attempt to see if “blacklist” connections were preceded by an intrusion detection event, or if a system repeatedly attempts to make connections to a blacklisted IP address.

For More Information

The topic of blacklist analysis as well as generic event analysis has been discussed on this blog several times:

• Working With Blacklists
• Passive SPAM Traffic Analysis
• Finding Low Frequency Events
• Working with Emerging Threats events
• Looking  for “Never Before Seen” Events
• ShmooCon - Network Monitoring Notes