ShmooCon - Network Monitoring Notes

Another ShmooCon has come and gone. Tenable had the opportunity to run our products on the ShmooCon network. We deployed two blades which ran Nessus, the Passive Vulnerability Scanner, the Security Center, the Log Correlation Engine and a few agents for monitoring network traffic. We also placed Snort on one of the blades and ran a set of Sourcefire VRT and Bleeding Threats rules. This blog entry discusses how each component worked and what types of activity was observed.

Deployment

The network was well organized and had well defined VLANs and IP address segmenting for attendee secure and unsecured wireless access, speaker podiums, registration, the hacker arcade, servers providing DNS, DHCP, a 'pf' firewall and so on.

This made it very easy to configure the Security Center with a set of assets that matched the ShmooCon network. Below is a screen shot of an asset summary of all detected vulnerabilities:

At the time we had taken this screen shot, we had not performed any active scans with Nessus, but were monitoring all traffic with the Passive Vulnerabiltiy Scanner. The same blade was also running Snort and the Tenable Network Monitor which logged the start and stop of every network session.

Log and Network Monitoring

This network data was sent to the Log Correlation Engine, along with logs from the DNS server, DHCP servers, the 'pf' firewall and a few others. This allowed us to show normalized and logs correlated logs over any time range such as the following screen shot:

In this screen shot, we're looking at a time graph of all normalized event types for the past 24 hours from approximately 1:00 PM on Friday to 1:00 PM on Saturday of the conference. You can see a few things in this screen shot:

• The "detected-change" events were mostly logs from the Passive Vulnerabiltiy Scanner detecting new hosts, new open ports and so on.
• The surge in "error" events were from a switch that had a few flapping ports.
• There was a spike in "firewall" logs and then nothing. A firewall rule change stopped outbound syslog messages at one point.
• P2P activity was detected by some of the Snort rules and this activity occurred throughout the conference.
• The first real large scan of the network was done by Qualys and you can see a spike occurring in multiple other event streams just after the spike in "scanning" activity.

Something that we monitored closely was what type of correlated events occurred below:

In these screen shots, we're looking for evidence of compromised hosts, such as a host that was attacked and then was used to attack another host. On a normal network, your servers don't normally start attacking each other. However, at ShmooCon, this activity may have been occurring often.

One of the more simpler correlation rules in the Log Correlation Engine is the brute force password guessing script. This rule is simply looking for a number of login failures in a certain period of time. It does differentiate between a single host failing a login across multiple targets as well as one host failing multiple logins against one host.  In the above screen shot on the right, the Qualys scan I mentioned above had attempted several login attempts to a Cisco device.

Active Scanning, Passive Scanning and Found Vulnerabilities

The typical ShmooCon attendee uses Windows and does not patch their version of Firefox. We were able to determine this using the Passive Vulnerabiltiy Scanner. Below is an example screen shot of the detected critical vulnerabilities around 2:00 PM on Saturday:

There are many client side issues found by the Passive Vulnerability Scanner. Not only were there several issues with web and chat clients, you can see that many users run older versions of 'curl', 'ssh' and 'wget'. Security issues in these types of clients are often forgotten to be audited by system administrators.

Another interesting, and non-surprising item, was the detection of people running certain types of scanners and security testing tools:

In a future blog post, we'll investigate the network logs from these hosts and analyze if multiple were using these servers, and what other types of activity was detected.

During the set-up of the network, having the passive scanner online was very valuable. Active scanning was initially prohibited because all traffic was going through a very underpowered firewall. At one point, even the basic network profiling of the Qualys scanner was overloading the firewall and had to be halted.

When we did start active scanning though, as expected, scanning many laptops with firewalls not offering any type of responses proved to be a difficult target. You can configure Nessus scans many different ways to try various techniques to determine if a host is indeed there.

However, with the Security Center and the Passive Vulnerability Scanner, we simply created an asset list of known live wireless devices and had Nessus focus on those. This made our scans, much more effective than trying to scan a full class B for any hosts that might be alive.

Future Posts

There was a lot of very interesting data gathered which we will be using as a topic of future blog posts here to show different types of network and log events. I would also like to thank the ShmooCon staff for letting us participate in the monitoring of their network.