Finding Events that have "Never Been Seen" Before

A useful event to know about on any network is when something new happens on a given server for the first time. This is a very simple concept and extremely useful.

Regardless if your event logs are from UNIX systems, router access control violations, wireless access DHCP logs, intrusion detection systems or so on, after a certain period of time, the same events tend to repeat themselves. This is because most of our networks run controlled and automated processes.

With this in mind, finding out when something "new" occurs could indicate a security or administration problem.

The nbs.tasl Script

Tenable's research team has written a TASL script which implements this concept for the Log Correlation Engine. The script, nbs.tasl, stands for "never before seen". By default, the script subscribes to all possible event IDs except for network connection events from the Tenable Network Monitor (TNM) and the Tenable Netflow Monitor (TFM).

The script checks to make sure each event has either a source IP or destination IP in the "Customer Ranges.asset" file which the LCE obtains automatically from the Security Center. If an organization had a different asset list they would like watched, one could be substituted easily by modifying the script.

Detecting Intrusions

Tagging "new" events originating on, from or too a specific IP is very useful, especially for intrusion detection events.

For example, the nbs.tasl can differentiate between when a Snort event is generated for the first time from a server as compared to just alert when a system is attacked for the first time.

Consider a network that is being attacked very often. The Log Correlation Engine will likely record normalized IDS events inbound to monitored hosts. Very seldom (if at all) will an IDS event be sent from a target because we hope that our systems aren't attacking anyone. The first time these events occur though, the nbs.tasl script will alert on this "new" type of event.

Example Logs

Below is a screen shot of a set of logs generated by the nbs.tasl script:

Issues and Complimentary Technology

The first time you run the nbs.tasl script, all events will be new. By default, it will wait one day to learn which events are "normal". After that, any event that hasn't  occurred in the past day will be recognized as something "new" for the first time. This will create a spike of events that get generated.

Tenable considered placing a longer "learning mode" into the script, but being able to graph how often a "new" event occurs initially, where they come from and how long it takes for these events to "quiet down", can actually generate useful information.

For dealing with intrusions, keep in mind that the nbs.tasl also won't alert on the same event twice, even if it is a very critical alert. This is on purpose. If a server is compromised one day for the first time, and then compromised the second day with the same exact technique, the nbs.tasl will stay silent.

We've also designed the nbs.tasl script to not consider network events from the TNM and TFM. Those logs are verbose and copious and don't add a lot of value. Similar logs for monitoring and detecting network change (as in detecting new hosts, new ports, .etc) are already generated by the Passive Vulnerability Scanner.

And lastly, the LCE's stats daemon is designed to look for changes in the frequency of events and connections. If the data from the nbs.tasl script seems interesting to detect events that have not occurred before, then alerts from the statistics daemon identify changes that also have not occurred in the past.

Setting it UP

Please download all of these files to your /usr/thunder/daemons/plugins directory using wget or some other web client:

• http://cgi.tenablesecurity.com/tasl/nbs.tasl
• http://www.tenablesecurity.com/PRM_Mappings.prm
• http://www.tenablesecurity.com/prm_map.prm
• http://www.tenablesecurity.com/lce_tasl.prm

Keep in ming that if you have an existing file (such as the PRM_Mappings.prm library) wget by default won't overwrite the existing file but will save it with a numerical extension such as PRM_Mappings.prm.1.

By default, the nbs.tasl considers events to or from the Security Center "Customer Ranges" asset group. If a different asset group is available or desirable, it should be entered into the script.

By default, the script also has a variable named LEARNING_PERIOD. This variable is set to the number of seconds in one day. When the script first runs, it will consider all events "new" but won't alert on them until the period of time specified in LEARNING_PERIOD has occurred. If you want to start alerting right away, set this variable to zero.

Once these files are installed, restart your thunderd daemon.

For More Information

If event analysis seems interesting to you, all Tenable TASL scripts are available online. They are easily extended and work with the Log Correlation Engine. Tenable also has two free webinars available and several white papers which consider log analysis and event correlation located here:

• (webinar) Network Based Anomaly Detection
• (webinar) Vulnerability and IDS Event Correlation
• (white papers) Security Event Management

Please contact Tenable's sales or support groups for more information on these scripts.