Log Correlation Engine 3.0 Released

Tenable is proud to announce the release of the Log Correlation Engine version 3.0. This blog entry highlights some of the LCE 3.0 enhancements and new features, plus some of the new functionality which will be made available with the upcoming release of Security Center 3.4.3.

If you are not familiar with the LCE, this product is an upgrade to the Tenable Security Center that can process logs from 100s of different applications, devices, operating systems and security monitoring technologies. Every log is normalized, correlated for a wide variety of security and compliance behaviors and analyzed for anomalies. The LCE is very easy to install and has very high performance in both processing logs in real-time and also analyzing millions of events in just a few seconds. Typically, our customers who add on an LCE to their Security Center not only experience higher performance over their existing SIM solution, they also drastically reduce the number of appliances or servers required for operation. 

Customers who use the LCE along with the Security Center, Nessus and the Passive Vulnerability scanner, have an immense amount of configuration and activity information at their fingertips which is a key part of Tenable's Unified Security Monitoring strategy. Having this information in one spot allows an organization to react faster to incidents and trends that impact their security or compliance status. 

The following items detail the major changes existing LCE customers will be able to take advantage of with this new release.

User Based Activity Tracking

User tracking by IP address is now part of the LCE daemon and not a separate TASL script. Previously, we had blogged about this correlation feature for LCE 2.0. However, in LCE 3.0, this functionality is built into the actual daemons.

As part of the LCE's normalization of logs, any log that can identify where a user is, such as a login to an authentication system or authenticated emails, can be used to tie that user to a specific IP address. When LCE encounters a log that has no username field, it will assign the username of the user most recently associated with the source IP of the incoming log, or associated with the destination IP of the log if a destination IP is provided but a source IP is not.

For example, a remote user might connect to the corporate email server when sending authenticated SMTP email. If configured to treat SMTP email logs as a source to track user identities from, the LCE would automatically learn the user IDs and their IP addresses. Once this occurs, any other log, such as the user visiting the corporate web server, or performing some sort of activity picked up by the corporate NIDS, would be associated with that user ID. 

When a user changes IP addresses a “user-ip-change” event is written to the database. For example this following example log occurred after a user authenticated to a Blue Socket network device:

Network user IP address change: user danny has changed from 192.168.20.10 to 192.168.10.101 with event BlueSocket-User_Login (5.5.5.5:0 -> 127.0.0.1:0)

There may be perfectly valid reasons for a user to access the network from a different IP address, but this can also indicate that a user’s credentials have been compromised. In DHCP environments, or in places where a user might move their system from a physical LAN to a wireless LAN, this type of tracking allows an analyst to know exactly who had which IP address and at which time while at the same time, associating specific users to normalized logs.

In Security Center 3.4.3, user tracking will further be exposed for analysis. Analysts will be able to sort any type of log activity (netflow, firewall, logins, IDS events, .etc) based on automatically detected users as shown below:

USB Device Activity Tracking

LCE 3.0 Windows agents can now make use of Windows Management Instrumentation (WMI) functionality to monitor local and remote systems for USB device, CD-ROM disc and DVD disc activity. WMI has several functions, dependent on the Windows version, that allow for the monitoring of media insertion and removal from a system. These functions allow the LCE to detect the insertion or removal of USB devices that can be mounted as a volume. For each occurrence of such activity, a log entry will be generated similar to the following:

This feature is particularly useful for organizations that are required to demonstrate PCI compliance (protection of cardholder data). Such organizations can use this feature to generate an alert when USB devices are inserted or removed.

This new functionality becomes part of Tenable's ability to "detect change" across user accounts, software applications, servers and the network.

Windows systems can be monitored with a local LCE agent. The same agent can also be configured to monitor multiple Windows servers through the use of credentials.

LCE Daemon Improvements

LCE now contains several additional configuration options to allow for better management of clients and data handling:

• Old silos can be automatically saved rather than overwritten. In SC 3.4.3, these silos can also be specified from the GUI for analysis of historical saved data.
• Plugins and TASL scripts can be set to automatically update on a regular basis.
• LCE can now manage and more easily configure up to 8,192 LCE clients.
• The LCE daemon can be configured to listen on multiple addresses.
• The stats daemon is now configured from the same configuration file as the LCE.
• Normalization rules can now accept DNS names in logs in addition to IPv4 addresses. The LCE will intelligently perform high-speed DNS lookups and cache their results for a configurable amount of time.

There are also significant file system changes from LCE 2.x to LCE 3.0, most notably in the product name and installation directories. Starting with LCE 3.0, the application files are stored under the /opt/lce directory instead of the /usr/thunder directory. This brings the application in line with standard RedHat application distributions.

LCE Daemon Performance Improvements

The LCE 3.0 daemon also offers enhanced performance compared to LCE 2.0.

• Separate processes are now used for log normalization and correlation. This leverages multiple core and multiple CPU systems more efficiently.
• The correlation engine is 10x faster than the one shipped with LCE 2.0.
• All TASL scripts automatically log performance statistics for analysis.
• Indexing and compression of the LCE silos is 50% faster than with LCE 2.0.
• More compression is used on disk. 

In addition, each LCE client will report the CPU, memory and disk usage of the server it is running on along with their heartbeat message. In Security Center 3.4.3, the status of all LCE clients can be displayed by an administrator as well.

LCE Client Enhancements

Package Distributions

The LCE Clients have also been enhanced. One of the most noticeable improvements is the inclusion of client start-up files (known as RC files) that ensure that the client is started on system reboot and provide a cleaner mechanism for starting the LCE Clients. All clients are now installed as managed applications. This makes remote installation, common configuration and upgrades easier.  

Heartbeats

Each LCE client that communicates with the LCE Server periodically sends a heartbeat message, indicating that the connection is active. This option enables the Security Center to display information about the connection activity, hostname and IP address of the LCE client and the revision number of the application and includes performance statistics on the LCE host.

Passive SYSLOG Monitoring

The Tenable Network Monitor (an LCE agent that sniffs network traffic) can be configured to sniff SYSLOG messages and treat them as if they were being sent directly to the LCE. This is a very easy way to monitor SYSLOG messages being sent to a corporate Splunk system or another type of log aggregation point.

WMI Windows Event Log Monitoring

The LCE 3.0 Windows agent can also monitor the event log of multiple remote Windows systems via WMI. 

Upcoming Security Center 3.4.3 Features for LCE 3.0

Version 3.4.3 of the Security Center will be available shortly and contains several enhancements that LCE users should be aware of. These include:

• The ability to query multiple LCEs at the same time and aggregate their results. This increases your overall storage of online events and also dramatically decreases query times and report generation. 
• Raw logs can be searched. For any displayed SYSLOG message, strings can be used to search for more exact matching. This is extremely useful for finding logs with specific user names, DNS lookups to known hostile malware locations and much more.
• The SC 3.4.3 LCE user interface has been enhanced to show more event information, to have time lines on all activity graphs and to have a more intelligent navigation and query system.
• There are also new GUI elements that take advantage of sorting any set of events by the associated user name and to perform queries against older archived data silos of normalized and indexed events.

For More Information

LCE 3.0 installation and upgrade instructions are available on the Tenable Customer Support Portal. After upgrading to version 3.0, it is recommended that users perform a plugin update and then manually audit their TASLs to see if they want to remove or replace any of them with the new ones which are now available.

Tenable has updated all documents detailing the LCE’s deployment, configuration, user operation and overall testing. These documents are listed here and are available on the Tenable Customer Support Portal:

• Log Correlation Engine Administration and User Guide – additional information for installing, configuring and operating the LCE
• Log Correlation Engine Client Guide – how to configure, operate and manage the various Unix, Windows, netflow, OPSEC and other clients
• Log Correlation Engine Log Normalization Guide – explanation of the LCE’s log parsing syntax with extensive examples of log parsing and manipulating the LCE’s .prm libraries
• TASL Reference Guide – explanation of the Tenable Application Scripting Language with extensive examples of a variety of correlation rules
• Log Correlation Engine Statistics Daemon Guide – configuration, operation and theory of the LCE’s statistic daemon used to discover behavioral anomalies

Please contact Tenable Support for any questions regarding the upgrade to LCE 3.0. Potential customers interested in evaluating or upgrading to the LCE should contact our sales staff.