Automated audit policy creation for UNIX Nessus compliance checks

Many UNIX applications and system settings are contained in proprietary text configuration files. Auditing these for unauthorized changes or configurations can be very cumbersome and time consuming. Nessus 3 Direct Feed and Security Center users have the ability to audit UNIX servers against a configuration policy without an agent. Being able to automatically prove that all remote systems and applications have a certain configuration setting or that they have not changed from a known good configuration has tremendous value for compliance monitoring and system management.

Today, Tenable released a UNIX utility named "Configuration to Audit" (c2a) which considers a text based configuration file and produces a Nessus 3 .audit file. The c2a tool can be used to quickly create .audit files suitable for Nessus 3 Direct Feed or Security Center users. These .audit files can be used to test for specific configuration file settings as well as MD5 checksums.

This blog entry discusses c2a tool usage and provides example use cases for Nessus and Security Center users with a Snort configuration file.

Creating an MD5 .audit File

Nessus can remotely audit many different aspects of a UNIX server. For our first example, we will audit the MD5 checksum values of three files on a demo system which runs the Security Center, Snort and the Passive Vulnerability Scanner. The configuration for each of these files are located here:

• /etc/snort/snort.conf
• /opt/sc3/daemons/daemons.cfg
• /opt/pvs/etc/pvs.conf

These file names are placed in a text file named files.txt and the c2a tool is run with the following results:

[root@megalon c2a]# ./c2a.pl -md5 -f ./files.txt -o test1_md5.audit
c2a.pl, v1.0 (conf to audit)
(c) 2006 - Tenable Network Security
Creating .audit file based on inputfile ./files.txt
Processing :/etc/snort/snort.conf
Processing :/opt/sc3/daemons/daemons.cfg
Processing :/opt/pvs/etc/pvs.conf

Finished creating .audit files based on ./files.txt
Please check test1_md5.audit for output.

The content of the test1_md5.audit file is as shown:

#
# This file is auto-generated with c2a.pl script
# Copyright 2006 Tenable Network Security Inc
#

<check_type : "Unix">

<custom_item>
        #System :       "Linux"
        type    :       FILE_CHECK
description     :       "Check MD5 for /etc/snort/snort.conf"
        file    :       "/etc/snort/snort.conf"
        md5     :       "823b28cbc726f68538abdb0451f29a01"
</custom_item>

<custom_item>
        #System :       "Linux"
        type    :       FILE_CHECK
description     :       "Check MD5 for /opt/sc3/daemons/daemons.cfg"
        file    :       "/opt/sc3/daemons/daemons.cfg"
        md5     :       "3403c4c155a94c4c915ac0b01d4a60dc"
</custom_item>

<custom_item>
        #System :       "Linux"
        type    :       FILE_CHECK
description     :       "Check MD5 for /opt/pvs/etc/pvs.conf"
        file    :       "/opt/pvs/etc/pvs.conf"
        md5     :       "3b4a4b0b7a3cdf7fa3aff3c54f5e6ad4"
</custom_item>

</check_type>

This .audit file can now be used within the Security Center or Nessus 3 to scan a UNIX system for their MD5 checksums of those three files. Here are screen shots of the test1_md5.audit file being loaded into a Nessus 3 Windows GUI, as well as passing results for a scan of the box with the correct MD5 values:

MD5 Audit Results	Loading UNIX .audit File

If a directory of files is required to be monitored, simply specify the directory in the list of targets given to the c2a.pl script. The c2a.pl script will recognize the entry as a directory and then recursively loop through each file and subdirectory and create a unique MD5 entry for each. For example, in the above files.txt, if we had added "/opt/nessus/etc", it would have created an MD5 value for each file in that directory.

Note on Solaris, the MD5 tool may not be installed by default.
 

Auditing UNIX Configuration Files

Most UNIX applications have one or more text configuration files. The c2a tool can be used to extract the variables when any existing configuration file, and then also make a second pass to extract the values for each variable to produce a UNIX .audit file.

Two passes are necessary because not all text configuration files have the same format. Some have different characters (other than "#") that indicate comments. Some  have a  "variablename =  value"  format while others have a "variablename value" format.

The c2a tool is ideal for processing configuration files that have unique line-by-line content. If your configuration file has multi-line functionality, such as an XML config file, c2a is not ideal.

The basic process to create a UNIX .audit file from a given configuration file is as follows:

1. Extract the variable names to create a variable mapping for the c2a.map file. This involves selecting a regular expression that will identify the variable names and their values and then running the cmv.pl tool to create a temporary map file.
2. Manually edit the temporary map file to avoid any variables that should not be audited and then append it to the c2a.map file.
3. Add in the regular expression you used with your map name to the c2a_regex.map file.
4. Add in the name of your regular expression and file name to the input.txt file.
5. Once this is all in place, create a UNIX .audit file by running the c2a.pl tool.

The resulting UNIX .audit file can be applied to any Nessus 3 scanner or Security Center installation.

NOTE: Please keep in mind that the easiest way to detect change is to get MD5 values of all your remote files, and then test for a new MD5 value. The c2a tools and Nessus can indeed look inside text files and see if certain value are set, but you should only be performing these audits if that is indeed what you require for an audit. We will now look at two examples with Snort.

Snort Configuration File Example

In the next two sections, we will generate some .audit files for a Snort IDS configuration file. Snort is a good example of auditing a remote system configuration file because it has a text configuration file and has specific settings that dramatically change the operation of the application.

This blog entry won't recommend specific settings for Snort (a good link on the subject is here though), but if your organization decides on a central set of configurations, the c2a tool can help audit them to ensure compliance.

Extracting 'var' Variables from a snort.conf File

Most Snort deployments install the configuration file to /etc/snort/snort.conf. Looking at the file, there are many settings which are specified with "var variable-name value" such as:

var TELNET_SERVERS $HOME_NET

We will start by extracting these variables and then move onto other snort.conf keywords such as 'preprocessor' and 'include'.

A regular expression to extract the variable names specified by the 'var' keyword in a snort.conf file is  '(var .+) (.+)'.  We can use the cmv.pl tool and this regular expression to create a SNORTVAR.map file and then add it to the c2a.map file as shown below:

[root@megalon c2a]# ./cmv.pl -r '(var .+) (.+)' -r SNORTVAR -f /etc/snort/snort.conf
[root@megalon c2a]# more SNORTVAR.map
SNORTVAR=var HOME_NET
SNORTVAR=var EXTERNAL_NET
SNORTVAR=var DNS_SERVERS
SNORTVAR=var SMTP_SERVERS
SNORTVAR=var HTTP_SERVERS
SNORTVAR=var SQL_SERVERS
SNORTVAR=var TELNET_SERVERS
SNORTVAR=var SNMP_SERVERS
SNORTVAR=var HTTP_PORTS
SNORTVAR=var SHELLCODE_PORTS
SNORTVAR=var ORACLE_PORTS
SNORTVAR=var AIM_SERVERS
SNORTVAR=var RULE_PATH
[root@megalon c2a]# cat SNORTVAR.map >> c2a.map

We must then add the regular expression used to create this map file to the c2a_regex.map file and create a line for SNORTVAR in our input.txt file as shown below:

[root@megalon c2a]# cat c2a_regex.map
HTTP=([a-zA-Z0-9_-]+) (.*$)
SENDMAIL=([a-zA-Z0-9_-]+)=(.*$)
SYSCTL=([A-Za-z0-9\._-]+) = ([0-9]+)
NESSUS=([A-Za-z0-9\._-]+) = (.+)
VSFTPD=([A-Za-z0-9_]+)=([A-Za-z0-9]+)
SNORTVAR=(var .+) (.+)
[root@megalon c2a]# cat input.txt
# Given below is a list default locations
# of audit files that will be audited by
# c2a.pl when -audit option is selected. If
# the default locations do not reflect the
# settings on your system, please edit the
# same

SNORTVAR=/etc/snort/snort.conf

Once this is all in place, we can run the c2a.pl tool and look at the initial part of the resulting snort-var.audit file:

[root@megalon c2a]# ./c2a.pl -audit -f input.txt -o snort-var.audit
c2a.pl, v1.0 (conf to audit)
(c) 2006 - Tenable Network Security
Processing SNORTVAR=/etc/snort/snort.conf


Finished creating .audit files based on input.txt
Please check snort-var.audit for output

[root@megalon c2a]# more snort-var.audit
#####################################################
# This file is auto-generated with c2a.pl script
# Copyright 2007 Tenable Network Security Inc
# Date: Fri Feb 16 16:33:31 2007
# User: root
# Hostname: megalon
#####################################################

<check_type : "Unix">

#############################################################
# List of values to be audited in /etc/snort/snort.conf
#############################################################

<custom_item>
        #System :       "Linux"
        type    :       FILE_CONTENT_CHECK
description     :       "Check if var HOME_NET entry in /etc/snort/snort.conf is correct. "
        file    :       "/etc/snort/snort.conf"
        regex   :       "var HOME_NET.*"
        expect  :       "var HOME_NET any"
</custom_item>

The full snort-var.audit file can be obtained here:  [Download snort-var.audit]

There is nothing special about this .audit file as it just audits the default "var" settings of a snort.conf file and is intended purely as an example.

Extracting 'preprocessor' and 'include' Settings from a snort.conf File

The snort.conf file also has other values. Many of these are settings for the Snort 'preprocessor' and also signature 'include' settings. Here is a short snippet of 'include' and 'preprocessor' settings for a typical snort.conf file:

• include classification.config
• include reference.config
• include $RULE_PATH/local.rules
• include $RULE_PATH/bad-traffic.rules
• preprocessor http_inspect: global \
• preprocessor http_inspect_server: server default \
• preprocessor rpc_decode: 111 32771
• preprocessor bo
• preprocessor ftp_telnet: global \

There are two things we need to realize now.

First, unlike the 'var' processing we did previously where each unique variable also had a unique value, with 'include' and 'preprocessor' we have the same keyword used over and over with different values. To capture the entire string, we will use some regular expression logic and then have the c2a.pl tool create a full snort-conf.audit file.

Second, those "\" characters in the snort.conf file mean there is a multiple lines for each keyword. For example, the 'http_inspect_server:' value actually looks like this:

preprocessor http_inspect_server: server default \
    profile all ports { 80 8080 8180 } oversize_dir_length 500

Since the c2a.pl script doesn't consider multiple lines, either the snort.conf file needs to be rewritten to take advantage of putting the settings on one line, or the auditor needs to realize that only one line can be audited.

The following steps show command line entires to create a snort.audit file which can test for all settings of a snort.conf file:

The regular expression '(^preprocessor (.+))' finds all snort.conf lines that make declare one of these variables. We place the entire regular expression in parentheses to tell the c2a tool  we want each line uniquely tested for.

[root@megalon c2a]# ./cmv.pl -r '(^preprocessor (.+))' -t SNORTPP -f /etc/snort/snort.conf
[root@megalon c2a]# more SNORTPP.map
SNORTPP=preprocessor flow: stats_interval 0 hash 2
SNORTPP=preprocessor frag3_global: max_frags 65536
SNORTPP=preprocessor frag3_engine: policy first detect_anomalies
SNORTPP=preprocessor stream4: disable_evasion_alerts
SNORTPP=preprocessor stream4_reassemble
SNORTPP=preprocessor http_inspect: global \
SNORTPP=preprocessor http_inspect_server: server default \
SNORTPP=preprocessor rpc_decode: 111 32771
SNORTPP=preprocessor bo
SNORTPP=preprocessor ftp_telnet: global \
SNORTPP=preprocessor ftp_telnet_protocol: telnet \
SNORTPP=preprocessor ftp_telnet_protocol: ftp server default \
SNORTPP=preprocessor ftp_telnet_protocol: ftp client default \
SNORTPP=preprocessor smtp: \
SNORTPP=preprocessor sfportscan: proto  { all } \

Notice in the above matches, some of the lines contain a "\" which indicates that the line wasa multiple-line entry in the snort.conf file. For 'include' lines, the same type of regular expression is used:

[root@megalon c2a]# ./cmv.pl -r '(^include (.+))' -t SNORTSIGS -f /etc/snort/snort.conf
[root@megalon c2a]# more SNORTSIGS.map
SNORTSIGS=include classification.config
SNORTSIGS=include reference.config
SNORTSIGS=include $RULE_PATH/local.rules
SNORTSIGS=include $RULE_PATH/bad-traffic.rules
SNORTSIGS=include $RULE_PATH/exploit.rules
SNORTSIGS=include $RULE_PATH/scan.rules
SNORTSIGS=include $RULE_PATH/finger.rules
SNORTSIGS=include $RULE_PATH/ftp.rules
SNORTSIGS=include $RULE_PATH/telnet.rules
SNORTSIGS=include $RULE_PATH/rpc.rules
SNORTSIGS=include $RULE_PATH/rservices.rules
SNORTSIGS=include $RULE_PATH/dos.rules
SNORTSIGS=include $RULE_PATH/ddos.rules
SNORTSIGS=include $RULE_PATH/dns.rules
SNORTSIGS=include $RULE_PATH/tftp.rules
SNORTSIGS=include $RULE_PATH/web-cgi.rules
SNORTSIGS=include $RULE_PATH/web-coldfusion.rules
SNORTSIGS=include $RULE_PATH/web-iis.rules
SNORTSIGS=include $RULE_PATH/web-frontpage.rules
SNORTSIGS=include $RULE_PATH/web-misc.rules
SNORTSIGS=include $RULE_PATH/web-client.rules
SNORTSIGS=include $RULE_PATH/web-php.rules
SNORTSIGS=include $RULE_PATH/sql.rules
SNORTSIGS=include $RULE_PATH/x11.rules
SNORTSIGS=include $RULE_PATH/icmp.rules
SNORTSIGS=include $RULE_PATH/netbios.rules
SNORTSIGS=include $RULE_PATH/misc.rules
SNORTSIGS=include $RULE_PATH/attack-responses.rules
SNORTSIGS=include $RULE_PATH/oracle.rules
SNORTSIGS=include $RULE_PATH/mysql.rules
SNORTSIGS=include $RULE_PATH/snmp.rules
SNORTSIGS=include $RULE_PATH/smtp.rules
SNORTSIGS=include $RULE_PATH/imap.rules
SNORTSIGS=include $RULE_PATH/pop2.rules
SNORTSIGS=include $RULE_PATH/pop3.rules
SNORTSIGS=include $RULE_PATH/nntp.rules
SNORTSIGS=include $RULE_PATH/other-ids.rules
SNORTSIGS=include $RULE_PATH/virus.rules
SNORTSIGS=include $RULE_PATH/experimental.rules

Both map files are added to the c2a.map file.

[root@megalon c2a]# cat SNORTSIGS.map >> c2a.map
[root@megalon c2a]# cat SNORTPP.map >> c2a.map

The SNORTSIGS and SNORTPP map names are added to the input.txt file:

[root@megalon c2a]# cat input.txt
# Given below is a list default locations
# of audit files that will be audited by
# c2a.pl when -audit option is selected. If
# the default locations do not reflect the
# settings on your system, please edit the
# same

SNORTVAR=/etc/snort/snort.conf
SNORTPP=/etc/snort/snort.conf
SNORTSIGS=/etc/snort/snort.conf

The regular expressions for SNORTSIGS and SNORTPP are added to the c2a_regex.map file:

[root@megalon c2a]# cat c2a_regex.map
HTTP=([a-zA-Z0-9_-]+) (.*$)
SENDMAIL=([a-zA-Z0-9_-]+)=(.*$)
SYSCTL=([A-Za-z0-9\._-]+) = ([0-9]+)
NESSUS=([A-Za-z0-9\._-]+) = (.+)
VSFTPD=([A-Za-z0-9_]+)=([A-Za-z0-9]+)
SNORTVAR=(var .+) (.+)
SNORTPP=(^preprocessor (.+))
SNORTSIGS=(^include (.+))

The tool is then run to produce a snort.audit file:

[root@megalon c2a]# ./c2a.pl -audit -f ./input.txt -o snort.audit
c2a.pl, v1.0 (conf to audit)
(c) 2006 - Tenable Network Security
Processing SNORTVAR=/etc/snort/snort.conf
Processing SNORTPP=/etc/snort/snort.conf
Processing SNORTSIGS=/etc/snort/snort.conf

Finished creating .audit files based on ./input.txt
Please check snort.audit for output

The resulting scan results with a Nessus 3 scanner look like this:

A PDF of the HTML report produced by Nessus 3 of the scan can be obtained here:

Download snort-audit.pdf

The actual snort.audit file generated for this demo is located here:

Download snort.audit

If you inspect the file, you will see several preprocessor entries which were the first part of a multiple line snort.conf entry.

Other Uses for the c2a.pl tool

Tenable has included several entries in the c2a.map and c2a_regex.map files to enable auditing of Sendmail, the Very Secure FTP Daemon (VSFTPD), Apache, the Red Hat /etc/sysctl.conf file and Nessus. More may be added in the near future. If you'd like to submit new mappings to Tenable to share with other Nessus users, please send them to [email protected].

With that in mind, the c2a.pl script can be used to help create a Nessus .audit files for several live UNIX applications. Consider the following ideas:

• If your organization has many UNIX based firewalls, a .audit file can be generated to audit the common and required settings that each firewall is supposed to have.  For example, if all firewalls are supposed to have filtering of RFC 1918 addresses, the actual firewall rules can be tested for.
• If many different custom applications are being run out of CRON, the various CRONTABs can be audited to make sure that the right applications are being run at the correct time.
• For centralized logging, remote UNIX systems can have their SYSLOG, SYSLOG-NG and LOGROTATE configurations checked.

Manual Tweaking of the .audit files

Lastly, the output of the c2a.pl script  can also be manually edited. For example, consider combining the MD5 checksum rules with the FILE_CONTENT_CHECK rules into one rule.  The output  generated by the c2a.pl script also assumes that a configuration file is always in one place. Consider modifying the "file" keyword to specify other locations where a configuration file may be located.   

Lastly, if you have content that you don't want in your remote file configurations, consider manually adding in checks for that with the  FILE_CONTENT_CHECK_NOT keyword. This can help you perform audits for settings that should be present and should also not be present.   

Working with the Security Center

Although each of the above examples focused on direct Nessus usage, any of the .audit files generated with  the c2a.pl tool can be loaded into the SecurityCenter's /opt/sc3/admin/nasl directory and then used to perform credentialed audits of remote UNIX system and application configurations. We've previously blogged about doing this with Windows UNIX checks and it is very similar for the UNIX .audit file audits.

For More Information

To purchase Direct Feed (now called ProfessionalFeed), please visit the Tenable Store. SecurityCenter is available for the Red Hat operating system and can manage multiple Nessus scanners, perform configuration audits for compliance reporting, process network IDS logs, and also manage multiple Passive Vulnerability Scanners and Log Correlation Engines.

In addition to auditing UNIX servers, Nessus 3 can also be used to audit Windows servers. To learn more about auditing Windows servers, please refer to these blog posts:

• Auditing Windows 2003 Servers for Disabled USB Drives and AutoRun CD-ROM
• Creating "Gold Build" Audit Policies
• Nessus 3 Agentless Compliance Checks

Tenable's 80+ page Real-Time Compliance Monitoring paper is available to current and potential customers and discusses for logging, network monitoring, vulnerability scanning and configuration auditing can help monitor PCI, SOX, FISMA, NERC and many other compliance standards.

Obtaining the c2a Tool

The c2a tool, and all other UNIX and Windows compliance utilities, plus many example .audit files can be obtained here.