by
Etienne Liebetrau
The Sophos SG series UTM has a number of log events that you can send to your remote syslog servers, however they are not selective, and there are no options for customized filtering.
If you specify multiple destination syslog servers, they will all receive the same syslog information. When you have several syslog servers for varying reasons, this results in more data being sent than is needed by the destination.
This article explains how to configure syslog-ng to filter and forward Sophos UTM syslog data to multiple syslog servers with different data requirements.
I have two syslog requirements.
Although Fastvue Sophos Reporter can filter reports and alerts by subdivision or subsidiary (defined as IP subnets), it will still import and store all web filtering log data, consuming unnecessary bandwidth and disk space. The live dashboards will also display all traffic as these cannot be filtered currently.
Fastvue Sophos Reporter will reject syslog data that is not of the 'web filter' format, so sending the extraneous logs is not a problem for disk space. But if bandwidth is a consideration, you definitely want to drop the very large streams of firewall and reverse proxy logs.
Since we cannot filter syslog data at the source or destination, we need to introduce a 'filtering syslog proxy' where we can apply the required filtering before passing it on to the destination.
All Sophos UTM syslog data will be sent to the syslog proxy, then the proxy will forward messages that meet the filtering criteria (the subsidiary's subnet) on to Fastvue Sophos Reporter instance that only needs data for the subsidiary.
For the syslog proxy, we can use a simple Linux box running syslog-ng. I am using the Ubuntu Server distro for this. There are a number of very good articles explaining how to setup an Ubuntu server, so I'll jump straight into the syslog configuration.
By default, syslog-ng will process configuration files contained within the following folder.
/etc/syslog-nd/conf.d/
I created a file here called utm.conf.
First, let's set syslog-ng to listen for syslog messages on UDP port 514 with the following:
source s_udp { udp(port(514)); };
Now that we are set up to receive messages, we can create some filters.
First, we limit the messages that contain a source address that matches the subsidiary's 10.55.0.0/16 subnet.
filter f_div-subnet {message("10.55.")};
Next, we filter the Syslog messages so that only the httpproxy events are forwarded.
filter f_utm-http {message ("httpproxy")};
By default, syslog-ng will accept inbound messages and then forward them on with the required RFC 5424 headers. Sophos UTM has already included these headers so this is what the result looks like:
2016-11-28T13:52:33+02:00 10.254.0.10 2016 - - - 2016:11:28-13:53:47 sutm01-1 httpproxy
[10821]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.55.229.181" dstip="" user="" group="" ad_domain="" statuscode="200" cached="1" profile="REF_HttProContaMgmt1Netwo3 (Wifi WW-Corp-LAN)" filteraction="REF_DefaultHTTPCFFBlockAction (Default content filter block action)" size="1141" request="0x820f2c00" url="https://crl.microsoft.com/pki/crl/products/MicRooCerAut2011_2011_03_22.crl" referer="" error="" authtime="0" dnstime="0" cattime="0" avscantime="0" fullreqtime="1156" device="0" auth="2" ua="Microsoft-CryptoAPI/6.1" exceptions="av,sandbox,auth,url,ssl,application,fileextension,size" content-type="application/pkix-crl"
This 'double-header' problem prevents the messages from being accepted by the destination syslog server as they no longer match the expected format.
To get around this we use a syslog-ng template. We specify the content of the message and explicitly define the header.
template no_head { template ("${MSGHDR}${MSG}\n"); template_escape(no); }
Now the messages are compliant again
2016:11:28-13:53:47 sutm01-1 httpproxy[10821]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.55.229.181" dstip="" user="" group="" ad_domain="" statuscode="200" cached="1" profile="REF_HttProContaMgmt1Netwo3 (Wifi WW-Corp-LAN)" filteraction="REF_DefaultHTTPCFFBlockAction (Default content filter block action)" size="1141" request="0x820f2c00" url="https://crl.microsoft.com/pki/crl/products/MicRooCerAut2011_2011_03_22.crl" referer="" error="" authtime="0" dnstime="0" cattime="0" avscantime="0" fullreqtime="1156" device="0" auth="2" ua="Microsoft-CryptoAPI/6.1" exceptions="av,sandbox,auth,url,ssl, application,fileextension,size" content-type="application/pkix-crl"
By default, syslog-ng simply writes to a text file. Instead, we want to forward the filtered syslog data to a network destination that will also be listening on UDP port 514. Messages sent to this destination also need to be formatted with our 'no_head' template:
destination d_fastvue { udp("10.40.5.146" port(514) template(no_head)); };
At the end of the config file we instruct syslog-ng what log actions to perform.
log { source(s_udp); filter(f_wfs-subnet); filter(f_utm-http); destination(d_fastvue); };
Below is the complete configuration file. There is an additional file location destination you can uncomment for troubleshooting purposes.
##########
Source
##########
source s_udp { udp(port(514)); };
##########
Filter
##########
filter f_div-subnet {message("10.55.")}; filter f_utm-http {message ("httpproxy")};
############
Template
############
template no_head { template ("${MSGHDR}${MSG}\n"); template_escape(no); };
###############
Destination
###############
# Set destination to Fastvue server destination d_fastvue { udp("10.40.5.146" port(514) template(no_head)); };
destination d_file { file("/var/log/syslogtemp" template(no_head)); };
##############
Log Action
##############
log { source(s_udp); filter(f_wfs-subnet); filter(f_utm-http); destination(d_fastvue); # destination(d_file); };
After creating and saving this file, restart syslog-ng for the changes to take effect. To do this, use the following command:
systemctl restart syslog-ng.service
You can check the status of the service with:
systemctl status syslog-ng.service
You can now specify the syslog proxy as a source in Fastvue Sophos Reporter (Settings | Sources) and if everything has been configured correctly, you should soon see log records flowing in, populating the live dashboards.
If you are seeing records arriving but the dates remain '1970 something', and no data is shown in Settings | Data Storage, then the messages are malformed and cannot be parsed by Sophos Reporter.
To troubleshoot the messages being received by Fastvue Sophos Reporter, enable 'Full' diagnostic logging, by going to Settings | Diagnostic and increasing the logging level to 'Full'. All syslog messages received by the Fastvue machine will be logged to the diagnostic Dashboard.log file in Fastvue Sophos Reporter's data location (shown in Settings | Data Storage). Check the format of these syslog messages against the Sophos UTM 'live log' or the archived log files.
If you are looking for a simple syslog server for Windows that logs your Sophos UTM syslog data to text files, automatically rolling over each day, and compressing and archiving them after 30 days (configurable), then you may like to check out our free Fastvue Syslog tool for Window. It also supports syslog forwarding, however filtering syslog data is not supported at this stage.
Download our FREE 30-day trial, or schedule a demo and we'll show you how it works.
Deploying Endpoint Protection with Sophos UTM and Enterprise Console
How to Accurately Monitor and Improve Sophos UTM CPU Performance