SpamAssassin How-to

[Guest anon]

SpamAssassin-ClamAV-Procmail-Howto

 

Version 1.0

Author: Falko Timme <ft [at] falkotimme [dot] com>

Last edited 02/29/2004

 

This document describes how to install SpamAssassin (for filtering SPAM) and ClamAV (for filtering viruses, trojans, worms, etc.) and how to invoke them by using procmail recipes. It is suitable for scenarios where Sendmail or Postfix deliver emails to local users. It should work (maybe with slight changes concerning paths etc.) on all *nix operating systems. I tested it on Debian Woody so far.

 

In the end you will have a system where Sendmail or Postfix deliver emails to a local user; the emails are passed to procmail which invokes SpamAssassin and ClamAV in order to filter the emails before they arrive in the user's inbox. However, the installation of Sendmail and Postfix are not covered in this document.

 

This howto is meant as a practical guide; it does not cover the theoretical backgrounds. They are treated in a lot of other documents in the web.

 

This document comes without warranty of any kind!

 

Please note: If you use the server control panel 42go ISP-Manager you do not need to follow this tutorial as the 42go ISP-Manager comes with SpamAssassin and ClamAV, and both can be configured through the 42go ISP-Manager!

 

 

 

1 Install SpamAssassin

 

There are multiple ways of installing SpamAssassin. I will describe three of them here:

1. 

 

Installation using the Perl Shell

 

Login to your command line as root and run the following command to start the Perl shell:

 

perl -MCPAN -e shell

 

If you run the Perl shell for the first time you will be asked some questions. In most cases the default answers are ok.

 

Please note: If you run a firewall on your system you might have to turn it off while working on the Perl shell in order for the Perl shell to be able to fetch the needed modules without a big delay. You can switch it on afterwards.

 

The big advantage of the Perl shell compared to the two other methods described here is that it cares about dependencies when installing new modules. I.e., if it turns out that a prerequisite Perl module is missing when you install another module the Perl shell asks you if it should install the prerequisite module for you. You should answer that question with "Yes".

 

Run the following commands to install SpamAssassin and some other needed modules:

 

install HTML::Parser

install DB_File

install Net::DNS (when prompted to enable tests, choose no)

install Digest::SHA1

install Mail::SpamAssassin

q (to leave the Perl shell)

 

If a module is already installed on your system you will get a message similar to this one:

 

HTML::Parser is up to date.

 

Successful installation of a module looks like this:

 

/usr/bin/make install -- OK

 

 

2. 

 

Installation from the Sources

 

(Please note: The prerequisite Perl modules (at least HTML::Parser) have to be installed before you compile SpamAssassin from the sources. If they are not, install them by using one of the other two methods described here, or get the sources from http://www.cpan.org and compile them. This is similar to the steps described here for SpamAssassin.)

 

cd /tmp

wget http://www.mirror.ac.uk/sites/spamassassin...sin-2.63.tar.gz

tar xvfz Mail-SpamAssassin-2.63.tar.gz

cd Mail-SpamAssassin-2.63

perl Makefile.PL

make

make install

 

 

3. 

 

Installation using Webmin

 

If you have webmin (http://www.webmin.com) installed on your system you can use it to install Perl Modules. Login to webmin, go to Others -> Perl Modules, and install SpamAssassin:

 

If you get error messages this is mostly due to the fact that some prerequisite modules are missing on your system. Install them (at least HTML::Parser is required), and then try to install the module again you wanted to install first.

 

SpamAssassin will be installed to /usr/local/share/spamassassin/.

 

 

 

2 Install ClamAV

 

cd /tmp

groupadd clamav

useradd -g clamav -s /bin/false -c "Clam AntiVirus" clamav

wget http://heanet.dl.sourceforge.net/sourcefor...mav-0.67.tar.gz

tar xvfz clamav-0.67.tar.gz

cd clamav-0.67

./configure --sysconfdir=/etc

 

(Please note: ./configure --help gives a list of all configuration options available.)

 

make

su -c "make install"

 

If you run

 

clamd

 

now you will get an error message:

 

ERROR: Please edit the example config file /etc/clamav.conf.

 

You must at least remove the Example directive. My /etc/clamav.conf looks like this:

 

##

## Example config file for the Clam AV daemon

## Please read the clamav.conf(5) manual before editing this file.

##

 

 

# Comment or remove the line below.

#Example

 

# Uncomment this option to enable logging.

# LogFile must be writable for the user running the daemon.

# Full path is required.

#LogFile /tmp/clamd.log

 

# By default the log file is locked for writing - the lock protects against

# running clamd multiple times (if want to run another clamd, please

# copy the configuration file, change the LogFile variable, and run

# the daemon with --config-file option). That's why you shouldn't uncomment

# this option.

#LogFileUnlock

 

# Maximal size of the log file. Default is 1 Mb.

# Value of 0 disables the limit.

# You may use 'M' or 'm' for megabytes (1M = 1m = 1048576 bytes)

# and 'K' or 'k' for kilobytes (1K = 1k = 1024 bytes). To specify the size

# in bytes just don't use modifiers.

#LogFileMaxSize 2M

 

# Log time with an each message.

#LogTime

 

# Use system logger (can work together with LogFile).

#LogSyslog

 

# Enable verbose logging.

#LogVerbose

 

# This option allows you to save the process identifier of the listening

# daemon (main thread).

#PidFile /var/run/clamd.pid

 

# Path to a directory containing .db files.

# Default is the hardcoded directory (mostly /usr/local/share/clamav,

# it depends on installation options).

#DatabaseDirectory /var/lib/clamav

 

# The daemon works in local or network mode. Currently the local mode is

# recommended for security reasons.

 

# Path to the local socket. The daemon doesn't change the mode of the

# created file (portability reasons). You may want to create it in a directory

# which is only accessible for a user running daemon.

LocalSocket /tmp/clamd

 

# Remove stale socket after unclean shutdown.

#FixStaleSocket

 

# TCP port address.

#TCPSocket 3310

 

# TCP address.

# By default we bind to INADDR_ANY, probably not wise.

# Enable the following to provide some degree of protection

# from the outside world.

#TCPAddr 127.0.0.1

 

# Maximum length the queue of pending connections may grow to.

# Default is 15.

#MaxConnectionQueueLength 30

 

# When activated, input stream (see STREAM command) will be saved to disk before

# scanning - this allows scanning within archives.

#StreamSaveToDisk

 

# Close the connection if this limit is exceeded.

#StreamMaxLength 10M

 

# Maximal number of a threads running at the same time.

# Default is 5, and it should be sufficient for a typical workstation.

# You may need to increase threads number for a server machine.

#MaxThreads 10

 

# Thread (scanner - single task) will be stopped after this time (seconds).

# Default is 180. Value of 0 disables the timeout. SECURITY HINT: Increase the

# timeout instead of disabling it.

#ThreadTimeout 500

 

# Maximal depth the directories are scanned at.

MaxDirectoryRecursion 15

 

# Follow a directory symlinks.

# SECURITY HINT: You should have enabled directory recursion limit to

# avoid potential problems.

#FollowDirectorySymlinks

 

# Follow regular file symlinks.

#FollowFileSymlinks

 

# Do internal checks (eg. check the integrity of the database structures)

# By default clamd checks itself every 3600 seconds (1 hour).

#SelfCheck 600

 

# Execute a command when virus is found. In the command string %v and %f will

# be replaced by the virus name and the infected file name respectively.

#

# SECURITY WARNING: Make sure the virus event command cannot be exploited,

#                    eg. by using some special file name when %f is used.

#                    Always use a full path to the command.

#                    Never delete/move files with this directive !

#VirusEvent /usr/local/bin/send_sms 123456789 "VIRUS ALERT: %f: %v"

 

# Run as selected user (clamd must be started by root).

# By default it doesn't drop privileges.

User clamav

 

# Initialize the supplementary group access (for all groups in /etc/group

# user is added in. clamd must be started by root).

#AllowSupplementaryGroups

 

# Don't fork into background. Useful in debugging.

#Foreground

 

# Enable debug messages in libclamav.

#Debug

 

##

## Mail support

##

 

# Uncomment this option if you are planning to scan mail files.

ScanMail

 

##

## Archive support

##

 

 

# Comment this line to disable scanning of the archives.

ScanArchive

 

 

# By default the built-in RAR unpacker is disabled by default because the code

# terribly leaks, however it's probably a good idea to enable it.

#ScanRAR

 

 

# Options below protect your system against Denial of Service attacks

# with archive bombs.

 

# Files in archives larger than this limit won't be scanned.

# Value of 0 disables the limit.

# WARNING: Due to the unrarlib implementation, whole files (one by one) in RAR

#        archives are decompressed to the memory. That's why never disable

#        this limit (but you may increase it of course!)

ArchiveMaxFileSize 10M

 

# Archives are scanned recursively - e.g. if Zip archive contains RAR file,

# the RAR file will be decompressed, too (but only if recursion limit is set

# at least to 1). With this option you may set the recursion level.

# Value of 0 disables the limit.

ArchiveMaxRecursion 5

 

# Number of files to be scanned within archive.

# Value of 0 disables the limit.

ArchiveMaxFiles 1000

 

# Use slower decompression algorithm which uses less memory. This option

# affects bzip2 decompressor only.

#ArchiveLimitMemoryUsage

 

##

## Clamuko settings

## WARNING: This is experimental software. It is very likely it will hang

##            up your system !!!

##

 

# Enable Clamuko. Dazuko (/dev/dazuko) must be configured and running.

#ClamukoScanOnLine

 

# Set access mask for Clamuko.

ClamukoScanOnOpen

ClamukoScanOnClose

ClamukoScanOnExec

 

# Set the include paths (all files in them will be scanned). You can have

# multiple ClamukoIncludePath options, but each directory must be added

# in a seperate option. All subdirectories are scanned, too.

ClamukoIncludePath /home

#ClamukoIncludePath /students

 

# Set the exclude paths. All subdirectories are also excluded.

#ClamukoExcludePath /home/guru

 

# Limit the file size to be scanned (probably you don't want to scan your movie

# files ;))

# Value of 0 disables the limit. 1 Mb should be fine.

ClamukoMaxFileSize 1M

 

# Enable archive support. It uses the limits from clamd section.

# (This option doesn't depend on ScanArchive, you can have archive support

# in clamd disabled).

# ClamukoScanArchive

 

Now we have to create an init script for ClamAV (/etc/init.d/clamd):

 

#!/bin/bash

 

 

TMPDIR=/tmp

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin

 

 

case "$1" in

    start)

        echo "Starting ClamAV..."

        if [ -S /tmp/clamd ]; then

          echo "ClamAV is already running!"

        else

          /usr/local/bin/freshclam -d -c 10 --datadir=/usr/local/share/clamav

          /usr/local/sbin/clamd

        fi

        echo "ClamAV is now up and running!"

    ;;

    stop)

        echo "Shutting down ClamAV..."

        array=(`ps ax | grep -iw '/usr/local/bin/freshclam' | grep -iv 'grep' \

      | awk '{print $1}' | cut -f1 -d/ | tr '\n' ' '`)

        element_count=${#array[@]}

        index=0

        while [ "$index" -lt "$element_count" ]

        do

          kill -9 ${array[$index]}

          let "index = $index + 1"

        done

        array=(`ps ax | grep -iw '/usr/local/sbin/clamd' | grep -iv 'grep' \

      | awk '{print $1}' | cut -f1 -d/ | tr '\n' ' '`)

        element_count=${#array[@]}

        index=0

        while [ "$index" -lt "$element_count" ]

        do

          kill -9 ${array[$index]}

          let "index = $index + 1"

        done

        if [ -S /tmp/clamd ]; then

          rm -f /tmp/clamd

        fi

        echo "ClamAV stopped!"

    ;;

    restart)

        $0 stop  && sleep 3

        $0 start

    ;;

    *)

    echo "Usage: $0 {start|stop|restart}"

    exit 1

esac

exit 0

 

chmod 755 /etc/init.d/clamd

 

Now we start ClamAV:

 

/etc/init.d/clamd start

 

If you run

 

ps aux

 

you will now notice some clamd processes (which use the socket /tmp/clamd) and a freshclam process which is responsible for getting the newest virus signature updates. They are located under /usr/local/share/clamav. The command

 

/usr/local/bin/freshclam -d -c 10 --datadir=/usr/local/share/clamav

 

in our clamd init script makes sure that freshclam checks for new signatures 10 times per day.

 

In order to start ClamAV at boot time do the following:

 

ln -s /etc/init.d/clamd /etc/rc2.d/S20clamd

ln -s /etc/init.d/clamd /etc/rc3.d/S20clamd

ln -s /etc/init.d/clamd /etc/rc4.d/S20clamd

ln -s /etc/init.d/clamd /etc/rc5.d/S20clamd

ln -s /etc/init.d/clamd /etc/rc0.d/K20clamd

ln -s /etc/init.d/clamd /etc/rc1.d/K20clamd

ln -s /etc/init.d/clamd /etc/rc6.d/K20clamd

 

 

 

3 Install trashscan

 

trashscan is a shell script that makes the connection between procmail and ClamAV (i.e., when an email arrives, procmail is invoked which itself invokes trashscan in order to have the mail scanned for viruses by ClamAV). It comes with ClamAV.

 

cd /tmp/clamav-0.67/contrib/trashscan

tar xvfz trashscan-0.08.tar.gz

cd trashscan-0.08

cp -pf trashscan /usr/local/sbin/

 

Now we have to adjust some variables in the "Settinx" section of /usr/local/sbin/trashscan. My settings are as follows:

 

#!/bin/bash

#

# TrashScan v0.08; Scan email for viruses

# ZapCoded by Trashware; 13.10.2002

# Email: trashware@gmx.de

# Web: http://trashware.mirrorz.com

#

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin

# --------------------------------------- Begin Settinx ---------------------------------------- #

SCANDIR=$HOME/tmp                                              # Temp directory for virus scans.

                                                            # Security: Don't define public

                                                            # accessible directories here !!!

                                                            # $HOME/tmp should be fine.

#DECODER=metamail                                              # Decoder: "metamail" or "uudeview"

#DECODPRG=metamail                                          # Absolute path to decoder: metamail

DECODER=uudeview                                            # Decoder: "metamail" or "uudeview"

DECODPRG=/usr/local/bin/uudeview                            # Absolute path to decoder: uudeview

VSCANPRG=/usr/local/bin/clamscan                            # Absolute path to the virus scanner

VSCANOPT="--quiet --tempdir=$HOME/tmp --recursive --max-files=500 \

        --max-space=30M --unzip=/usr/bin/unzip --unrar=/usr/bin/unrar \

        --unarj=/usr/bin/unarj --zoo=/usr/bin/zoo --lha=/usr/bin/lha \

        --jar=/usr/bin/unzip --tar=/bin/tar --tgz=/bin/tar"    # Parameters for the virus scanner.

                                                            # Security: Don't choose public

                                                            # accessible directories for the

                                                            # --tempdir definition !!!

                                                            # --tempdir=$HOME/tmp should be fine.

VSCANVEX=1                                                  # Exitcode of the virus scanner if a

                                                            # virus was found

VSCANSUSP=mail.virus                                        # File to store suspicious mail (see

                                                            # procmail.trashscan)

FORMAIL=formail                                                # Absolute path to formail

PROCMAIL=procmail                                              # Absolute path to procmail

SENDMAIL=sendmail                                              # Absolute path to sendmail

CAT=cat                                                        # Absolute path to cat

GREP=grep                                                      # Absolute path to grep

LOGGER=logger                                                  # Absolute path to logger

LOGPRIO=mail.warn                                              # Log level for logger

MKDIR=mkdir                                                    # Absolute path to mkdir

RM=rm                                                          # Absolute path to rm

SED=sed                                                        # Absolute path to sed

ALERTRCVR=virusadmin@example.com                            # Receiver of virus alert messages

ALERTSNDR=virusadmin@example.com                            # Sender of virus alert messages

ALERTCTCT=virusadmin@example.com                            # Person to contact (appears in the

                                                            # mail body of the virus alert)

# ---------------------------------------- End Settinx ---------------------------------------- #

 

 

Please note that I set the PATH variable at the beginning of the script:

 

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin

 

This allows me not to specify the absolute path of most of the programs needed by trashscan (e.g. formail, procmail, sendmail) as long as they are in the PATH.

 

VSCANOPT specifies the paths to some programs needed to unpack files in various compression formats (if an email comes with a compressed attachment, e.g. zip, tar.gz). You do not need all the programs there, but I recommend that you have at least unzip and tar installed (if you have not, use http://www.rpmfind.net to search for unzip and tar if you use an rpm based distribution, and install the appropriate packages with

 

rpm -ivh name-of-package.rpm

 

If you use Debian, all you have to do is

 

apt-get install unzip tar

 

).

 

Be sure to specify the correct email address of the person that will receive a notification if a virus is found.

 

 

 

4 Install uudeview

 

trashscan needs a program to decode email messages. In the trashscan settings above I specified that trashscan should use uudeview which we will install now.

 

cd /tmp

wget http://www.fpx.de/fp/Software/UUDeview/dow...w-0.5.19.tar.gz

tar xvfz uudeview-0.5.19.tar.gz

cd uudeview-0.5.19

./configure

make

make install

 

 

 

5 Configure Procmail

 

procmail is normally installed on most distributions by default so I will not cover procmail installation here. Run

 

which procmail

 

to find out where your procmail is located (in my case it is /usr/bin/procmail).

 

I will now show how to configure procmail for the user testuser who has his homedir under /home/www/web1/user/testuser. Be sure that none of the directories in this path (/home, /home/www, /home/www/web1, /home/www/web1/user, /home/www/web1/user/testuser) is group- or world-writable. They should have the permissions rwxr-xr-x (or 755). Otherwise procmail could refuse to work properly!

 

First we have to create the file /home/www/web1/user/testuser/.forward so that procmail will be invoked when a mail for testuser arrives. It has the following contents:

 

"|/usr/bin/procmail -f-"

 

chown testuser /home/www/web1/user/testuser/.forward

chmod 600 /home/www/web1/user/testuser/.forward

 

Now we create the file /home/www/web1/user/testuser/.procmailrc. This is the file where procmail will look for recipes (i.e., commands to run). For reasons of clearness we simply include our main recipes in this file:

 

## MAILDIR=$HOME/Maildir/

## DEFAULT=$MAILDIR

 

INCLUDERC=/home/www/web1/user/testuser/.antivirus.rc

INCLUDERC=/home/www/web1/user/testuser/.html-trap.rc

INCLUDERC=/home/www/web1/user/testuser/.spamassassin.rc

 

(Please note: Uncomment the first two lines if you use Maildir for your emails, i.e., your emails are stored under /home/www/web1/user/testuser/Maildir/ instead of /var/spool/mail.)

 

Our first recipe is /home/www/web1/user/testuser/.antivirus.rc:

 

#

# procmail configuration for TrashScan: ZapCoded by Trashware; 13.10.2002

#

 

# [ ... ]

 

# ------------------------------------------------------------------------------------- #

# Virus scan section ...                                                                #

# ------------------------------------------------------------------------------------- #

 

# 1. Run TrashScan

:0

* multipart

* !^X-Virus-Scan:

| /usr/local/sbin/trashscan

 

 

# 2. Filter tagged virus mails

:0:

* ^X-Virus-Scan: Suspicious

/dev/null

 

/home/www/web1/user/testuser/.html-trap.rc is discussed below so our second recipe is /home/www/web1/user/testuser/.spamassassin.rc:

 

# SpamAssassin sample procmailrc

#

# Pipe the mail through spamassassin (replace 'spamassassin' with 'spamc'

# if you use the spamc/spamd combination)

# The condition line ensures that only messages smaller than 250 kB

# (250 * 1024 = 256000 bytes) are processed by SpamAssassin. Most spam

# isn't bigger than a few k and working with big messages can bring

# SpamAssassin to its knees.

:0fw

* < 256000

| /usr/local/bin/spamassassin --prefs-file=/home/www/web1/user/testuser/.user_prefs

 

# All mail tagged as spam (eg. with a score higher than the set threshold)

# is moved to "/dev/null".

#:0:

#* ^X-Spam-Status: Yes

#/dev/null

 

# Work around procmail bug: any output on stderr will cause the "F" in "From"

# to be dropped.  This will re-add it.

:0

* ^^rom[ ]

{

  LOG="*** Dropped F off From_ header! Fixing up. "

 

  :0 fhw

  | sed -e '1s/^/F/'

}

 

This will cause all emails to be accepted, even SPAM (which will be marked as SPAM and can be soted out by the user's email client). This strategy is recommended in the first stage until you are sure that SpamAssassin identifies your emails correctly. If you want to delete SPAM take this .spamassassin.rc instead:

 

# SpamAssassin sample procmailrc

#

# Pipe the mail through spamassassin (replace 'spamassassin' with 'spamc'

# if you use the spamc/spamd combination)

# The condition line ensures that only messages smaller than 250 kB

# (250 * 1024 = 256000 bytes) are processed by SpamAssassin. Most spam

# isn't bigger than a few k and working with big messages can bring

# SpamAssassin to its knees.

:0fw

* < 256000

| /usr/local/bin/spamassassin --prefs-file=/home/www/web1/user/testuser/.user_prefs

 

# All mail tagged as spam (eg. with a score higher than the set threshold)

# is moved to "/dev/null".

:0:

* ^X-Spam-Status: Yes

/dev/null

 

# Work around procmail bug: any output on stderr will cause the "F" in "From"

# to be dropped.  This will re-add it.

:0

* ^^rom[ ]

{

  LOG="*** Dropped F off From_ header! Fixing up. "

 

  :0 fhw

  | sed -e '1s/^/F/'

}

 

Next we create the file /home/www/web1/user/testuser/.user_prefs which will contain testuser's SpamAssassin settings:

 

# SpamAssassin user preferences file.  See 'perldoc Mail::SpamAssassin::Conf'

# for details of what can be tweaked.

#*

#* Note: this file is not read by SpamAssassin until copied into the user

#* directory. At runtime, if a user has no preferences in their home directory

#* already, it will be copied for them, allowing them to perform personalised

#* customisation.  If you want to make changes to the site-wide defaults,

#* create a file in /etc/spamassassin or /etc/mail/spamassassin instead.

###########################################################################

 

# How many hits before a mail is considered spam.

required_hits                5.0

 

rewrite_subject    1

subject_tag        ***SPAM***

 

SpamAssassin runs a number of tests on each email in order to determine whether it is SPAM or not. Each test assigns a certain amount fo points to that email (if the test is positive). The points will be added. required_hits is the amount of points above which the email is considered as SPAM. 5.0 is a reasonable value to start with.

 

If rewrite_subject is 1 the subject of the email will be tagged with the value of subject_tag if the email is considered as SPAM so that the email can be sorted by testuser's email client if he chose the appropriate .spamassassin.rc above.

 

6 Configure the Email Sanitizer

 

The Email Sanitizer (http://www.impsec.org/email-tools/procmail-security.html) is a set of procmail recipes that form a sort of content filter. E.g., it can deactivate malicious javascript code in HTML emails and rename suspicious attachments (e.g. example.exe is renamed to example.12345DEFANGED-exe so that it cannot be opened by a simple double-click under Windows. It has to be saved to the disk first and then be renamed consciously. So the recipient is forced to think about if he should open the attachment.).

 

cd /tmp

wget http://www.impsec.org/email-tools/html-tra...rap.procmail.gz

gunzip html-trap.procmail.gz

echo 'PATH="/usr/bin:$PATH:/usr/local/bin"' > /home/www/web1/user/testuser/.html-trap.rc

echo 'SHELL=/bin/sh' >> /home/www/web1/user/testuser/.html-trap.rc

cat html-trap.procmail >> /home/www/web1/user/testuser/.html-trap.rc

 

7 Test your Configuration

 

You can now test your configuration by sending .exe attachments, sample SPAM and sample viruses (if you have some) to testuser.

 

Look at the header of received emails. It should contain the following lines:

 

X-Security: MIME headers sanitized on server1.example.com See http://www.impsec.org/email-tools/sanitizer-intro.html for details. $Revision: 1.140 $Date: 2004-02-11 20:47:43-08

 

X-Virus-Scan: Scanned by TrashScan v0.08 running on server1.example.com

 

X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server1.example.com

Links

SpamAssassin: http://www.spamassassin.org/