Managing amavisd-new quarantine from the command line

Just a quick one today. It seems to be a recurring problem for me that every once in a while I want to go into Amavisd-new’s quarantine and look for false positives (not that there should be any if it’s setup right). There are a number or graphical ways of doing this but most of them aren’t available in the Ubuntu repositories. The 2 that I could find are Horde-sam and Webmin’s Clamav module (which I have used before and is pretty easy to use if you are command-line averse). I didn’t want to add another service in order to keep resource usage as low as possible so I set about finding a way to check each email from the command line. So firstly to cycle through every quarantined email use the following command in the quarantine directory (normally /var/lib/amavis/virusmails in Ubuntu):-

Continue reading Managing amavisd-new quarantine from the command line

Filtering POP3 mailboxes with fdm

Place the following in /usr/local/bin/, use the /etc/procmailrc below, and make the perl script executable.


use Regexp::Common qw/net/;

while (<>) {
 /$RE{net}{IPv4}{dec}{-keep}/ and $last = $1 unless /127\.0\.0\.1|(10\.\d+|172\.(1[6-9]|2\d|3[0-1])|192\.168)(\.\d+){2}/;

print $last, "\n";
# Global procmail definitions
# Define a file for procmail to send it's log information.
# Make sure procmail verbose logging is turned off.
# Define a new line character for use in procmail LOG entries.
# note: the quote spanning two lines below is deliberate.
# Directory where we will store mail folders
# Note: This directory MUST exist!
#Mail folder for incoming whitelisted listmail
# Location of formail on our system. (for use in procmail actions since
# those typically need a shell meta pattern in procmail action lines to work as intended)
# Location of file containing From: addresses of people we correspond with on a regular basis
#Location of a folder containing blacklisted email.
# Uncomment this if you would rather just delete the blacklisted email.
# Location of a file containing regular expressions of patterns that we don't want.
# to see in the Subject: From: or Reply-to: headers
# Location of file containing To: addresses we have given to news letters
# or web sites that map to my real account via sendmail aliases.
# Capture the message ID string (if any) for future reference in log entries.
* ^Message-ID:
{ MESSAGEID=`${FORMAIL} -cx "Message-ID:" |sed -e 's/[ \t]\{1,\}//g'` }
:0 E
{ MESSAGEID='none' }
# Sample procmail recipe to enumerate the Received: headers, and store them
# in the ${RECEIVEDHEAD} variable. Note the backtics that launch an embedded
# shell script.
:0 W
* H ?? 1^1 ^Received:
RECEIVEDHEAD=`${FORMAIL} -X "X-Originating-IP" -X "Received" | /usr/local/bin/`
# Sample procmail recipe which will extract the IPv4 address from the first
# Received: header. This could be adapted if you have several internal
# servers through which the mail passes.
# Also, the header IP extraction in this recipe is assuming that the header line was
# generated by sendmail. If you are using another server, you may need to adjust
# the regular expression to accommodate that.
# Initialize the SOURCEIP variable
* RECEIVEDHEAD ?? [0-9]+\.[0-9]+\.[0-9]+\.[0-9]+
LOG="[$$]$_: Extracted IP ${SOURCEIP} from Received: headers.${NL}"
:0 E
{ LOG="[$$]$_: Failed to find any source IP in the first Received: header.${NL}" }
# Sample procmail recipe which will generate the reverse IPv4 from
# the SOURCEIP, for use in blocklist lookups.
# It will also verify that the number we are looking at is a real Internet
# address.
# Initialize the SOURCEIPREV variable
# Check for valid IPv4 address range.
# Then if the address is not an IANA non-routable address
# generate the reverse IP for use in subsequent DNS lookups.
# Build a procmail style regular expression to test for a valid IPv4 range.
# Build a procmail style regular expression to test for IPv4 ranges that should not be used on the Internet.
# These are based on RFC-3330 Para 3 summary table.
# Note: These expressions should be periodically verified and updated as needed
# Combine the above into one regular expression.
# Note: IP is included in network defined above
#       as part of the CLASSA regular expression variable.
* ! SOURCEIP ?? ^(000\.000\.000\.000)$
* $ ! SOURCEIP ?? ^${RFC_3330_INVALID}$
* SOURCEIP ?? ^[0-9]+\.[0-9]+\.[0-9]+\.\/[0-9]+
{ QUAD4=${MATCH} }
* SOURCEIP ?? ^[0-9]+\.[0-9]+\.\/[0-9]+
{ QUAD3=${MATCH} }
* SOURCEIP ?? ^[0-9]+\.\/[0-9]+
{ QUAD2=${MATCH} }
* SOURCEIP ?? ^\/[0-9]+
{ QUAD1=${MATCH} }
LOG="[$$]$_: IP ${SOURCEIP} is a valid IPv4 address${NL}"
:0 E
LOG="[$$]$_: IP ${SOURCEIP} is an IANA Non-Routable IPv4 address${NL}"
:0 E
LOG="[$$]$_: Error - ${SOURCEIP} has an invalid range for an IPv4 address.${NL}"
# Here is another example of a more complex blocklist lookup technique
# which will lookup an IP on, decode the response, and
# tag the email.
# References:
* SPAMHAUSLOOKUP ?? 127\.0\.0\.([2-9]|1[01])$
# SBL Spamhaus Maintained
* SPAMHAUSLOOKUP ?? 127\.0\.0\.2$
# --- reserved for future use
* SPAMHAUSLOOKUP ?? 127\.0\.0\.3$
# XBL CBL Detected Address
* SPAMHAUSLOOKUP ?? 127\.0\.0\.4$
# XBL NJABL Proxies (customized)
* SPAMHAUSLOOKUP ?? 127\.0\.0\.5$
# XBL reserved for future use
* SPAMHAUSLOOKUP ?? 127\.0\.0\.6$
# XBL reserved for future use
* SPAMHAUSLOOKUP ?? 127\.0\.0\.7$
# XBL reserved for future use
* SPAMHAUSLOOKUP ?? 127\.0\.0\.8$
# --- reserved for future use
* SPAMHAUSLOOKUP ?? 127\.0\.0\.9$
# PBL ISP Maintained
* SPAMHAUSLOOKUP ?? 127\.0\.0\.10$
# PBL Spamhaus Maintained
* SPAMHAUSLOOKUP ?? 127\.0\.0\.11$
{ SPAMHAUSLOG="${SPAMHAUSLOG}PBL-SpamHaus Maintained, " }
SPAMHAUSLOG=`echo "${SPAMHAUSLOG}" |sed -e "s/, $/\n\tSee: http:\/\/www\.spamhaus\.org\/query\/bl\?ip=${SOURCEIP}/"`
LOG="[$$]$_: Result codes: ${SPAMHAUSLOG}${NL}"
:0 f
|${FORMAIL} -A "X-blocklists: ${SOURCEIP} found in SpamHaus. Blocklist lookup results: ${SPAMHAUSLOG}"

Obviously this is open to abuse because the spammer could add additional received headers to throw the recipe off the scent, however I can’t see that happening unless this becomes a popular way of filtering. Also if there are any non RFC compliant SMTP servers between the spammer and you they may mess with the headers which could screw the whole thing up also. You can’t please all the people all the time.

Update 09/09/10 – While this is an interesting and educational excercise it’s important to point out that the protection afforded by default in Ubuntu by using Amavisd-new and Spamassassin will automatically do all this and more. Spamassassin checks by default and additionally scans every ip in the received headers against this black list (and many more) except for “trusted” and reserved addresses. By default in Ubuntu, Spamassassin will assume every IP in the headers except your own is un-trusted.

The merits of LXC containers

First of all what are containers? In this context we are referring to Linux running inside a “containerised” environment i.e. an environment which is to all intents and purposes isolated from the main operating system. The containerised environment doesn’t have it’s own kernel or virtualised hardware and runs at native speeds because of this.

In what ways could a containerised environment be useful and desirable from a business perspective? The first use that springs to mind is being able to easily move your dedicated server from one hoster to another (which is the way I use it). You can simply “rsync” the contents of the container from the source hoster, then take the container down, run another quick rsync and start the container at the new hoster. This is really useful, saving hours and possibly days rebuilding the server at the new hoster in a more conventional manner.

Continue reading The merits of LXC containers