Tuesday, November 20, 2007

The War on Spam

It was earlier this year that I last detailed some steps taken in an attempt to thwart the growing junk email (a.k.a. SPAM) epidemic. dspam has continued to work like a champ, with an accuracy rate currently standing at 99.13%.

However, the volume of spam continued to increase, almost as inevitable as the rising tide. I didn't let it bother me too much, as I could simply delete the messages sent to my spam folder, although I had to admit that looking through all of those messages for false positives was getting a bit tiresome.

And then it happened... The event that moved me past the tipping point, and thrust me forcefully back tot he front lines of the war on spam. The event seemed innocent enough: I went to send an email, and our smtp server (qmail) did not respond. wtf?

I shut down qmail, restarted it, and it seemed to work for a while. However, after an hour or so, the same problem arose. wtf?

Looking at the process list, I noticed that there were 40 qmail-smtpd processes running. What were they trying to send? Time to get my hands dirty and learn a bit about qmail's queue. I read a bit on the queue, and installed a tool called qmqtool. By using this new tool, I discovered that I had over 6,000 messages awaiting deliver in my mail queue!

Why so many? Because for every spammer that sends an email to a bogus address, a mail server will craft a bounce message and attempt to send it back to the (usually faked) sender. Gah!

As a first step, I decrease the queue lifetime to 24 hours. After 24 hours, any message still left undelivered in the queue would now be deleted. Then, I cleaned out the queue and restarted qmail. It worked!

...for a while. Ok, so let's do some more furious reading on the subject of spam prevention in the qmail environment. First of all, I needed to increase the maximum number of qmail-smtpd processes which were allowed to run simultaneously by tcpserver. By adding a "-c 400" to the tcpserver command used to execute qmail-smtpd, I increased the maximum number of simultaneous connections from 40 to 400. That ought to be a good start.

Now, how else can I decrease the volume of spam? I had heard of greylisting before, and thought that it might be a good next step. Luckily, there is a simple greylist implementation for qmail called qgreylist. It installs fairly quickly, and does a fairly good job. However, it is written in perl and with all of the spam emails flowing in, it can cause a bit of a performance hit.

So, maybe there is something another check I could do before sending the incoming email to qgreylist? How about an RBL check! When another computer attempts to deliver an email to our mail server, the RBL (Real-time Blackhole List) check sees if the sender's IP address in a list of know spam servers. If it is, then the connection is dropped and the email never even is seen by the greylist app.

With the above changes made to our mail server, I am now receiving about 10% of the spam volume that I was before the changes were made. Furthermore, our remote queue sits at a fairly steady level of 400 messages.

Only time will tell for how long this solutions will suffice! Wish me good luck and my email godspeed!


  1. RBLs are a godsend. I love the people who run respectable ones.

    Another possibility for checking before or in addition to your current anti-spam filters is setting up a lookup to a URIBL, which means checking message bodies for links to blacklisted sites such as porn sites, phishing, etc. Of course, this means a further hit to server performance, but it might end up being a net performance gain.

  2. URIBL could be done by SpamAssassin, which could replace DSpam.