Canned Meat

Posted by Roel Gloudemans on 7 February 2009 | 0 Comments

Tags:

Preventing Spam e-mails is so easy and yet, because a seemingly substantial amount server admins refuse to read up on the standards, so very hard. For instance, when you try to implement all the measures at http://calum.org/posts/postfix-antispam, you'll find that a substantial amount of legitimate mail is refused by your mailserver. When I tried here in the Netherlands, almost 25% of all mail was rejected. I am talking about a seemingly amount of admins, because often it is not the admin who is to blame, but the ISP.

For instance, a mailserver should have a forward and a reverse DNS lookup defined, as required per RFC1912. Next, a mailserver should HELO/EHLO (mailserver handshake) with a FQDN that is defined in DNS

Implementing these rules alone will mean that many legitimate mails will not be accepted by your mailserver. All legitimate servers have the forward DNS lookup defined, but the reverse lookup if often missing or not matching the server name. This is almost always true for consumer type ADSL lines and also for many corporate SDSL ones. This is a flaw on the side of ISPs.

Problems with the second rule are also common. Many mailservers have a different name on the internal network than on the Internet. This is not a problem, because with most mailservers you can configure the name the server uses in the handshake. In practice, many admins forget this.

If these criteria cannot be used to fight Spam and Malware what measures are left? Plenty. Many of these are easy to implement. A how to implement in Postfix in included with each point.

Use blacklists. There are many sources for good blacklists out there. A well known one is Spamhouse (www.spamhouse.org) another one is njabl (www.njabl.org). These list work using DNS lookups and are realtime. These lists can be applied in two ways; tag the incoming message as Spam when the sending server is on the blacklist or reject the mail altogether. I am using the last option now for a couple of years and I am quite happy with it. It keeps the load on the server down and hardly ever creates false positives.

Use SPF (or DKIM). Sender Policy Framework (www.openspf.org) can be a good measure to not only prevent Spam from going in, but it also prevents others from using your domain name for Spamming purposes. DNS already contains a record for each server dat receives mail on behalf of your domain. An SPF record defines which servers are authorized to send mail on behalf of your domain.

Use Greylisting (www.greylisting.org). This measure is not without controversy. Greylisting looks at the sender and receiver addresses as well as the sending server. If this combination is new, the server will return a try again later message. A legitimate mailserver will keep retrying until the mail is delivered. The time after which the mail is accepted is configurable. Spam servers will give up, or will be identified by a blacklist when trying again. The next time a mail cones in with that same recipient, sender and server the mail is accepted immediately. Greylisting can cause problems. Some legitimate servers will not retry and mail is getting delayed, so users should be aware of this mechanism.

Only accept mail that can be delivered. This sounds trivial, but many servers will accept mail to any username on a given domain. If the mail cannot be delivered to an end-user the server will send an undeliverable message back. This adds to the load of the mailserver and, more importantly, Spammers are using this mechanism to get Spam into your mailbox. Send a Spam message with the intended recipient as sender to a non-existing user on a legitimate server. That server will send a non-delivery report, with the original message to the intended recipient. This is called backscatter Spam. Implementing this point is harder than it looks. It is easy enough for an all in one mailserver, but when a mail gateway, backup mail gateway or a Spam/virus scanning appliance is used, this is often not implemented if at all possible. Also, do be careful when implementing. All my users are in a LDAP directory. The MTA can talk to the LDAP server directly, but what happens when the LDAP server is down? Right, all mail gets rejected. That is why I use a script to generate a recipient list a couple of times per day.

Use the right tool for the trade and read up. Not an experienced admin? No problem, but then you should stay away from an Exchange server, because you can configure it without knowing what you are doing, but stay away from Sendmail as well because it is too complex. Rather start with a simpler MTA like Postfix or Courier.

RHEL/CentOS use sendmail by default. Executing

You can change the default from sendmail to postfix. Suse systems use postfix by default

These measures all sound very fine, but if you are in the circumstance that your ISP requires you to send mail via their gateway, you are out of luck. Number 1 and 3 won't work, since mail is coming from the gateway always and 2 cannot work for inbound mail. It could work for outbound mail, but when I asked my previous provider (KPN) for a list of gateway addresses they told me it was secret. I switched providers immediately.

Now lets look to my setup, or at least, how it was until a couple of weeks ago. My mail solution comprises of 2 servers. One is a Zimbra installation (open source edition) and one acts as filtering MTA. I chose this setup, because Zimbra is rather intolerant of other software on the server and configurations done outside of the Zimbra GUI or command line interface is bound to get lost on upgrades. Zimbra includes Spam and virus scanning and is doing a good job, but I wanted more.

That's why I added a separate MTA, in which all of the above rules are implemented. On top of the measures in Postfix I also added MailScanner to the solution because I wanted:

1. have a second virusscanner
   
2. prevent problems because of HTML mails which redirect users to malicious sites
   
3. blacklist some file types (can be done in Zimbra as well)
   
4. MailScanner can be used to convert incoming Word documents to text. Ideal if you want to have a quick scan of the included document.
   

This setup worked really well for most of the time, but I had some problems with it as well:

1. Mail that is forwarded from other mail accounts, or fetched using fetchmail bypasses all checks in Postfix. There is still a double check on virusses, but enabling SpamAssassin (http://www.spamassassin.org) is pointless, since Zimbra is using SpamAssassin as well. Performance of SpamAssassin is still impressive though. Only 5% of all Spam in forwarded mail was let through, which translates to 4 to 5 Spam mails in my personal box per day.
   
2. MailScanner will automatically try to update virus and phishing definition libraries, but does not detect of the update was successful. Because of this the MTA went without virus definition updates for a month.
   
3. MailScanner has a few conflicts with other packages on the system. This causes problems when those other packages need to be patched. (The commercial version of MailScanner doesn't have this problem)
   
4. MailScanner is relatively slow. For instance, if you want MailScanner to scan for viruses, you can use it with almost every scanner out there. Sadly, only the command line versions of those scanners are free for non-commercial use, except ClamAV (www.clamav.com). But since Zimbra is already using ClamAV, it is pointless to use it with MailScanner as well. The result is, that for each incoming mail the virus scanner needs to be started. This causes quite a load.
   

In December I got an invitation from the People at BitDefender to partake in the beta of the next version of the Unix desktop virus scanner product. Out of curiosity I downloaded the beta. Before I knew it I was ranked as the top tester and god rewarded with a BitDefender product of choice for 35 users with a one year license. After sending a mail back with the message that I was not representing a company and that a one-year license was not of too much use, they kindly granted me a three year license for the same.

Last september I became a dad. This is a wonderful experience, but tends to diminish the time you have available to look after your server. From a commercial product I expect an emphasis on Total Cost of Ownership/Return On Investment (choose your poison ;-) ). In my mind this translates not only to a good hit rate on viruses and spam, but also to low effort for administration. This together with the 4 points mentioned earlier lead me to replace MailScanner with BitDefender for Unix Mailservers. I had to sacrifice some functionality like the conversion of word documents but I would gain:

1. A second sophisticated Spam scanner
   
2. Guaranteed updates of the virus scanner plus alerts
   
3. A slick interface (GUI and command line) to configure BitDefender
   
4. A better maintained server with less effort
   

BitDefender works a bit different from MailScanner. Where MailScanner picked up queue files, BitDefender works as a SMTP proxy. Since the earlier mentioned anti-spam measures are not implemented in BitDefender (except real-time blacklists, but only as tag as Spam only), I need a Postfix in front of BitDefender and one after to deliver the mail. The BitDefender installation process configures all this automatically. Starting from a properly configured Postfix, the installation an configuration only took a few minutes.

The setup is not optimal in terms of performance, but compared to the old MailScanner setup it is better. Performance can be enhanced by following the instructions in the Postfix manual (http://www.postfix.org/FILTER_README.html).

The setup has been working flawlessly ever since. One patch came out after installation. I was alerted automatically and patch installation cost me 2 minutes. This is much less compared to the 30 minutes I had to spend at least for each MailScanner update.

As for the Spam volume that is getting through? I'm now down to one Spam mail per day. I estimate this is less than 1% of the total Spam volume. I had to set BitDefender to a medium aggressive setting. And no false positives so far. The fact that I have nothing more to report on BitDefender shows it strength; it just works.