Mail

I work (and have worked) on many E-mail related projects, including the anti-spam systems used at Yahoo. I've written a few pages on various email topics here.

Checking your IP or domain's status

This is a small collection of URLs and sites you can use to see if your IP or domain is considered "spammish" or not. This is not a complete list, but I'll update it as I find more useful tools.

Senderbase

This is an IP clearinghouse run by IronPort, and used by many commercial anti-spam systems (like, Barracuda). You can check your IP or domain status here.

RBL checks

There are plenty of RBL systems out there, and plenty of tools to check your status. Here are a few:

Hacking: 

Anti-Spam and email tools

This is a short list of tools I use to manage my mail servers (and mailboxes). The main building blocks for my system are
* [http://www.ogre.com|sendmail] for the MTA, with support for [http://www.milter.org|Milters]
* [http://asg.web.cmu.edu/cyrus/|Cyrus IMAP] server for IMAP over SSL
* Server side mail filters using [http://asg.web.cmu.edu/cyrus/sieve|Sieve]
* Webmail access using [http://www.squirrelmail.org/|SquirrelMail]
!!! sendmail
I'm running sendmail v8.13.1 currently, supporting a number of domains and users. sendmail uses the Cyrus mailer for delivery, which in turns talks to an [http://www.faqs.org/rfcs/rfc2033.html|LMTP] server. This injects the new messages directly into the IMAP folders. As an extra benefit, I also get the __+__ style addressing support automatically, meaning I can sort into sub-folders by the envelope address. For instance, if you send an email to

{CODE()}
leif+other.spam@ogre.com
{CODE}

it is automatically sorted into my IMAP folder named __other.spam__.
!!! Cyrus IMAP
Cyrus has been my choise of IMAP server for quite a while, way back when there was really no comparison in features and performance. Netscape Mail server was very promising, but unfortunately died during the AOL and Sun massacres. Cyrus supports SASL of course, but also a somewhat unknown sub-system called [http://asg.web.cmu.edu/cyrus/sieve|Sieve]. This is a 100% server side mail filtering language, and it's quite powerful, much more so than I can document here. But, here's an example filter file:

{CODE()}
require "fileinto";

if header :is "X-Spam-Flag" ["YES"] {
fileinto "INBOX.zSpam";
} elsif address :is :all ["from", "to", "cc", "bcc"] ["foo@ogre.com", "bar@ogre.com"] {
fileinto "INBOX.filtered.giants";
} elsif address :is :all ["to", "cc", "bcc"] ["dist-update@perldap.org"] {
fileinto "INBOX.filtered.dist";
} elsif header :contains "List-Id" "some_mail_list" {
fileinto "INBOX.mailists.games";
}
{CODE}

!!! Spam tools
I use a few sendmail milters to help get rid some of the worst spam:
* [http://www.mimedefang.org/|MIMEDefang]
* Snert's [http://www.milter.info/milter-spamc/index.shtml|milter-spamc]

I'm fully aware that MIMEdefang supports [http://www.spamassassin.org|SpamAsassasin], but I prefer to run these two systems independently. In fact, I wish MIMEdefang would make it easier to disable the user of SA! For SpamAssassin I use pretty much every bell and whistle there is, including:
* DCC
* Pyzor
* Razor

!!! Anti-virus
Although MIMEDefang does some AV already, it also supports "pluggable" virus detectors. I currently use
* [http://www.clamav.net/|ClamAV]

Hacking: 

sendmail, SASL, TLS and SSL

{maketoc}
! Project goal
When I started this project, my primary goal was to allow for authenticated SMTP over a secure channel. I use [http://www.sendmail.org/|sendmail] as my MTA, not because I'm particularly fond of it, but because it's what I know, and it works well for me. My setup is pretty straight forward:
* ))RedHat(( linux (mix of 8.0 and 9.0)
* sendmail 8.12.10
* Latest OpenSSL and SASL libraries (I gave up on using RHs outdated builds)
* A myriad of milters, spam filters, and tools around sendmail and Cyrus IMAP.

I'm not going to go into details on the exact mail server configurations, but I'm concentrating on the changes I made to enable SSL and SASL for sendmail.
! Clients and TLS vs SSL-only
We mostly use well behaved mail clients for MUAs, but my wife insists on using Entourage on her Mac. To her defense, there's not a lot of alternatives out there for Macs, Mozilla isn't working very well on her system. Being a Microsoft client, Entourage of course doesn't follow the specifications. In particular, it doesn't support the __STARTTLS__ command. The client would simply complain that the server didn't provided any SASL authentication method that it could use:

{CODE()}
The SMTP server for "Ogre" does not recognize any of the authentication
methods supported by Entourage. To send mail, try disabling SMTP
authentication in the account settings or talk to your administrator.

An unknown error (5530) occurred
{CODE}

The lack of proper SASL support in Entourage forced me to also include support for a dedicated SSL/TLS SMTP server (__SMTPS__). Fortunately, sendmail allows me to serve both the regular __SMTP__ port (25) and __SMTPS__ (port 465) using the same configuration.

As if this wasn't enough, once I got past this problem, Entourage misinterpreted the sendmail greetings as if client authentication (certificate) was required. I would get a client error like:

{CODE()}
Security failure. Personal certificate required.
{CODE}

Fortunately sendmail provides a configuration solution for this as well, naturally. But seriously, why does MS have to make my life miserable all the time? I don't even hate them, I just prefer not to use most of their junk...

! Rebuilding sendmail with SASL support
This was the easy part, simply modify your __devtools/Site/site.config.m4__ file to something like:

{CODE()}
APPENDDEF(`confENVDEF', `-DSASL=2 -DSTARTTLS')
APPENDDEF(`conf_sendmail_LIBS', `-lsasl2 -lssl -lcrypto')
APPENDDEF(`confLIBDIRS', `-L/usr/local/lib -R/usr/local/lib')
APPENDDEF(`confINCDIRS', `-I/usr/local/include')

APPENDDEF(`conf_sendmail_ENVDEF', `-D_FFR_SMTP_SSL')
{CODE}

You'll obviously have to change your directories etc. to match your build directory. The last configuration option enables an (experimental?) support for the SSL-only daemon support. I'm not 100% sure, but I think support for __SMTPS__ is enabled by default in __sendmail__ 8.13. In any case, this feature is important to work around the lack of proper TLS support in Entourage (which our new sendmail daemon will announce).

Make sure to save a copy of your old sendmail binary before installing this new build. Nothing worse that not being able to do a quick rollback on your oh so important mail servers.

! SSL certificates
I'm not going to go into detail on how to generate the appropriate SSL certificates here, there are plenty of sites that describes that. For example:
* [http://sapiens.wustl.edu/~sysmain/info/openssl/|OpenSSL HowTo]
* [http://www.pseudonym.org/ssl/ssl_cook.html|OpenSSL cookbook]

I personally just use my own root-CA, so I can avoid the Verisign tax. And this is plenty secure for all my needs.

! Reconfigure sendmail
First of all, we need to configure sendmail to find all the required certificate information. Assuming you are using the M4 macro packages for configuring sendmail, add something like this to your __sendmail.mc__:

{CODE()}
define(`confCACERT_PATH',`/usr/local/SSL/private')dnl
define(`confCACERT',`/usr/local/SSL/private/CAcert.pem')dnl
define(`confSERVER_CERT',`/usr/local/SSL/certs/sendmail-cert.pem')dnl
define(`confSERVER_KEY',`/usr/local/SSL/certs/sendmail-key.pem')dnl
{CODE}

Obviously, you must change the paths here. Next, we need to configure SASL properly, and here's the first stumbling block trying to support Entourage. As far as I can tell, you ''must'' have support for the __LOGIN__ authentication mechanism enabled. This might require a recompile of your SASL libraries, since it's an optional feature. My __sendmail.mc__ adds the following lines for SASL:

{CODE()}
define(`confAUTH_MECHANISMS',`LOGIN PLAIN')dnl
TRUST_AUTH_MECH(`LOGIN PLAIN)dnl
define(`confDONT_BLAME_SENDMAIL', `GroupReadableSASLDBFile')dnl
{CODE}

I only support "plain" SASL authentication, but other authentication mechanisms includes CRAM-MD5 and DIGEST-MD5. Using such mechanism requires you to setup and manage the SASL authentication database properly. The last configuration eliminates a sendmail runtime warning, if your SASL DB is group readable (common, to give sendmail read permissions?).

Entourage still won't talk to this sendmail configuration, because of it's broken STARTTLS implementation. The only workaround I know of is to enable the SSL-only port/daemon. This is done with a few more configuration lines to your __sendmail.mc__:

{CODE()}
DAEMON_OPTIONS(`Port=smtp, Name=MTA')dnl
DAEMON_OPTIONS(`Port=465, Name=SSA, M=Eas')dnl
define(`confAUTH_OPTIONS', `A,p,y')dnl
{CODE}

The last option isn't absolutely necessary, but it prevents people from accidentally trying to use SMTP Authentication over a non-secure channel. It also denies anonymous logins. Alright, we're almost there. Now there's only one last problem, Entourage still complains about the requirement for certificate based authentication. Once again, sendmail provides configurations options for this:

{CODE()}
define(`confTLS_SRV_OPTIONS',`V')dnl
{CODE}

!Conclusions
If it hadn't been for Entourage (and I'm guessing, Outlook), enabling SASL and TLS for sendmail is pretty straight forward. I had it up and running for most of my clients in less than an hour, but figuring out all the tweaks for the rouge MUAs added several more hours of debugging and reconfigurations.

Is it worthwhile doing this? Absolutely! With SASL enabled (over TLS hopefully), your users can send email through your system from anywhere. One major benefit is that an authentication SMTP sessions is allowed to "gateway" mail just as if the client came from one of your blessed (internal) networks. And spammers can not abuse is, unless they can compromise your authentication mechanisms (i.e. hacking accounts).

!Credits
Many thanks to my wife Michelle for using Entourage, and forcing me into fixing all this crud ... :-)

-- [mailto:leif@ogre.com?Subject=Re: ABIT IC7 audio for Linux|Leif Hedstrom]

Hacking: