Weekly Edition
Return to the Development page
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
A comparison of Mail Transfer Agents - Part Two
[ Editor's note: Mr. Shearer is maintaining an updated version of this article on his web site.]
In part one of this article series, we covered the criterion for selecting a Mail Transfer Agent (MTA), and took a detailed look at Postfix and qmail. This week, we take a look at Sendmail and Exim, and come to some conclusions about which MTA is best. Sendmail
Design goals: Current Sendmail must be backwards-compatible, and the forthcoming Sendmail X is a total rewrite. Sendmail consists of about 118k lines of code, but that does not count the functionality in the M4 scripts used to generate the config file, nor any external milters. Documentation is good, and uniquely among MTAs, there is a dominant company (Sendmail, Inc.) dedicated to Sendmail services. The Sendmail Consortium is dedicated to maintaining the Sendmail code base. Sendmail has an extraordinarily obscure configuration file, a poor history of security breaches and a design centered around Unix in the early 1980s. It is a fact that hundreds of thousands of Sendmail sites are currently advertising themselves as having remotely exploitable security vulnerabilities. Add to this sendmail's renowned inefficiency and it might be hard to see why Sendmail is still used at all, but history has its own inertia. There is no good reason for a site without Sendmail experience to install it, given the effectiveness of the alternatives. Despite all this, Sendmail:
Although there are no recent surveys, Sendmail usage appears to be dropping over time. Dan Bernstein's 2001 SMTP survey (without published source code, and therefore not replicable) put Sendmail at about 42% market share. In 2006 it seems reasonable to assume [4] that Sendmail is on substantially fewer than 40% of the world's SMTP servers. Sendmail has been ported to many systems, including some that are not Unix-like such as Windows. Postfix isn't realistically portable to Windows, and Exim is something of a second-class citizen on Windows since it runs via Cygwin. So portability might be a reason to run Sendmail. Exim
Design goal: General-purpose MTA for Unix machines. Exim was inspired by the author's work with the smail 3 source code, which was itself provoked by the many problems of sendmail. So Exim too is a Sendmail drop-in replacement. The outstanding feature of Exim is the intention that it be a general-purpose mailer. Exim is not a total rethink about how mail works, like qmail is. Nor does it restrict its feature set in order to achieve theoretical security, like Postfix. Exim instead tries to give administrators what they asked for, with a strong interest in security, reliability and performance. Exim behaves much like any other Unix daemon, with a monolithic configuration file, a monolithic daemon, small number of log files and a standard style of spooling. It has a very good security record over the last seven years (early releases had classic security issues), it can cope with high load, and it has excellent integration facilities. Exim can be extended in many ways - it is even possible to compile in the entire Perl interpreter to call from the configuration file! If there is an MTA feature, then Exim can support that feature in some way or another. Exim is very tightly specified and documented. Many features can be omitted at compile-time, making a special-purpose Exim easy to create. Exim has its own filter language, implementing much of the functionality of procmail, and more. Exim is used at some very high-volume sites where it provides good service. Performance comparisons that say qmail and Postfix are faster and handle queuing better don't necessarily have any bearing on real-world conditions (in 2006 on current hardware and with current definitions of high load.) Open Source at WorkOne of the interesting things about the three non-Sendmail MTAs here is the ideas and code that are shared. Postfix uses the Perl Compatible Regular Expressions library developed for Exim. Exim understands the Constant Database Format developed for qmail, and the Maildir mail file format, also from qmail. Postfix can use the Constant Database Format and Sendmail milters. When Local Security Isn't a ProblemThe main reason why MTAs have to work so hard at security is because of the Unix tradition of local delivery. The mixture of setuid binaries, specially-owned directories, pedantic authentication of local destinations and paranoia over filesystem access all has to do with having the MTA write to a file owned by some other user, usually by becoming that user. Of course that is fraught with danger. No matter how well the code is written, a careless administrator can still make it behave in an unsafe manner. But in millions of sites this is no longer an issue because mail is kept in a central IMAP mailstore until the user chooses to view it. Mail comes into the SMTP daemon, which then makes an LMTP delivery to the IMAP daemon. In this scenario, local deliveries are completely avoided. It is possible to compile at least two of these mailers so that none of the potentially dangerous code is even in the mailer. Here's how it is done with Exim: All routers, directors, and transports are compiled only when specified in the Local/Makefile. You can compile Exim with only the SMTP transport - and make that use LMTP to address 127.0.0.1 for "local" delivery. Then you can run Exim entirely in "unprivileged" mode, where it runs as user exim the entire time, except during startup of the listening daemon. Usability comparisonThe following table compares the above MTAs for usability:
The quick answerMy recommendation for an MTA choice is Exim, here's why: Exim can solve any MTA problem at least as well, if not better than any of the other MTAs listed here. It has very good documentation and a most supportive community. It is the only modern mailer which expressly aims to be general-purpose. That is why it is my first choice. There are no ordinary circumstances where Exim is a bad choice, although there may be special circumstances where another MTA may be superior. Think of Exim as the Linux of free MTAs. There are many free Operating Systems and some of them are better than Linux for specific tasks. But Linux can do (at least) a good job for nearly everyone [5]. Some Home Truths
Footnotes4. I'm working on doing a survey of my own. Let me know if you want to help. 5. Which doesn't stop me learning from the others -- thank you NetBSD for ISBN 0-201-79940-5 and ISBN 0-321-16607-8. More articles by Dan Shearer are available here. (Log in to post comments)
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 4:19 UTC (Thu) by RedHatOpus (guest, #4260) [Link] I'm a bit incredulous here:
exim = 15 points, postfix = 17 (from the chart)
If you're going to do a shameless plug for your favorite product. Either doctor your stats or come up with some real reasons to recommend one over the other.
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 6:59 UTC (Thu) by dvrabel (guest, #9500) [Link] Surprisingly, there's more to a review than a single number. Exim is recommended over Postfix as it's general purpose: "[Exim] is the only modern mailer which expressly aims to be general-purpose."
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 4:45 UTC (Thu) by kimoto (subscriber, #5244) [Link] Sendmail gets a "0" suitability score for those who are "relying on sendmail milters". If this is right (and I don't know, as I am not a sendmail person), it cries out for some kind of explanation.
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 12:06 UTC (Thu) by rise (guest, #5045) [Link] It's wrong. Milter was developed for sendmail and is still the primary target, by definition it has to be at least somewhat suitable and I doubt it's less suitable than Postfix (my preferred MTA).
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 5:49 UTC (Thu) by einstein (subscriber, #2052) [Link] We used sendmail for a long time, but had been keeping an eye open for alternatives, so as to have options. qmail was too weird and gratuitously different, and had some odd limitations which made it impractical for our environment.
OTOH postfix was a drop-in sendmail replacment, but with *much* higher performance than sendmail, and it turns out to be the default MTA in suse linux, while we were migrating all our infrastructure from redhat to suse. We've since discovered there are nice postfix packages for solaris and other unices as well.
We've found that doing spam/virus filtering and weird procmail tricks is really easy and convenient with postfix, so it has my vote.
As far as having a windoze version, that is of absolutely no importance to us, no idea why it was even mentioned. In fact, I think I'd give more points to a unix MTA that just strives to be the best, and doesn't waste cycles jumping through hoops, trying to produce a windoze version. We are a unix shop, you know. YMMV.
size-constrained? Posted Aug 31, 2006 6:13 UTC (Thu) by dwmw2 (subscriber, #2063) [Link] Could you justify the scoring in the 'size-constrained' row of the table? I'm not sure about qmail, but judging by the package sizes in Fedora Core 5 (PowerPC), the scoring seems to be backwards -- you seem to give more points for being bigger:Sendmail: 1521746 Exim: 3640124 Postfix: 11521454That doesn't take into account the fact that Exim includes features out of the box which can't be matched by the others unless you add extra software. Greylisting, for example, is fairly simple to do in Exim's configuration language alone -- but the other MTAs will require you to install separate software to do it. I don't think much weight should be given to either the 'run sendmail milters' or 'runs on Windows' rows. However, your table should probably include a row for 'overall capabilities', including such things as greylisting, integration with SpamAssassin/ClamAV/etc., mail filtering, ...
Strange Conclusion Posted Aug 31, 2006 6:22 UTC (Thu) by ctg (subscriber, #3459) [Link] From reading the article, particularly the scoring system; I found the conclusion to be a bit counter intuitive.
For what it is worth:
[I've used sendmail, postfix and qmail extensively. I no longer use sendmail, but still use postfix/qmail depending on the application.]
qmail - qmail isn't that hard to set up and manage. It is a bit different, but the are plenty of resources to help you get going, plus some simple installation scripts. I would give it a 2 or 3 on that score.
qmail's downfall is also its strength. It is very very simple, and almost simplistic. It can't do all the magic things you might need to do - but its simplicity makes it good from a security point of view - simple set of well defined functions, nicely broken down into separate processes. My "use case" for qmail is a front line email server, attached to the internet, doing simple relaying - into which I add AV and anti-spam. My qmail setups always relay on to another machine on the internal network.
For this mode of use the qmail install/config is completely trivial - the only changes to be made is a one line entry to the file which contains the list of "Known" domains, and the one which tells which internal server to route those domains to. In use qmail just gets on with it.
sendmail - sendmail is really hard to manage - unless you stick to cookbook examples - working with the m4 files is hard enough, but to directly manipulate the .cf file is something else.
postfix - postfix has a rich set up - and is very comprehensive, so you have to study the manual too. I think a 2 or 3 for postfix - certainly it is a bit more complex to maintain than qmail - which becomes trivial after it has been installed.
I use postfix were qmail doesn't have the necessary features - typically on an internal network, in conjunction with cyrus-imapd - at the point where the email system needs managing (defining aliases, routing rules etc, controlling the queue).
I've looked at exim, but never seen a compelling reason to switch from postfix.
Conclusion
Minor corrections... Posted Aug 31, 2006 7:38 UTC (Thu) by nigelm (subscriber, #622) [Link] Damn exim got there fast as a sendmail replacement... or maybe it didn't start in 1982 after all. [1995/6 would be a better timeline - in fact all of the sendmail replacements here started within 2 years of each other].
Personally I would ignore qmail due to licensing/maintenance issues. Postfix and exim are both good systems. I happen to have worked on exim for a long time including serious multi-million user installations, so know it better than Postfix, so I am biased by what I know.
If you want to work on personalities, both Wietse and Philip are great people - courteous, helpful and willing to give good explanations as to how and why. Unfortunately that does not hold for some of the other systems.
My experiences of sendmail are biased by sendmail 5.x
sendmail easier than qmail? Posted Aug 31, 2006 9:32 UTC (Thu) by pkolloch (subscriber, #21709) [Link] Alright, this was years ago, when I was about to administer my first MTA server, it was setup to run sendmail. I spent a considerable amount of time to understand the elaborate configuration, but was unable to understand it fully. I decided to convert to qmail. It took some time to get used to all its principles, yet after the effort I felt that I understood the configuration completely. This is a very good feeling I have not attained with any other MTA in a reasonable time frame. Since I was a complete newbie to MTA configuration, I would argue that qmail is considerably easier to setup. That said the license restrictions of qmail make it an odd choice.
qmail's *too* simple Posted Aug 31, 2006 17:16 UTC (Thu) by smurf (subscriber, #17840) [Link] Sure, qmail is very easy to understand, but that's partly because it just can't do most of the stuff I'd want to have in a reasonable mailer. (And no, adding the exact same line to 1000+ .qmail-FOO files doesn't help WRT "reasonable".)
I happen to like Exim's configuration syntax. It's reasonably clear, has sensible defaults, the documentation is understandable and seems to be complete, and it has a filter language with a rich-enough syntax that actually makes sense (as opposed to qmail's, which doesn't really do anything except distinguish program exit codes -- or things like procmail, which are frankly dangerous).
The only serious gripe I have about Exim is that AFAIK it *still* cannot encapsulate bounce emails in MIME. If you get a bounce from an Exim mailer because you mistyped an address, extracting the thing and re-sending it is a nontrivial exercise. With MIME and a MIME-enabled mailer, this becomes child's play by comparison -- at least with mutt. ;-)
I don't know how the other mailers handle this.
Courier-MTA Posted Aug 31, 2006 14:50 UTC (Thu) by markc (guest, #4419) [Link] Here we go, the article missed *my* favourite MTA... courier-mta, indeedthe whole courier suite which is very complete. I didn't even bother reading either of the 2 articles because I know how much *I* prefer courier so therefor reading anything about "the others" is a waste of my time. I post this comment in case anyone cruising the issue of what MTA to use happens upon a mention of courier and bothers to check it out (at courier-mta.org). A nano-review of good points would go something like; a complete (as in mta, pop/imap, webmail, a barely usable list manager, great filtering options) that is bound together by a unified auth system of various backends types. I use MySQL for virtual users and the same DB table gets used by RADIUS and FTP servers as well. It uses Maildir format for storing messages, like qmail, and is a far better qmail than qmail could ever be as it is at least well maintained. I know I am conceited and bigotted but I am not sure why anyone would bother using anything else.
The Milter line Posted Aug 31, 2006 16:22 UTC (Thu) by dskoll (subscriber, #1630) [Link] The "Relying on Sendmail Milter" line gives Sendmail a 0, Exim a 1 and Postfix a 3??? That's complete and utter rubbish.
Sendmail clearly has the best Milter support and should get a 3. Postfix 2.3 has incomplete Milter support, so it should only get a 1 or 2, though I am keeping my eye on it for the day when Postfix completely supports Milter.
If Exim got a high score because it can do "equivalent" things to a Milter, then the author hasn't read the Milter API. It's far and away the best e-mail filtering API for any of the free MTAs, and possibly for any MTA.
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 17:53 UTC (Thu) by malor (subscriber, #2973) [Link] I appreciate the effort, but this really isn't a very good article. As pointed out above, you give SENDMAIL low points for working well with Milters. Um, hello?
You also score Postfix as essentially perfect for all situations except Windows and small systems, yet Exim, which doesn't score as well, is preferred. And the reason for that is *not specified*... you simply say that Exim does everything and Postfix doesn't. You spend all this time reviewing mailers, but when it actually comes down to why we should choose one over the other, the cited reason is nebulous and unspecified. Sheesh!
In my years with Postfix, I have found only one situation where it didn't cope well... at one time, it didn't speak Maildir, so I had to tie it into maildrop to support those. (I think that's been added since.) It's very fast, very configurable, and *incredibly* robust. NASA's rovers crash more often. And in all its history, I remember seeing only one security bug.
For my uses, I'd rather trade off a little bit of unspecified functionality to get better security. I'm not belittling Exim, as I don't know it well enough to judge it, but I can certainly say that Postfix is grade AAA, and one of the single best pieces of free software.
Are there really a lot of vulnerable sendmail systems? Posted Aug 31, 2006 18:40 UTC (Thu) by smoogen (subscriber, #97) [Link] We went through 2 security audits where all our sendmail systems were shown to be running vulnerable versions.. the reason was that the vendor backported fixes to that version of sendmail and the vulnerability scanner only used a very simple regex to figure out what sendmail was running. Talking to other system administrators this seemed to be a common occurence, and from the head auditor it seemed that they had lots of sites giving in variance reports saying "Hey we are running a backported version.. update your scanning software."
This is not a recommendation for running sendmail. While I like the software having written rules since 1991 or so.. I also realize that I know enough about it to make sendmail perform better than an out of the box postfix or exim.. which of course makes me a very biased opinion. But I do think that reports about how many vulnerable systems out there need to be looked at a bit more sceptically.
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 18:59 UTC (Thu) by acorliss (guest, #3710) [Link] The sheer incompetence of these articles are staggering. In fact, your final "matrix" seems to be more emotionally influenced than factually based. How about a row comparing scalability (and provide some actual benchmarks and metrics)? How including scalability in context of mail routing & filtering configuration complexitity?
Giving sendmail a zero for "*sendmail* milter" support is assinine. Giving it a zero for "worried about security" is ludicrous in context of the last several years -- it only makes sense if you are comparing complete histories, and even that's not entirely fair given how much longer a history sendmail than all the others. They learnt all the hard lessons for everyone else.
You gave sendmail a zero on "wanting minimum hassle" even though your own text states that "sendmai works with little or no modification to the default settings"! For that matter, shouldn't that also affect the score for inexperience users?!
Size constraints? Have you looked at the compiled footprint of the other daemons?
Yes, I am a sendmail user. I know it has warts, but I also know I've evaluated several other alternatives and found none of them to have the flexibility and scalability for complex ISP & Enterprise environments.
Your scoring system is inconsistent with even the facts you presented and is utter rubbish.
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 20:00 UTC (Thu) by dmantione (guest, #4640) [Link] Exactly. I've been maintaining systems with Sendmail, Postfix and Exim,and after time I slowly came to the conclusion that Sendmail still suits best. As a Sendmail user I'm the first to agree its configuration file is spartan. But on the other hand, configuration is a matter of commenting and uncommenting the right lines in the m4 file, and therefore quite easy. Now look at master.cf from Postfix. Who understands this? Now you can argue that for a simple setup you don't need to touch master.cf, but you do need it if you want to integrate it with a virus scanner for example, which is often requested these days. Sendmail? Copy/paste one line from the milter documentation to the m4 file. The biggest power of Sendmail is IMHO that it is adaptive; its functionality can be tweaked/enhanced by simple m4 files anoyone can post on its website, while other MTA's need new releases for such things.
A comparison of Mail Transfer Agents - Part Two Posted Aug 31, 2006 20:17 UTC (Thu) by cdmiller (subscriber, #2813) [Link] I feel compelled to repeat my response to:http://lwn.net/Articles/85493/
At this time the first line should read 8 years rather than 6.
Posted May 27, 2004 17:01 UTC (Thu) by subscriber cdmiller [Link]
We have used qmail for about 6 years now. It's rock solid, secure, and easy to configure. We have 16 lines of configuration in the 7 files of our qmail/control directory on the main mail server. My default exim.conf for a debian/GNU workstation has 200 lines of configuration. The postfix setup on one of our redhat servers has 76 active config lines in main.cf.
It's interesting to watch something like postfix or exim evolve alongside a fairly small and static product like qmail. The free software products in this case appear to become more complex and feature ladden over time.
My organization can't redistribute modified versions of qmail, but we don't need to. We can patch our version to our hearts content if need be, and patches are merrily produced and distributed by the community.
It would be interesting to see what the community would do with a free version of qmail. Maybe DJ Bernstein will be convinced to allow a free software fork of qmail under a different name one of these years. Even then, qmail will be hard to beat for it's ongoing record of stability, security, and ease of configuration.
sendmail is a security nightmare, for versions <=7 Posted Sep 1, 2006 1:03 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link] since the 8.x release sendmail has been pretty good from a security point of view, to the extent that the old saw about the biggest target having the most reports has some validity.
it does suffer a little from supporting every OS under the sun (and the complications that are nessasary to do this)
sendmail also has a ton of features that most people don't need, but when you need them they are relativly easy to turn on.
sendmail has three different ways of configuring it
from easiest to hardest they are
1. the Sendmail INC GUI, (interprets and creates .m4 files)
2. manipulating m4 files (get compiled into .cf files)
3. manipulating .cf files
saying that sendmail is hard to administer becouse .cf files are bad is like saying that developing a compiled python program is hard becouse the resulting binary is hard to understand and change (here the three levels are python->C->machine code) only experts who are doing really strange things should ever need to manipulate the .cf files
this hasn't always been the case, back in the sendmail 5 days people were expected to work with the .cf files directly. Frankly I'm glad I'm young enough to have missed those days :-)
A comparison of Mail Transfer Agents - Part Two Posted Sep 1, 2006 8:18 UTC (Fri) by pilif (guest, #3857) [Link] Hi there,
woah! I'd never expected this outcome.
While for me, Exim was the favourite all along (see my comment on last weeks article http://
For me and obviously for the reviewer, these points were less important than the obvious
Still... I'm somewhat surprised to see the reviewer agree with me - usually my opinions are not the
Philip
A comparison of Mail Transfer Agents - Part Two Posted Sep 1, 2006 10:58 UTC (Fri) by Tet (subscriber, #5433) [Link] What a terrible article. Design goals of sendmail are to be backwardly compatible? Strange, I always thought the design goal was to send mail to anyone, anywhere. The anti-sendmail bias is just too hard to stomach. Yes, sendmail has historically had various problems. But recent (and by recent, I mean in the last decade) versions have largely fixed them.The reason I use sendmail is that it lets me do anything I want in terms of sending mail. That's a feature I haven't found in any other MTA. For various historical reasons, I have a fairly complex mail setup, and need to route mail based on arbitrary features of the message (sender address, and even sometimes content of the message body). Sendmail lets me do that. It just seems that the whole point of this article was to plug the author's preferred MTA, rather than provide an unbiased overview.
A comparison of Mail Transfer Agents - Part Two Posted Sep 1, 2006 23:03 UTC (Fri) by dwmw2 (subscriber, #2063) [Link] "The reason I use sendmail is that it lets me do anything I want in terms of sending mail. That's a feature I haven't found in any other MTA. For various historical reasons, I have a fairly complex mail setup, and need to route mail based on arbitrary features of the message (sender address, and even sometimes content of the message body). Sendmail lets me do that.Actually, that's the reason I run Exim, too. The ACLs are flexible enough that I can trigger rejection (and hence also greylisting) based on whatever criteria I like. Mail routing is similarly versatile -- for example, some of my domains route mail according to the contents of TXT records in DNS, because when I went to set up a shared database between the mail servers I realised I already had one. Implementing that in Exim was trivial, and the config files are actually readable, even when they do things as bizarre as mine.
A comparison of Mail Transfer Agents - Part Two Posted Sep 1, 2006 14:30 UTC (Fri) by robdinn (guest, #30753) [Link] Quoting From article:"Postfix is limited by design (for security considerations) and has a tiny development community (not to be confused with its large user community.) So it has a less predictable future. The license is odd (no longer used by anyone) and precludes sharing with GPL code."
Disclaimer: I am just a Mail system administrator, not a developer,
However I don't think anyone should be put off using Postfix because
As for a less predictible future, in the last 3-4 years that I have
On the License, it doesn't bother me as a user. The code belongs
A comparison of Mail Transfer Agents - Part Two Posted Sep 5, 2006 13:27 UTC (Tue) by job (guest, #670) [Link] This article was rock bottom compared to the usual LWN standards. I amnot very interested in Dans ramblings about which software he chooses. On top of that, much of the information is just wrong or distorted (Sendmail can't run milters? Please!).
I especially disliked the two remarks on how qmails code is old and
Perhaps related, I use TeX a lot which also hasn't changed in some 15
exim 1982? smail, perhaps Posted Sep 6, 2006 2:46 UTC (Wed) by chip (subscriber, #8258) [Link] Perhaps the date of 1982 is "inspired" by the origins of smail, which begat the completely new program smail 3, which begat in turn the other completely new program exim ...?
Qmail facts Posted Sep 6, 2006 20:44 UTC (Wed) by erwbgy (subscriber, #4104) [Link]
Controversial articles have to get their facts straight to be taken seriously.
A comparison of Mail Transfer Agents - Part Two Posted Sep 8, 2006 13:06 UTC (Fri) by lgb (guest, #784) [Link] "Postfix is limited" ... Where? It has got very flexible abilties to configuration, even the running daemons (master.cf) and parameters, and very nice configuration possibilities (main.cf), all documented quite well at www.postfix.org. Postfix can also use external tables: file based, LDAP based, MySQL, etc ... Postfix also have policy delegation, where it can pass available information got from SMTP transaction to external software at every step of SMTP "stages", and recently it also has got some milter support. I can't see where it's so limited, please enlighten me ... Sure, I don't know Exim very well, but the sentence that "postfix is limited" is false.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![[Sendmail logo]](/images/ns/sendmail-logo.png){kind=logo}
![[Exim logo]](/images/ns/exim-logo.png){kind=logo}