LCA: Lessons from 30 years of Sendmail [LWN.net]

LCA: Lessons from 30 years of Sendmail

Posted Feb 4, 2011 2:51 UTC (Fri) by HelloWorld (guest, #56129)
In reply to: LCA: Lessons from 30 years of Sendmail by dskoll
Parent article: LCA: Lessons from 30 years of Sendmail

I have a lot of experience with C and I like it. There's certainly something to be said for familiarity.

I don't mean to insult you, but this is exactly the mindset I mean. It's just another way of saying "because I always did it that way", really.

C is old and well-tested.

Being well-tested helps against bugs. It doesn't help one bit against design mistakes (except if you change the design, but that never happened to a meaningful extent for the C language (I guess that's what you meant with "keeping C true to the original C spirit"))

There are many C libraries and tools available, far more than for newer languages.

Any decent language has a foreign function interface for calling C functions. And at least some of them, like D, make it absolutely trivial to create bindings.

In some cases, a 20% performance hit isn't worth it. An MTA on a busy mail system is one of those cases. While it's true that most mail systems are not that busy, MTA authors rightly attempt to make their MTA as fast and reliable as possible.

No, they don't. C won't give you the fastest possible result, assembly will (portability is not an excuse, use a portable assembly language like LLVM assmbly). Yet, people seem to think that C is somehow the be-all and end-all of programming languages when it comes to performance - it's preposterous!

(Log in to post comments)

LCA: Lessons from 30 years of Sendmail

Posted Feb 4, 2011 3:19 UTC (Fri) by dskoll (subscriber, #1630) [Link]

I don't mean to insult you, but this is exactly the mindset I mean.

Meh... everyone tends to like tools he/she is familiar with. That's just human nature. Maybe some Perl or Ruby fanatics will eventually write an MTA in their language...

Any decent language has a foreign function interface for calling C functions.

But... but... then you're back in dangerous territory, no?

No, they don't. C won't give you the fastest possible result, assembly will (portability is not an excuse, use a portable assembly language like LLVM assmbly)

OK, now I know you're just arguing for the sake of argument. :) That's totally ridiculous and you know it.

Anyway... go ahead. Write a UNIX MTA in something other than C or C++ and see how it fares. The proof is in the pudding.

LCA: Lessons from 30 years of Sendmail

Posted Feb 4, 2011 12:59 UTC (Fri) by james (subscriber, #1325) [Link]

Using qpsmtpd for traps.spamassassin.org describes how SpamAssassin had been using Postfix on a donated server for their spamtraps.

Unsurprisingly, they were getting a lot of spam on that server, and postfix was having trouble keeping up. So they switched to qpsmtpd, "a flexible smtpd daemon written in Perl" which uses "Danga Interactive / Six Aparts insanely scalable event-driven asynchronous socket class".

Justin reports: "results have been great we now have a pure-perl system handling heavy volumes without breaking a sweat, certainly compared to the previous system."

LCA: Lessons from 30 years of Sendmail

Posted Feb 4, 2011 16:07 UTC (Fri) by dskoll (subscriber, #1630) [Link]

I read that article. The problem with the Postfix solution was not Postfix, but the fact that they were filtering from a procmail-invoked Perl script. That's a huge fail.

We run our Perl spam-scanner in conjunction with Sendmail, but we use the Milter interface to keep persistent Perl scanners running. The bottleneck in this system is by far the spam-scanning; Sendmail doesn't even show up on the radar.

LCA: Lessons from 30 years of Sendmail

Posted Feb 5, 2011 1:23 UTC (Sat) by HelloWorld (guest, #56129) [Link]

Meh... everyone tends to like tools he/she is familiar with. That's just human nature.

This is another way of saying "I do it that way because I always did, but that's OK because everybody else does too.".

But... but... then you're back in dangerous territory, no?

Using a C library and writing the rest of the program in another language is still better than writing it all in C.

OK, now I know you're just arguing for the sake of argument. :) That's totally ridiculous and you know it

It's not, you just didn't understand my point. You said that MTA authors attempt to make their software as efficient as possible, which, together with the fact that most MTAs are written in C, implies that C is the language you can write the most efficient software in. But this simply isn't the case: C is just one spot among many, many, many others on the efficiency-vs.-comfort curve. More specifically, it is not on the performance maximum of that curve, since that's where assembly language is. Yet, many (most?) people seem to believe that C somehow hit the perfect spot for "systems" programming. Even if that was true in 1970, people should start getting comfortable with the idea that it's not any longer.

LCA: Lessons from 30 years of Sendmail

Posted Feb 5, 2011 1:51 UTC (Sat) by mjg59 (subscriber, #23239) [Link]

Yet, despite C++ having been available for so long, there's still a distinct lack of widely-used operating systems written in it. Good system programmers are often tend to be the ones familiar with the kernel underlying their code, and that means having to have a good grasp of C regardless of what you'd prefer to code in. There's a natural selection pressure in favour of C even if there are arguably better choices.

(The first significant codebase I worked on was C++, and I've probably still written more Perl than I have C)

LCA: Lessons from 30 years of Sendmail

Posted Feb 5, 2011 17:10 UTC (Sat) by dskoll (subscriber, #1630) [Link]

But this simply isn't the case: C is just one spot among many, many, many others on the efficiency-vs.-comfort curve. More specifically, it is not on the performance maximum of that curve, since that's where assembly language is.

It's not a smooth curve. The transition from C to assembly involves a *huge* increase in the difficulty curve with a relatively modest increase in the efficiency curve. C is a just-high-enough/just-low-enough level language to hit a sweet spot in efficiency-vs-difficulty.

But you already know this and are just arguing for the sake of it.

LCA: Lessons from 30 years of Sendmail

Posted Feb 6, 2011 1:37 UTC (Sun) by HelloWorld (guest, #56129) [Link]

> But you already know this and are just arguing for the sake of it.
No, I merely disagree with you. C doesn't sit in a sweet spot, as there are many, many useful features that could be added without compromising efficiency, like templates, modules, proper macros and many more. Anyway, discussing this with you is obviously pointless by now.

LCA: Lessons from 30 years of Sendmail

Posted Feb 4, 2011 23:45 UTC (Fri) by giraffedata (subscriber, #1954) [Link]

I find it interesting that you detract from C as an unsafe language by advocating for assembler.

He didn't advocate for assembler. It was reductio ad absurdum -- if fast is the only consideration, you would use assembler. Since you don't use assembler, fast is not the only consideration.

Since fast is not the only consideration, maybe you should consider safety. Or lower development cost (with which you can probably pay for the extra hardware you need compared to the C or assembler implementation at the same speed).

LCA: Lessons from 30 years of Sendmail

Posted Feb 5, 2011 2:07 UTC (Sat) by nybble41 (subscriber, #55106) [Link]

Actually, even if speed is your only consideration you are still likely to be better off using C rather than assembler where possible. A decent compiler can generate better-optimized code from C source than all but the best assembler programmers could write by hand.

That is not to say that there are no other relevant considerations, but other languages are not necessarily any safer or easier to develop in than C; it varies depending on the project. Sometimes there really is no other viable alternative. For example, the only other realistic systems programming language I know of is D, which (while otherwise a great language) is not as portable as C (yet), nor nearly as well established so far as libraries and tool support are concerned.